Surgical operation system with training or auxiliary function

Abstract

The invention discloses a surgical operation system with a training or auxiliary function, particularly relates to the field of intelligent surgical operation systems, and cooperatively works through five modules including multi-modal sensing, layered AI decision making, dynamic feedback, closed-loop learning and an expansion interface. MEMS pressure sensing and biological signal fusion sensing are adopted, strategic layer three-dimensional navigation and execution layer magneto-rheological damping control are combined, and an operation whole-process optimization system is constructed; a multi-source data persistent evolution model is integrated through federal learning, and precise man-machine interaction is realized by matching eye movement tracking AR and a variable stiffness touch glove. According to the system, the intraoperative risk identification accuracy is improved, the operation efficiency is improved, third-party equipment extension and doctor skill quantitative evaluation are supported, and the operation safety and the training standardization level are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent surgical systems, and particularly to a surgical system with training or assisting functions. Background Art

[0002] Traditional surgical operations rely on doctors' experience and manual operations, and have limitations such as low operation accuracy and long learning curves. In recent years, robotic-assisted surgical systems (such as the da Vinci surgical system) have significantly improved the precision of minimally invasive surgeries through robotic arm manipulation and 3D visual feedback. At the same time, virtual reality (VR) surgical simulators (such as SurgicalTheater) provide doctors with a risk-free training environment. However, the existing technologies still have the following core defects:

[0003] (1) The training and intraoperative assisting functions are separated, and data cannot flow in a closed loop; (2) The human-computer interaction feedback dimension is single, and the situation awareness ability is insufficient; (3) The AI-assisted decision-making is rigid and lacks personalized adaptation ability.

[0004] Therefore, we propose a surgical system with training or assisting functions. Summary of the Invention

[0005] The main objective of the present invention is to provide a surgical system with training or assisting functions, which can effectively solve the problems in the background art through a training-assisting closed-loop architecture, multi-modal perception fusion, hierarchical AI decision-making, and modular expansion design.

[0006] To achieve the above objective, the technical solution adopted by the present invention is as follows:

[0007] A surgical system with training or assisting functions, comprising:

[0008] A multi-modal perception and data acquisition module, configured to collect surgical environment data, doctor operation data, and patient physiological data in real time;

[0009] An intelligent decision-making and hierarchical control module, receiving the multi-modal perception and data acquisition module, and generating hierarchical assisting instructions through a strategic planning layer, a tactical decision-making layer, and an execution layer control unit;

[0010] A multi-modal feedback and interaction module, configured to provide dynamic tactile feedback, enhanced visual feedback, and biological adaptation information to the doctor according to the hierarchical assisting instructions;

[0011] A training-assisting closed-loop learning module, feeding back the doctor operation data and patient physiological data collected during the surgery to the virtual training interface, and optimizing the AI model of the intelligent decision-making and hierarchical control module based on an incremental learning engine;

[0012] Modular expansion interface, providing a hardware magnetic connector and a software open-source framework, allowing third-party devices and algorithms to access the system;

[0013] Among them, the data of the multi-modal feedback and interaction module is dynamically coupled with the hierarchical AI decision instruction through the timestamp synchronization protocol, and the incremental learning engine of the training-assisted closed-loop learning module personalized updates the AI decision weight according to the doctor's actual surgical performance.

[0014] Preferably, the multi-modal perception and data acquisition module includes:

[0015] A micro pressure sensor array, which is integrated at the tip of the surgical instrument and is used to quantify the tissue compression deformation parameters;

[0016] An electromyography bracelet and a nine-axis inertial sensor, which are used to monitor the doctor's hand muscle activity and tremor frequency;

[0017] An infrared spectrometer and an ECG patch, which are used to synchronize the tissue oxygen saturation and the patient's electrocardiogram signal;

[0018] It also includes, after obtaining the above sensor data, fusing through the Kalman filter algorithm to generate a multi-modal feature vector of the surgical scene.

[0019] Preferably, the intelligent decision-making and hierarchical control module includes:

[0020] Strategic planning layer: Generate a 3D surgical path plan based on the patient's preoperative medical images and anatomical variation database, and overlay the key structure navigation atlas through the AR transparent display screen;

[0021] Tactical decision-making layer: Adopt a deep reinforcement learning model to dynamically adjust the instrument movement trajectory according to the real-time tissue mechanics data, and output a priority list of operation suggestions;

[0022] Execution layer control unit: It includes a safety threshold generation sub-module, which is based on the patient's individual anatomical structure feature database and intraoperative real-time physiological parameters, and calculates the maximum allowable deformation threshold of the organ tissue through finite element simulation; and applies a micro torque compensation to the surgical instrument through a magnetorheological fluid damper to suppress hand tremors and limit the operation amplitude exceeding the maximum allowable deformation threshold.

[0023] Preferably, the multi-modal feedback and interaction module includes:

[0024] A variable stiffness tactile glove, which dynamically adjusts the resistance according to the tissue elasticity parameters output by the tactical decision-making layer;

[0025] An eye movement tracking-driven AR information density adaptive system, which automatically simplifies or enhances visual annotations based on the doctor's experience level;

[0026] A linear resonance actuator that triggers a high-frequency vibration warning when AI detects a risk of accidental blood vessel resection.

[0027] Preferably, the training-assisted closed-loop learning module includes:

[0028] An incremental learning engine under the federated learning framework that aggregates desensitized data from multiple hospitals to update the weights of the AI model;

[0029] An entropy value quantization model that evaluates a doctor's operating skills based on the complexity of the instrument movement trajectory and the risk aversion index;

[0030] A detachable force feedback handle and an open-source surgical simulation software that support doctors to train specific scenario skills in a non-surgical environment.

[0031] Preferably, the low-cost modular expansion interface includes:

[0032] A magnetic multi-instrument compatible mechanical coupling structure that supports the rapid access of multi-degree-of-freedom robotic arm instruments;

[0033] A blockchain-based data sharing incentive protocol where hospitals can exchange desensitized data for the right to use advanced AI functions;

[0034] A USB-C universal interface that provides power supply and data transmission for peripherals.

[0035] Preferably, the collaborative logic between the intelligent decision-making and hierarchical control module and the multi-modal feedback and interaction module further includes:

[0036] When the decision confidence level of the tactical decision-making layer is lower than the preset threshold, the warning label display intensity of the enhanced visual feedback unit is forcibly increased, and the operation authority of the execution layer control unit is restricted;

[0037] According to the doctor's historical misoperation type database, dynamically adjust the vibration intensity and resistance threshold of the dynamic tactile feedback unit;

[0038] In the training mode, quantify the difference between the recommended operation path generated by the tactical decision-making layer and the doctor's actual operation path as a skill evaluation parameter, and generate a visual training report.

[0039] A coupling method for multi-modal perception data and hierarchical AI decision-making, applied to the above-mentioned surgical system with training or assistance functions, includes the following steps:

[0040] Step S1: Align tissue deformation data, doctor's hand movement data, and patient's physiological signals through a timestamp synchronization protocol;

[0041] Step S2: Input the fused multi-modal data into the tactical decision-making layer to generate instrument movement trajectory correction parameters;

[0042] Step S3: Verify the legality of the correction parameters according to the global path planning of the strategic planning layer and the safety threshold of the execution layer;

[0043] Step S4: If the correction parameters are legal, guide the doctor to perform the operation through the tactile feedback unit and the AR display unit;

[0044] Step S5: Record the difference between the actual operation data and the AI suggestions, and update the model weights of the incremental learning engine.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. Through the Kalman filter fusion of multi-modal perception data and the hierarchical AI decision-making architecture, the present invention achieves breakthroughs in the accuracy and real-time performance of surgical assistance. The 3D path planning of the strategic planning layer combined with the dynamic trajectory correction of reinforcement learning in the tactical layer improves the operation accuracy; the magnetorheological fluid damper in the execution layer can eliminate hand tremors with a frequency > 5Hz, reducing the risk of accidental resection. The multi-modal feedback system realizes millisecond-level coupled feedback of touch-vision-biological information through the timestamp synchronization protocol, improving the doctor's decision-making response speed. Especially in delicate operations such as vascular anastomosis, the AR enhanced display can improve the recognition accuracy of key structures.

[0047] 2. The present invention creates a new paradigm for the continuous optimization of surgical skills through the training-assisted closed-loop learning mechanism. The incremental learning engine under the federated learning framework enables the AI model to iteratively evolve, and constructs a personalized training path in cooperation with the entropy quantization model. The modular expansion interface realizes on-demand expansion of system functions through magnetic coupling and open-source architecture, making high-end devices and primary hospital instruments compatible and shared, and reducing the equipment transformation cost. Description of the Drawings

[0048] Figure 1 It is a flowchart of the coupling method steps of multi-modal perception data and hierarchical AI decision-making of the present invention. Detailed Embodiments

[0049] In order to make the technical means, creative features, achieved purposes and effects of the present invention easy to understand, the present invention will be further described below in conjunction with specific embodiments.

[0050] The present invention discloses an intelligent surgical training assistance system, which realizes the intelligent optimization of the entire surgical process through the closed-loop cooperation of five functional modules. The system architecture significantly improves the safety and training efficiency of surgical operations through the organic cooperation of multi-dimensional perception, hierarchical decision-making, dynamic feedback and autonomous learning mechanisms. The specific module composition is as follows:

[0051] 1. Multi-modal Perception and Data Acquisition Module

[0052] This module constructs a three-dimensional perception network, which works in collaboration with a micro pressure sensor array (manufactured by MEMS process, range 0 - 50N / accuracy ±0.1N) and a bio-signal acquisition device.

[0053] Among them:

[0054] The tip of the intelligent surgical forceps integrates a pressure sensing unit, which can monitor the tissue compression deformation in real time (a third-level warning is triggered when the liver parenchyma compression rate > 20%);

[0055] In practice, the pressure sensor integrated in the surgical forceps directly quantifies the tissue compression deformation. The dynamic ultrasound detection of liver elasticity belongs to the application scenario description (such as liver cirrhosis diagnosis), which is used to demonstrate the versatility of the sensor data.

[0056] A nine-axis inertial sensor (sampling at 100Hz) combined with an EMG bracelet (sampling at 1kHz / 8 action mode recognition) constructs a doctor operation feature model;

[0057] An infrared spectrometer and an ECG patch synchronously obtain tissue blood oxygen and patient ECG parameters;

[0058] Through Kalman filtering, multi-source data spatio-temporal alignment is achieved, and a fusion feature vector containing dimensions such as pressure, deformation rate, and EMG intensity is generated. Compared with single-modal solutions such as the Da Vinci system, this design improves the risk recognition accuracy by 37.6% (clinical verification data) through cross-validation of mechanics and bio-signals.

[0059] 2. Intelligent Decision-making and Hierarchical Control Module;

[0060] Adopt a three-layer architecture of strategy - tactics - execution to achieve precise assistance:

[0061] (1) Strategic Planning Layer

[0062] Based on the patient's DICOM images and a database of 100,000 anatomical variations, a three-dimensional navigation map is generated by improving the U-Net algorithm (the key structure segmentation accuracy reaches 0.8mm). In the HoloLens2 AR interface, a semi-transparent path guide is superimposed (depth perception error < 1mm), and the vascular and nerve dense areas are dynamically marked.

[0063] (2) Tactical Decision-making Layer

[0064] Deploy a PPO reinforcement learning model (reward function = efficiency × safety factor - damage penalty), and generate an instrument motion correction amount according to real-time tissue mechanics data. For example, when tissue adhesion is detected, the optimal deflection angle (such as a 45° → 30° adjustment suggestion) is calculated immediately, and the auxiliary permission control is triggered through a confidence threshold (Monte Carlo Dropout variance > 0.3).

[0065] (3) Execution Control Layer

[0066] Integrate a magnetorheological fluid damper (10 ms response / 5 N·m torque) with a finite element simulation module. When the instrument speed exceeds a preset threshold (such as 20 mm / s for laparoscopy), the speed is forced to decrease by a reverse torque. The safety mechanism is based on a patient-specific anatomical model, which calculates the maximum organ deformation threshold in real time and enforces hard constraints.

[0067] 3. Multi-modal feedback and interaction module

[0068] Establish a multi-channel feedback mechanism to achieve enhanced "eye-hand-brain" coordination:

[0069] Tactile dimension: The shape memory alloy composite glove (0.1 - 5 N / mm stiffness adjustment) dynamically adjusts the resistance according to the tissue elastic modulus. For example, when encountering a fibrotic liver (>50 kPa), the touch sensation automatically increases to 3 N / mm.

[0070] Visual dimension: The eye-tracking AR system adaptively adjusts the information density according to the doctor's experience level (novice / expert mode). In the novice mode, the blood vessel names and a 3.2 mm safety distance annotation are displayed, while in the expert mode, only the warning of the red high-risk area is retained.

[0071] Among them, the data basis for dividing the doctor's experience level is as follows: In the operation data acquisition module, the electromyography bracelet and the nine-axis inertial sensor are used to monitor the doctor's hand muscle activities and tremor frequencies, and the "and" LSTM model analyzes the doctor's past 100 operation records to judge the doctor's experience level according to this record.

[0072] Warning dimension: The linear actuator triggers an 800 Hz vibration warning when the risk of accidental blood vessel cutting > 85%, and the vibration intensity is dynamically adjusted according to the historical error records (LSTM analyzes 100 operation records, and the intensity for those with frequent accidental cutting is 0.5 G → 1.2 G);

[0073] Specifically, the data basis for judging that the risk of accidental blood vessel cutting > 85% is as follows: By analyzing 100 historical operation records through LSTM, the correlation between accidental cutting events and operation parameters (such as instrument speed, pressure value) is statistically analyzed.

[0074] In the specific detection process, the data support for the risk of accidental blood vessel cutting is reflected in the following parts:

[0075] Data input: The pressure sensor monitors tissue deformation (such as liver parenchyma compression rate), the nine-axis inertial sensor captures the instrument movement trajectory, combined with preoperative imaging data (such as three-dimensional blood vessel positioning).

[0076] Model output: The risk value is the predicted probability of the AI model (tactical decision-making layer) for the current operation parameters, and the threshold of 85% may be based on the sensitivity and specificity balance point determined by ROC curve analysis.

[0077] 4. Training - Assisted Closed - Loop Learning Module

[0078] Build a positive cycle of "Intraoperative Data - Virtual Training - AI Evolution":

[0079] Federated learning framework: Use the Secure Aggregation protocol to achieve data collaboration among hundreds of hospitals, and update the global model weights every 100 surgeries;

[0080] Skill assessment system: Calculate the operation deviation through the Hausdorff distance of the instrument trajectory, and generate a special training course (such as vascular anastomosis simulation) when the difference > 2mm;

[0081] Open - source training platform: Equipped with a 5N force - feedback handle (USB - C / multi - platform compatible) and surgical simulation software developed by Unity, supporting the import of STL organ models and custom training scenarios.

[0082] 5. Modular Expansion Interface

[0083] Provide a hardware magnetic connector and a software open - source framework, allowing third - party devices and algorithms to access the system.

[0084] For example, it includes the following interfaces:

[0085] Magnetic connector: Structural design: Neodymium iron boron permanent magnet array, adsorption force > 20N, supporting 360° rotation for alignment; Electrical connection: Pogopin contacts transmit data and power, compatible with the CAN bus protocol of da Vinci instruments.

[0086] Blockchain data sharing: Smart contract logic: For every 1TB of valid surgical data uploaded by a hospital, 10 points can be obtained to exchange for advanced AI functions; Data verification: Verify the data authenticity through zero - knowledge proof without disclosing the original content.

[0087] USB - C universal interface: Power supply specification: Support the PD3.0 protocol, with a maximum power output of 100W; Data transmission: USB3.2 Gen2, rate 10Gbps, capable of synchronously transmitting 4 - channel sensor data streams.

[0088] Furthermore, the collaborative logic between the intelligent decision - making and hierarchical control module and the multi - modal feedback and interaction module in the present invention further includes:

[0089] When the decision confidence level of the tactical decision - making layer is lower than the preset threshold, forcibly enhance the display intensity of the warning sign of the enhanced visual feedback unit and limit the operation authority of the execution - layer control unit;

[0090] Confidence calculation: Based on the Monte Carlo Dropout method, when the predicted variance of the tactical decision-making layer > 0.3, it is determined as low confidence; Response strategy: Force the AR to display alternative solutions (such as switching blunt / sharp separation tools), and limit the instrument movement speed to 50% of the normal value;

[0091] According to the doctor's historical misoperation type database, dynamically adjust the vibration intensity and resistance threshold of the dynamic tactile feedback unit; Feedback intensity dynamic adjustment:

[0092] Historical misoperation type recognition: Use the LSTM model to analyze the doctor's past 100 operation records;

[0093] Tactile gain rule: For doctors who frequently have vascular missection, increase the vibration intensity from the default 0.5G to 1.2G.

[0094] In the training mode, quantify the difference between the recommended operation path generated by the tactical decision-making layer and the doctor's actual operation path as a skill evaluation parameter, and generate a visual training report.

[0095] Difference index: Calculate the Hausdorff distance between the AI-recommended path and the actual path; Report generation: Automatically mark the operation stages with a difference > 2mm, and generate targeted training courses (such as special vascular anastomosis practice).

[0096] Based on the above, the present invention discloses a coupling method of multi-modal perception data and hierarchical AI decision-making, constructs a complete technical chain from multi-modal perception to hierarchical decision-making, dynamic feedback and closed-loop learning, and the core innovation lies in the dynamic coupling logic of the hierarchical AI decision-making mechanism and multi-modal interaction. Through the collaborative optimization of data fusion, intelligent decision-making and real-time feedback, the double improvement of surgical precision and safety is achieved.

[0097] As Figure 1 shown, a coupling method of multi-modal perception data and hierarchical AI decision-making includes the following steps:

[0098] Step S1: Align the tissue deformation data, doctor's hand movement data and patient's physiological signals through the timestamp synchronization protocol;

[0099] Data acquisition dimension:

[0100] Tissue mechanics parameters: The micro pressure sensor array integrated at the tip of the instrument monitors the tissue deformation characteristics in real time;

[0101] Operation dynamic parameters: The nine-axis inertial sensor captures the instrument movement trajectory at a high sampling rate, and combines the electromyogram signal to identify the hand movement pattern;

[0102] Physiological state parameters: The infrared spectroscopy and electrocardiogram monitoring devices synchronously obtain the patient's blood oxygen saturation and cardiac electrophysiological indexes.

[0103] Synchronization mechanism:

[0104] Implement high-precision alignment of multi-source data based on the timestamp protocol to eliminate time drift and spatial deviation across sensors.

[0105] Step S2: Input the fused multi-modal data into the tactical decision-making layer to generate instrument motion trajectory correction parameters;

[0106] Data processing objectives:

[0107] Filter environmental noise and operation interference signals;

[0108] Extract cross-modal correlation features and generate data vectors in a unified spatio-temporal coordinate system;

[0109] Ensure data accuracy, for example, control the instrument positioning error within the sub-millimeter level and the pressure measurement accuracy is better than the standard threshold.

[0110] Technical implementation:

[0111] Adopt the Kalman filtering algorithm as the preferred embodiment, and use its output feature vector as the input benchmark for subsequent hierarchical decision-making.

[0112] Step S3: Verify the legality of the correction parameters according to the global path planning of the strategic planning layer and the safety threshold of the execution layer;

[0113] Strategic planning layer:

[0114] Generate a global navigation path by combining preoperative medical images and real-time feature vectors, and define the safe operation boundaries of key anatomical structures.

[0115] Tactical decision-making layer:

[0116] The deep reinforcement learning model analyzes the real-time data stream, dynamically generates instrument motion correction parameters, and evaluates the decision confidence.

[0117] Execution control layer:

[0118] Verify the legality of the correction parameters based on the individualized safety threshold, and implement operation permission constraints through magnetorheological damping technology.

[0119] Step S4: If the correction parameters are legal, guide the doctor to perform the operation through the tactile feedback unit and the AR display unit;

[0120] Tactile warning module:

[0121] Trigger gradient vibration feedback according to the risk level of accidental resection, and the frequency and intensity are dynamically adjusted according to the risk index.

[0122] Visual enhancement module:

[0123] Adaptive adjustment of AR information density, showing all navigation markers in the novice mode and only retaining the warnings for high-risk areas in the expert mode.

[0124] Step S5: Record the differences between the actual operation data and the AI suggestions, and update the model weights of the incremental learning engine;

[0125] Model iteration mechanism:

[0126] Aggregate multi-center surgical data through the federated learning framework, and regularly update the global AI model to ensure continuous optimization of the decision-making logic.

[0127] Personalized adaptation mechanism:

[0128] Generate customized training courses based on the doctor's historical operation data, such as special simulation training for vascular anastomosis deviation.

[0129] Kalman filtering, as a well-known data processing method in the art, is used to illustrate the generation method of multi-modal feature vectors, and its technical effects are decoupled from the core innovation points through modular interface design; the present invention establishes a new paradigm for intelligent surgical assistance systems through the deep cooperation of hierarchical decision-making and multi-modal feedback.

[0130] The present invention is further disclosed below in conjunction with specific embodiments:

[0131] The specific scenario of this embodiment is as follows: The patient is diagnosed with a malignant tumor in the right lobe of the liver and needs to undergo laparoscopic right hepatectomy. Traditional surgeries have the risks of accidental cutting of the hepatic vein and the problem of controlling liver parenchymal damage. This system achieves precise assistance through the following steps:

[0132] I. Preoperative preparation and data loading

[0133] Medical image modeling:

[0134] Import the patient's CT / MRI data to construct a 3D model of the liver (including hepatic veins, portal vein branches, and tumor boundaries);

[0135] The strategic planning layer generates a safe resection path and marks the hepatic vein protection area (safety distance ≥ 5 mm).

[0136] Equipment configuration:

[0137] The intelligent surgical forceps are integrated with a micro pressure sensor (range 0 - 50 N, accuracy ±0.1 N); the doctor wears a nine-axis inertial sensor glove and an AR transparent eyepiece; ECG monitoring patches and infrared spectroscopy probes are attached to the patient's abdomen.

[0138] II. Intraoperative real-time data fusion

[0139] Multi-modal data acquisition:

[0140] Organizational mechanics data: The surgical forceps pressure sensor monitors the liver parenchyma compression rate (real-time calculation of the deformation ratio);

[0141] Operating dynamic data: The inertial sensor captures the movement trajectory of the instrument (spatial positioning accuracy ±0.3 mm);

[0142] Physiological parameters: The infrared spectrometer monitors the blood oxygen saturation of the liver section, and the ECG synchronizes the electrocardiogram signal.

[0143] Kalman filter processing:

[0144] Aligning spatio-temporal data: Mapping the instrument position, pressure value, and blood oxygen parameters to a unified coordinate system;

[0145] Noise filtering: Eliminating the interference signal of the laparoscopic electrosurgical knife (>10 kHz high-frequency noise);

[0146] Outputting the fused feature vector: The position error of the instrument tip <0.2 mm, and the calculation error of the liver parenchyma compression rate <2%.

[0147] III. Hierarchical AI Decision-making and Feedback

[0148] Verification at the strategic planning level:

[0149] Comparing the real-time instrument position with the preset safe path, when the instrument deviates from the planned path >3 mm, the AR eyepiece projects a red deviation warning frame.

[0150] Correction at the tactical decision-making level:

[0151] The deep reinforcement learning model analyzes the trend of decreasing blood oxygen in the liver section (when it decreases >5% per minute), and recommends adjusting the cutting angle;

[0152] Predicting the risk of accidental cutting of the hepatic vein: When the distance between the instrument and the hepatic vein <3 mm and the pressure value >15 N, the risk value rises to 90%.

[0153] Execution control and feedback:

[0154] Tactile feedback: When the risk of accidental cutting >85%, the surgical forceps handle triggers a high-frequency vibration of 800 Hz (the intensity is dynamically adjusted according to the doctor's historical error rate);

[0155] Visual feedback: The AR eyepiece enhances the display of the hepatic vein contour (a 3.2 mm safety distance annotation is superimposed in the novice mode);

[0156] Forced intervention: When the liver parenchyma compression rate >20% lasts for 2 seconds, the magnetorheological damper locks the moving direction of the instrument.

[0157] IV. Examples of Key Operation Nodes

[0158] Scenario: Separating the branches of the hepatic vein

[0159] Risk Warning:

[0160] The tip of the instrument was close to the hepatic vein (distance 2.5 mm), and the pressure sensor detected a local pressure of 18 N;

[0161] The system calculates the risk of miscutting to be 92%, triggering: the AR eyepiece projects a pulse flashing warning light band; the surgical forceps handle produces a 1.0G intensity vibration; the tactical decision-making layer recommends switching to a blunt separation tool.

[0162] Doctor's response: The angle of the instrument was adjusted according to the AR prompt, and the pressure value was reduced to 12N; the risk value of miscutting dropped to 65%, and the system lifted the forced intervention.

[0163] Data recording: record the operation trajectory deviation (Hausdorff distance 1.8mm), AI suggestion adoption rate (100% adoption rate for this operation); update the doctor's personal skill profile (vascular separation operation score +15%).

[0164] 5. Postoperative Closed-Loop Learning

[0165] Model iteration:

[0166] The surgical data (including 10 risk warning events) were uploaded encrypted via federated learning;

[0167] Global AI model update: Optimized the prediction threshold for the risk of mis-resection of the hepatic vein (adjusted from 85% to 88%).

[0168] Personalized training: Generate special training courses for vascular separation based on intraoperative records: The variability of hepatic veins in the virtual scene is increased by 30%; the force feedback handle simulates real tissue elasticity (5N-20N gradient resistance).

[0169] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A surgical system with training or auxiliary functions, characterized in that: include: Multimodal perception and data acquisition module, used to collect surgical environment data, doctor operation data and patient physiological data in real time; An intelligent decision-making and hierarchical control module receives the multimodal perception and data acquisition module and generates hierarchical auxiliary instructions through the strategic planning layer, tactical decision layer and execution layer control unit; a multimodal feedback and interaction module, which provides the physician with dynamic tactile feedback, enhanced visual feedback, and biofit information according to the hierarchical auxiliary instructions; The training-assisted closed-loop learning module feeds back the doctor's operation data and patient's physiological data collected during the operation to the virtual training interface, and optimizes the AI ​​model of the intelligent decision-making and hierarchical control module based on the incremental learning engine; Modular expansion interface, providing hardware magnetic connector and software open source framework, allowing third-party devices and algorithms to access the system; Among them, the data of the multimodal feedback and interaction module are dynamically coupled with the hierarchical AI decision instructions through a timestamp synchronization protocol, and the incremental learning engine of the training-assisted closed-loop learning module personalizes the AI ​​decision weights according to the doctor's actual surgical performance.

2. A surgical system with training or auxiliary functions according to claim 1, characterized in that: The multimodal sensing and data acquisition module includes: An array of micro pressure sensors integrated into the tip of surgical instruments to quantify tissue compression deformation parameters; The myoelectric bracelet and nine-axis inertial sensor are used to monitor the doctor's hand muscle activity and tremor frequency; Infrared spectrometer and ECG patch to synchronize tissue oxygen saturation with the patient’s ECG signal; It also includes generating a multimodal feature vector of the surgical scene by fusing the above sensor data through a Kalman filter algorithm after acquiring the above sensor data.

3. A surgical system with training or auxiliary functions according to claim 1, characterized in that: The intelligent decision-making and hierarchical control module includes: Strategic planning layer: Generate 3D surgical path planning based on the patient's preoperative medical images and anatomical variation database, and superimpose key structure navigation maps through AR transparent display screen; Tactical decision-making layer: Using deep reinforcement learning models, the movement trajectory of the equipment is dynamically adjusted according to real-time tissue mechanics data, and a priority list of operation suggestions is output; Execution layer control unit: It includes a safety threshold generation submodule, which calculates the maximum allowable deformation threshold of organ tissue through finite element simulation based on the patient's individual anatomical structure feature database and real-time physiological parameters during surgery; and applies micro-torque compensation to surgical instruments through magnetorheological fluid dampers to suppress hand tremors and limit the operation amplitude that exceeds the maximum allowable deformation threshold.

4. A surgical system with training or auxiliary functions according to claim 3, characterized in that: The multimodal feedback and interaction module includes: Variable stiffness tactile gloves that dynamically adjust resistance based on tissue elasticity parameters output by the tactical decision layer; An eye-tracking-driven AR information density adaptive system that automatically simplifies or enhances visual annotations based on the physician’s experience level; The linear resonant actuator triggers a high-frequency vibration warning when the AI ​​detects the risk of mis-cutting a blood vessel.

5. A surgical system with training or assistance function according to claim 1, wherein the training-assistance closed-loop learning module comprises: The incremental learning engine under the federated learning framework aggregates desensitized data from multiple hospitals to update the AI ​​model weights; The entropy value quantification model evaluates the doctor's operating skills based on the complexity of the instrument's motion trajectory and the risk aversion index; The detachable force feedback handle and open source surgical simulation software support doctors to train scenario-specific skills in non-surgical environments.

6. A surgical system with training or auxiliary functions according to claim 1, characterized in that: The modular expansion interface includes: The magnetic multi-instrument compatible mechanical coupling structure supports rapid access to multi-DOF robotic arm instruments; A blockchain-based data sharing incentive agreement where hospitals contribute desensitized data in exchange for access to advanced AI features; The USB-C universal interface provides power supply and data transmission for peripherals.

7. A surgical system with training or auxiliary functions according to claim 1, characterized in that: The collaborative logic of the intelligent decision-making and hierarchical control module and the multimodal feedback and interaction module also includes: When the decision confidence of the tactical decision layer is lower than a preset threshold, the warning mark display intensity of the enhanced visual feedback unit is forcibly enhanced, and the operation authority of the execution layer control unit is restricted; Dynamically adjust the vibration intensity and resistance threshold of the dynamic tactile feedback unit according to the doctor's historical wrong operation type database; In the training mode, the difference between the recommended operation path generated by the tactical decision layer and the doctor's actual operation path is quantified as skill evaluation parameters, and a visual training report is generated.

8. A method for coupling multimodal perception data with hierarchical AI decision making, applied to a surgical system with training or auxiliary functions as described in any one of claims 1 to 7, characterized in that: The following steps are involved: Step S1: aligning tissue deformation data, doctor's hand motion data and patient's physiological signals through a timestamp synchronization protocol; Step S2: Input the fused multimodal data into the tactical decision layer to generate the equipment motion trajectory correction parameters; Step S3: Verify the legitimacy of the correction parameters according to the global path planning of the strategic planning layer and the security threshold of the execution layer; Step S4: If the correction parameters are legal, the doctor is guided to perform the operation through the tactile feedback unit and the AR display unit; Step S5: Record the difference between the actual operation data and the AI ​​recommendation, and update the model weight of the incremental learning engine.

Images