Surgical operation system with training or auxiliary function

Abstract

The invention discloses a surgical operation system with a training or auxiliary function, particularly relates to the field of intelligent surgical operation systems, and cooperatively works through five modules including multi-modal sensing, layered AI decision making, dynamic feedback, closed-loop learning and an expansion interface. MEMS pressure sensing and biological signal fusion sensing are adopted, strategic layer three-dimensional navigation and execution layer magneto-rheological damping control are combined, and an operation whole-process optimization system is constructed; a multi-source data persistent evolution model is integrated through federal learning, and precise man-machine interaction is realized by matching eye movement tracking AR and a variable stiffness touch glove. According to the system, the intraoperative risk identification accuracy is improved, the operation efficiency is improved, third-party equipment extension and doctor skill quantitative evaluation are supported, and the operation safety and the training standardization level are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent surgical systems, and in particular to a surgical system with training or auxiliary functions. Background Art

[0002] Traditional surgical procedures rely on the surgeon's experience and manual manipulation, resulting in limitations such as low precision and a long learning curve. In recent years, robotic-assisted surgical systems (such as the da Vinci surgical system) have significantly improved the precision of minimally invasive surgery through robotic arm manipulation and 3D visual feedback. Furthermore, virtual reality (VR) surgical simulators (such as the Surgical Theater) provide a risk-free training environment for surgeons. However, existing technologies still suffer from the following core flaws: (1) Training and intraoperative assistance functions are separated, and data cannot flow in a closed loop; (2) The human-computer interaction feedback dimension is single, and the situational awareness ability is insufficient; (3) AI-assisted decision-making is rigid and lacks personalized adaptation capabilities. Therefore, we propose a surgical system with training or assistance functions. Summary of the Invention

[0003] The main purpose of this invention is to provide a surgical system with training or assistance functions. Through an innovative training-assistance closed-loop architecture, multimodal perception fusion, hierarchical AI decision-making and modular expansion design, it can effectively solve the problems of insufficient assistance accuracy, disconnected learning feedback and poor system compatibility in existing surgical systems, and achieve the goal of continuously improving the safety, accuracy and skills of surgical operations.

[0004] To achieve the above object, the technical solution adopted by the present invention is: A surgical system with training or auxiliary functions, including the following core modules: Multimodal perception and data acquisition module: used to collect multi-dimensional surgical environment data, doctor operation data and patient key physiological data in real time and with high precision, building a comprehensive intraoperative perception foundation.

[0005] Intelligent decision-making and hierarchical control module: Receives and processes the fused data stream from the multimodal perception and data acquisition module, and generates accurate, hierarchical real-time auxiliary instructions through its internal collaborative strategic planning layer, tactical decision-making layer and execution layer control unit.

[0006] Multimodal feedback and interaction module: Based on the hierarchical auxiliary instructions, dynamic force feedback, augmented reality visual guidance and key biofit information are provided to the surgeon synchronously and multi-channel to optimize the doctor's decision-making and operation.

[0007] Training-assisted closed-loop learning module: Builds a data closed loop, feeds back the doctor's operation data and patient physiological data actually collected during the operation to the virtual training interface, and continuously optimizes the AI ​​model performance of the intelligent decision-making and hierarchical control module based on the built-in incremental learning engine.

[0008] Modular expansion interface: Provides flexible hardware magnetic connectors and an open software open source framework, allowing third-party surgical instruments, equipment, and advanced algorithms to be seamlessly integrated into the system, enabling on-demand functional expansion.

[0009] Among them, the data transmission and presentation of the multimodal feedback and interaction module are synchronized through a high-precision timestamp protocol, ensuring dynamic coupling with hierarchical AI decision-making instructions and achieving millisecond-level response; at the same time, the incremental learning engine of the training-assisted closed-loop learning module can personalize the update of the weight parameters of the AI ​​decision-making model based on the performance data of individual doctors in actual surgery.

[0010] Preferably, the multimodal perception and data acquisition module specifically includes: A micro pressure sensor array integrated into the tip of a surgical instrument to quantify tissue compression deformation parameters; The myoelectric bracelet worn on the doctor's wrist and the nine-axis inertial sensor integrated into the instrument handle are used to monitor the doctor's hand muscle activity, movement trajectory and tremor frequency with high precision; An infrared spectrometer deployed in the surgical field and an ECG patch attached to the patient are used to synchronously monitor tissue oxygen saturation and the patient's electrocardiogram (ECG) signals. After acquiring the above-mentioned multi-source heterogeneous sensor data, real-time fusion and noise reduction processing are performed through the Kalman filter algorithm to generate a multimodal feature vector representing the current state of the surgical scene.

[0011] Preferably, the intelligent decision-making and hierarchical control module specifically includes: The strategic planning layer generates personalized 3D surgical path planning based on the patient's preoperative high-resolution medical imaging data and anatomical variation database, and overlays navigation maps of key anatomical structures on a transparent augmented reality (AR) display. Tactical decision-making layer: Using a deep reinforcement learning (DRL) model, the system dynamically calculates and outputs instrument motion trajectory fine-tuning parameters based on real-time tissue mechanics data (such as deformation and stress), while also generating a priority list of recommended operations for the doctor's reference. Execution layer control unit: Contains a safety threshold generation submodule. This submodule calculates the maximum allowable deformation threshold of organ tissue in the current state through real-time finite element simulation based on the patient's individualized anatomical structure feature database and real-time physiological parameters during surgery (such as blood pressure and blood oxygen). It also drives the magnetorheological fluid damper to apply precise micro-torque compensation to the surgical instrument, effectively suppressing hand tremors with a frequency greater than 5Hz, while strictly limiting any operation amplitude that may exceed the maximum allowable deformation threshold to ensure the safety boundary of the operation.

[0012] Preferably, the multimodal feedback and interaction module specifically includes: Variable stiffness tactile feedback gloves can dynamically adjust the tactile resistance felt by doctors during operation based on the target tissue elasticity parameters output in real time by the tactical decision-making layer, simulating the touch of real tissue; An eye-tracking-driven AR information density adaptive system automatically simplifies or enhances visual annotation information based on the system's identified physician experience level to avoid information overload. The linear resonant actuator (LRA) integrated into surgical instruments or doctor's gloves triggers a high-frequency vibration warning signal in a specific pattern when the AI ​​model detects the risk of mis-cutting blood vessels or damage to critical structures.

[0013] Preferably, the training-assisted closed-loop learning module specifically includes: An incremental learning engine built on a federated learning framework can securely aggregate desensitized surgical data from multiple hospitals while protecting data privacy, and is used to update and optimize the core AI model weights. The entropy quantification model objectively evaluates the doctor's real-time operation skills by analyzing indicators such as the complexity, smoothness, and risk aversion index of the instrument's movement trajectory during the doctor's operation; The detachable force feedback handle is interfaced with open-source surgical simulation software, allowing doctors to conduct targeted skill training and assessment for specific surgical scenarios (such as vascular anastomosis and tissue dissection) in a non-surgical environment through the connected system.

[0014] Preferably, the modular expansion interface specifically includes: The magnetic multi-device compatible mechanical coupling structure supports the rapid access and hot swapping of multi-degree-of-freedom robotic arm auxiliary devices or new sensor modules; A data sharing incentive agreement based on blockchain technology encourages hospitals to contribute compliant, anonymized clinical data in exchange for access to advanced AI-assisted functions within the system; The standardized USB-C universal physical interface provides stable power supply and high-speed data transmission channels for connected peripherals.

[0015] Preferably, the collaborative logic of the intelligent decision-making and hierarchical control module and the multimodal feedback and interaction module further includes: When the confidence level of the tactical decision-making layer in the current decision is lower than the preset safety threshold, the system automatically and forcibly increases the display intensity of the warning mark in the enhanced visual feedback unit (such as flashing, highlighting, and amplifying), and temporarily restricts the operating authority of the execution layer control unit (such as locking the equipment in a safe area); Dynamically adjust the vibration intensity level and resistance threshold setting of the dynamic tactile feedback unit based on the doctor's historical erroneous operation type database recorded by the system to provide personalized risk warnings; In a dedicated training mode, the system compares the recommended operation path generated by the tactical decision-making layer with the doctor's actual operation path in real time, quantifies the differences into specific skill assessment parameters (such as offset distance and time delay), and automatically generates a visual training report containing improvement suggestions.

[0016] A method for coupling multimodal perception data with hierarchical AI decision-making, applied to the above-mentioned surgical system with training or auxiliary functions, includes the following steps: Step S1: Through a high-precision timestamp synchronization protocol, strictly align the real-time tissue deformation data from different sensor channels, the doctor's hand movement data, and the patient's physiological signal data; Step S2: Input the multimodal data stream fused and calibrated in step S1 into the deep reinforcement learning model of the tactical decision layer to generate correction parameters and recommended operations for the surgical instrument's motion trajectory in real time; Step S3: Verify the legitimacy of the correction parameters or recommended operations output by the tactical decision layer based on the global surgical path planning preset by the strategic planning layer and the real-time safety boundary calculated by the safety threshold generation submodule of the execution layer; Step S4: If the correction parameters are verified to be legal in step S3, the doctor is guided to perform the optimized operation through the tactile feedback unit (such as resistance change) and the AR display unit (such as visual guide line); Step S5: Record the differences between the doctor's actual operation data and the AI ​​recommended data in real time, and use these difference data to update the model weights of the incremental learning engine to complete a closed-loop learning iteration.

[0017] Compared with the prior art, the present invention has the following significant beneficial effects: 1. Breakthroughs in accuracy and real-time performance: This invention achieves breakthroughs in accuracy and real-time performance in surgical assistance through the Kalman filter fusion of multimodal perception data and a hierarchical AI decision-making architecture. The individualized 3D path planning at the strategic planning layer combined with the dynamic trajectory correction of reinforcement learning at the tactical layer improves the operating accuracy of surgical instruments by approximately 30%; the magnetorheological fluid damper at the execution layer can effectively eliminate hand tremors with a frequency greater than 5Hz, reducing the risk of mis-cutting of key structures by more than 40%. The multimodal feedback system uses a high-precision timestamp synchronization protocol to achieve millisecond-level coupled feedback of tactile-visual-bioinformation, increasing the doctor's decision-making response speed by 25%. In particular, in delicate operations such as vascular anastomosis, AR enhanced display can increase the accuracy of identifying key anatomical structures by 35%.

[0018] 2. A New Paradigm for Continuous Skill Optimization: This invention's pioneering training-assisted closed-loop learning mechanism creates a new paradigm for continuous surgical skill optimization. The incremental learning engine within the federated learning framework enables AI models to continuously iterate and evolve based on real-world data, significantly improving performance. Combined with an entropy quantification model, it builds a personalized physician skill assessment and training path, accelerating the growth of novice physicians.

[0019] 3. High compatibility and low-cost expansion: The modular expansion interface achieves flexible and on-demand expansion of system functions through magnetic mechanical coupling and open source software architecture, enabling high-end auxiliary equipment to be compatible and shared with existing surgical instruments in grassroots hospitals, significantly reducing the cost of equipment modification and upgrades. The system modification cost is expected to be reduced by more than 50%, promoting the universal application of advanced technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flowchart of the steps of the coupling method of multimodal perception data and hierarchical AI decision-making of the present invention. DETAILED DESCRIPTION

[0021] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific implementation methods.

[0022] This invention discloses an intelligent surgical training assistance system that achieves intelligent optimization of the entire surgical process through closed-loop collaboration of five functional modules. The system architecture significantly improves surgical safety and training efficiency through the organic combination of multi-dimensional perception, hierarchical decision-making, dynamic feedback, and autonomous learning mechanisms. The specific modules are as follows: 1. Multimodal perception and data acquisition module This module builds a three-dimensional sensing network, using a micro pressure sensor array to work in conjunction with a biological signal acquisition device.

[0023] In a more specific embodiment, the micro pressure sensor is manufactured using MEMS technology and has a range of 0-50N / ±0.1N accuracy; The tip of the intelligent surgical forceps is integrated with a pressure sensing unit, which can monitor tissue compression deformation in real time. For example, in actual use, when the liver parenchyma compression rate is greater than 20%, a level 3 warning is triggered. In practice, pressure sensors integrated into surgical forceps directly quantify tissue compression deformation, and dynamic ultrasound detection of liver elasticity are examples of application scenarios, such as cirrhosis diagnosis, used to demonstrate the versatility of sensor data.

[0024] The nine-axis inertial sensor uses a sampling rate of ≥100Hz and is combined with an electromyographic wristband. The myoelectric wristband uses 1kHz sampling and recognizes eight surgical action patterns: grasping, rotating, pushing and pulling, cutting, suturing, pressing, pinching, and separating. This captures the operator's gesture intentions in real time, assisting the system in identifying key operational behaviors during surgical steps and building a characteristic model of the doctor's operation. The infrared spectrometer and ECG patch synchronously obtain tissue blood oxygen and patient ECG parameters; Kalman filtering achieves spatiotemporal alignment of multi-source data, generating a fused feature vector encompassing dimensions such as pressure, deformation rate, and myoelectric strength. Compared to single-modality solutions like the Da Vinci system, this design improves risk identification accuracy by 37.6% through cross-validation of mechanical and biological signals.

[0025] 2. Intelligent decision-making and hierarchical control module Adopting a three-tiered architecture of strategy, tactics, and execution to achieve precise assistance: (1) Strategic planning layer Based on patient DICOM images and a database of 100,000 anatomical variations, a 3D navigation map is generated using an improved U-Net algorithm, achieving a segmentation accuracy of 0.8mm for key structures. Semi-transparent path guidance is overlaid on the HoloLens 2 AR interface, achieving a depth perception error of less than 1mm and dynamically identifying areas with dense blood vessels and nerves.

[0026] (2) Tactical decision-making level A PPO reinforcement learning model is deployed, whose reward function combines efficiency and safety factors, deducts damage penalties, and generates device motion corrections based on real-time tissue mechanics data. For example, when tissue adhesion is detected, the optimal deflection angle is instantly calculated, such as adjusting from 45° to 40°. Assistance permissions are controlled using confidence thresholds, specifically triggering the assistance mechanism when the Monte Carlo Dropout variance exceeds 0.3.

[0027] (3) Execution control layer An integrated magnetorheological fluid damper with a 10ms response time and 5N·m output torque, along with a finite element simulation module, forcibly decelerates the instrument through reverse torque when the instrument speed exceeds a preset threshold, such as 20mm / s during laparoscopic procedures. This safety mechanism, based on a patient-specific anatomical model, calculates the maximum deformation threshold of the organ in real time and enforces hard constraints.

[0028] 1. Multimodal feedback and interaction module Establish a multi-channel feedback mechanism to achieve "eye-hand-brain" synergy enhancement: Tactile Dimension: A shape memory alloy composite glove with an adjustable stiffness range of 0.1 to 5 N / mm is used to dynamically adjust resistance based on the tissue elastic modulus. In the case of fibrotic liver, that is, when the elastic modulus is greater than 50 kPa, the tactile resistance is automatically increased to 3 N / mm. Visual dimension: The eye-tracking AR system adaptively adjusts information density based on the doctor's experience level, such as novice or expert mode. In novice mode, the system displays the blood vessel name and a 3.2mm safety distance mark, while in expert mode, only the red high-risk area warning is retained. Doctors' experience levels are determined based on the operation data collection module: an electromyographic bracelet and a nine-axis inertial sensor are used to monitor the doctor's hand muscle activity and tremor frequency. An LSTM model is then used to analyze the doctor's past 100 operation records to determine their experience level. Warning dimension: The linear actuator triggers an 800Hz vibration warning when the risk of mis-cutting a blood vessel exceeds 85%. The vibration intensity is dynamically adjusted based on historical error records. For example, by analyzing 100 operation records through LSTM, the intensity can be increased from 0.5G to 1.2G for doctors who frequently make mis-cutting attempts. The judgment of a risk of mis-cutting greater than 85% is based on LSTM analysis of 100 historical operation records, statistically analyzing the correlation between mis-cutting events and operating parameters such as device speed and pressure values; Data supporting the risk of misvascularization are as follows: Data input: Pressure sensors monitor tissue deformation, such as liver parenchyma compression rate, while nine-axis inertial sensors capture instrument motion trajectory and combine preoperative imaging data, such as 3D vascular positioning. Model output: The risk value is the predicted probability of the AI ​​model at the tactical decision-making level for the current operating parameters. The 85% threshold is based on the sensitivity and specificity balance point determined by ROC curve analysis.

[0029] 4. Training-assisted closed-loop learning module Building a positive cycle of "intraoperative data-virtual training-AI evolution": Federated Learning Framework: Secure Aggregation protocol is used to achieve data collaboration among hundreds of hospitals, and global model weights are updated every 100 surgeries. Skill Assessment System: This system calculates the Hausdorff distance of instrument trajectories to assess operational deviations. When the deviation is greater than 2 mm, a specialized training course, such as vascular anastomosis simulation, is generated. Open source training platform: Equipped with a 5N force feedback controller that supports USB-C and multi-platform compatibility, as well as surgical simulation software developed based on Unity, it supports STL organ model import and customized training scenarios.

[0030] 5. Modular expansion interface Provides hardware magnetic connectors and software open source frameworks to allow third-party devices and algorithms to access the system.

[0031] For example: Magnetic connector: The structural design uses a neodymium iron boron permanent magnet array with an adsorption force greater than 20N and supports 360° rotation and alignment; the electrical connection uses pogo pin contacts to transmit data and power, and is compatible with the CAN bus protocol of Da Vinci instruments; Blockchain data sharing: Smart contract logic sets that hospitals can earn 10 points for every 1TB of valid surgical data uploaded, which can be used to redeem advanced AI functions. Data verification uses zero-knowledge proof to verify authenticity, without the need to disclose the original content. USB-C universal interface: The power supply specification supports the PD3.0 protocol with a maximum power output of 100W; data transmission supports USB3.2 Gen2 with a speed of up to 10Gbps, and can simultaneously transmit 4 sensor data streams.

[0032] Furthermore, the collaborative logic between the intelligent decision-making and hierarchical control module and the multimodal feedback and interaction module also includes: When the decision confidence of the tactical decision-making layer is lower than the preset threshold, the visual feedback unit warning sign display intensity is forcibly enhanced, and the operation authority of the execution layer control unit is restricted; Confidence calculation is based on the Monte Carlo Dropout method. When the prediction variance of the tactical decision layer is greater than 0.3, it is judged as low confidence. The response strategy forces the AR to display alternative solutions, such as switching between blunt and sharp separation tools, and limits the movement speed of the instrument to 50% of the normal value. Dynamically adjust the vibration intensity and resistance threshold of the tactile feedback unit based on the doctor's historical error operation type database; Dynamic adjustment of feedback intensity is based on analyzing the doctor's past 100 operation records using an LSTM model to identify historical error types; The tactile gain rule increases the vibration intensity from the default 0.5G to 1.2G for doctors who frequently make mis-cuts. In training mode, the difference between the recommended operation path generated by the tactical decision-making layer and the doctor's actual operation path is quantified into skill evaluation parameters, and a visual training report is generated; The difference indicator is calculated by calculating the Hausdorff distance between the AI-suggested path and the actual path; the report generation automatically marks the operation stages with differences greater than 2mm, and generates targeted training courses such as special exercises on vascular anastomosis.

[0033] Based on the above, the present invention discloses a method for coupling multimodal perception data with hierarchical AI decision-making, constructing a complete technical chain from multimodal perception to hierarchical decision-making, dynamic feedback, and closed-loop learning. The core innovation lies in the hierarchical AI decision-making mechanism and the dynamic coupling logic of multimodal interaction. Through the coordinated optimization of data fusion, intelligent decision-making, and real-time feedback, it can achieve a dual improvement in surgical precision and safety.

[0034] like Figure 1 As shown in Figure 1, the method for coupling multimodal perception data with hierarchical AI decision-making includes the following steps: Step S1: aligning tissue deformation data, doctor's hand motion data and patient physiological signals through a timestamp synchronization protocol; Data collection dimensions include: Tissue mechanical parameters monitor tissue deformation characteristics in real time through a micro pressure sensor array integrated into the tip of the instrument; The dynamic parameters of the operation are captured by a nine-axis inertial sensor at a high sampling rate to capture the movement trajectory of the instrument, and the hand movement pattern is identified by combining it with electromyographic signals; Physiological status parameters are obtained by synchronizing infrared spectroscopy with ECG monitoring equipment to obtain the patient's blood oxygen saturation and cardiac electrophysiological indicators; The synchronization mechanism achieves high-precision alignment of multi-source data based on the timestamp protocol, eliminating time drift and spatial deviation across sensors.

[0035] Step S2: Input the fused multimodal data into the tactical decision layer to generate the equipment motion trajectory correction parameters; Data processing objectives include: Filter environmental noise and operational interference signals; Extract cross-modal correlation features and generate data vectors in a unified spatiotemporal coordinate system; Ensure data accuracy, such as controlling instrument positioning error to sub-millimeter level and pressure measurement accuracy better than standard thresholds; The technical implementation adopts the Kalman filter algorithm as a preferred embodiment, and its output feature vector is used as the input reference for subsequent hierarchical decision-making.

[0036] Step S3: Verify the legitimacy of the modified parameters based on the global path planning of the strategic planning layer and the security threshold of the execution layer; The strategic planning layer combines preoperative medical images with real-time feature vectors to generate a global navigation path and define the safe operation boundaries of key anatomical structures; The tactical decision-making layer uses deep reinforcement learning models to analyze real-time data streams, dynamically generate equipment motion correction parameters, and evaluate decision confidence. The execution control layer verifies the legitimacy of the correction parameters based on individualized safety thresholds and implements operation authority constraints through magnetorheological damping technology.

[0037] Step S4: If the correction parameters are legal, the doctor is guided to perform the operation through the tactile feedback unit and the AR display unit; The tactile warning module triggers gradient vibration feedback based on the risk level of mis-cutting, and the frequency and intensity are dynamically adjusted according to the risk index; The visual enhancement module adaptively adjusts the density of AR information, displaying all navigation signs in novice mode and only retaining high-risk area warnings in expert mode.

[0038] Step S5: Record the difference between the actual operation data and the AI ​​suggestion, and update the incremental learning engine model weight; The model iteration mechanism aggregates multi-center surgical data through a federated learning framework and regularly updates the global AI model to ensure continuous optimization of decision-making logic; The personalized adaptation mechanism generates customized training courses based on the doctor's historical operation data, such as special simulation training for vascular anastomosis deviation.

[0039] Kalman filtering, a well-known data processing method in the field, is used to illustrate the method of generating multimodal feature vectors. Its technical effect is decoupled from the core innovation point through modular interface design. The present invention establishes a new paradigm for intelligent surgical assistance systems through deep collaboration of hierarchical decision-making and multimodal feedback.

[0040] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A surgical system with training or auxiliary functions, characterized in that: include: Multimodal perception and data acquisition module, used to collect real-time surgical environment data, doctor operation data and patient physiological data; An intelligent decision-making and hierarchical control module receives the multimodal perception and data acquisition module and generates hierarchical auxiliary instructions through the strategic planning layer, tactical decision layer and execution layer control unit; a multimodal feedback and interaction module that provides the physician with dynamic tactile feedback, enhanced visual feedback, and biofit information based on the layered auxiliary instructions; A training-assisted closed-loop learning module feeds back the physician's operational data and patient physiological data collected during surgery to the virtual training interface and optimizes the AI ​​model of the intelligent decision-making and hierarchical control module based on an incremental learning engine; Modular expansion interface, providing hardware magnetic connector and software open source framework, allowing third-party devices and algorithms to access the system; Among them, the data of the multimodal feedback and interaction module are dynamically coupled with the hierarchical AI decision instructions through a timestamp synchronization protocol, and the incremental learning engine of the training-assisted closed-loop learning module personalizes the AI ​​decision weights according to the doctor's actual surgical performance.

2. A surgical system with training or auxiliary functions according to claim 1, characterized in that: The multimodal perception and data acquisition module includes: A micro pressure sensor array, integrated into the tip of a surgical instrument, is used to quantify tissue compression deformation parameters; Myoelectric wristband and nine-axis inertial sensor are used to monitor the doctor's hand muscle activity and tremor frequency; Infrared spectrometer and ECG patch for synchronizing tissue oxygen saturation with the patient's ECG signal; It also includes generating a multimodal feature vector of the surgical scene by fusing the above-mentioned sensor data through a Kalman filter algorithm after obtaining the above-mentioned sensor data.

3. The surgical system with training or auxiliary function according to claim 1, characterized in that: The intelligent decision-making and hierarchical control module includes: Strategic planning layer: Generates 3D surgical path planning based on the patient's preoperative medical images and anatomical variation database, and overlays key structure navigation maps through an AR transparent display screen; Tactical decision-making layer: Using a deep reinforcement learning model, the device's trajectory is dynamically adjusted based on real-time tissue mechanics data, and a prioritized list of recommended actions is output; Execution layer control unit: includes a safety threshold generation submodule, which calculates the maximum allowable deformation threshold of organ tissue through finite element simulation based on the patient's individual anatomical structure feature database and real-time physiological parameters during surgery; and applies micro-torque compensation to surgical instruments through magnetorheological fluid dampers to suppress hand tremors and limit the operation amplitude that exceeds the maximum allowable deformation threshold.

4. A surgical system with training or assisting function according to claim 3, characterized in that: The multimodal feedback and interaction module includes: Variable stiffness tactile gloves that dynamically adjust resistance based on tissue elasticity parameters output by the tactical decision layer; An eye-tracking-driven AR information density adaptive system automatically simplifies or enhances visual annotations based on the physician's experience level; The linear resonant actuator triggers a high-frequency vibration warning when the AI ​​detects the risk of accidental blood vessel cutting.

5. The surgical system with training or assistance function according to claim 1, wherein the training-assistance closed-loop learning module comprises: The incremental learning engine under the federated learning framework aggregates desensitized data from multiple hospitals to update AI model weights; Entropy quantification model, which evaluates doctors' operating skills based on the complexity of instrument motion trajectory and risk aversion index; The detachable force feedback handle and open source surgical simulation software allow doctors to train scenario-specific skills in non-surgical environments.

6. The surgical system with training or assisting function according to claim 1, characterized in that: The modular expansion interface includes: The magnetic multi-instrument compatible mechanical coupling structure supports rapid access to multi-degree-of-freedom robotic arm instruments; A blockchain-based data sharing incentive agreement allows hospitals to contribute desensitized data in exchange for access to advanced AI capabilities. The USB-C universal interface provides power supply and data transmission for peripherals.

7. The surgical system with training or assisting function according to claim 1, characterized in that: The collaborative logic of the intelligent decision-making and hierarchical control module and the multimodal feedback and interaction module also includes: When the decision confidence of the tactical decision layer is lower than a preset threshold, the warning mark display intensity of the enhanced visual feedback unit is forcibly enhanced, and the operation authority of the execution layer control unit is restricted; Dynamically adjust the vibration intensity and resistance threshold of the dynamic tactile feedback unit according to the doctor's historical erroneous operation type database; In the training mode, the difference between the recommended operation path generated by the tactical decision layer and the doctor's actual operation path is quantified as skill evaluation parameters, and a visual training report is generated.

8. A method for coupling multimodal perception data with hierarchical AI decision-making, applied to the surgical system with training or auxiliary functions according to any one of claims 1 to 7, characterized in that: The following steps are involved: Step S1: aligning tissue deformation data, doctor's hand motion data and patient physiological signals through a timestamp synchronization protocol; Step S2: Input the fused multimodal data into the tactical decision layer to generate the equipment motion trajectory correction parameters; Step S3: Verify the legitimacy of the modified parameters based on the global path planning of the strategic planning layer and the security threshold of the execution layer; Step S4: If the correction parameters are legal, the doctor is guided to perform the operation through the tactile feedback unit and the AR display unit; Step S5: Record the differences between actual operation data and AI recommendations, and update the model weights of the incremental learning engine.

Images