Virtual Surgery Simulation Method and System Based on Mixed Reality Technology

Abstract

The present application relates to the field of image processing technology, and discloses a virtual surgery simulation method and system based on mixed reality technology. The method includes: collecting three-dimensional spatial information to perform feature extraction and registration on the real surgical environment and video data, and generating mixed reality mapping and monitoring data; performing medical image registration processing based on the mixed reality mapping to obtain virtual scene data; processing the instrument sensing data according to the mapping data and monitoring parameters to obtain interaction data; simulating the surgical process according to the virtual scene and the interaction data; analyzing and identifying the surgical trajectory based on the simulation data to generate evaluation and early warning information; analyzing the evaluation and early warning information and outputting an analysis report. The present application effectively improves the authenticity and accuracy of surgical simulation training through a multi-source data fusion and real-time evaluation feedback mechanism.

Description

Technical Field

[0001] This application relates to the field of image processing, and particularly to a virtual surgery simulation method and system based on mixed reality technology. Background Art

[0002] With the continuous development of medical technology, surgical simulation training plays an increasingly important role in medical education and surgical skills training. Traditional surgical simulation training mainly relies on physical models and virtual reality technology. Although physical models can provide real tactile feedback, they are costly and lack repeatability; although virtual reality technology has good repeatability and customizability, it has obvious deficiencies in terms of realism and interactivity. In recent years, the emergence of mixed reality technology has provided a new technical path for surgical simulation training. By integrating the real surgical environment with the virtual surgical scene, it can provide a rich virtual interaction experience while ensuring the realism of training. However, current mixed reality surgical simulation systems still face many technical challenges in aspects such as real-time acquisition of the surgical environment, accurate registration of virtual and real scenes, dynamic tracking of surgical instruments, and evaluation and feedback of the surgical process.

[0003] Existing mixed reality surgical simulation systems have the following deficiencies: First, in terms of environmental perception, it is difficult to accurately obtain the dynamic change information of the surgical space, resulting in insufficient registration accuracy of virtual and real scenes; second, in terms of surgical process simulation, the existing systems do not accurately simulate the physical characteristics of tissue deformation and cannot truly reflect the tissue response during the surgical operation; third, in terms of surgical evaluation, there is a lack of accurate analysis and real-time evaluation mechanism for surgical operation trajectories, making it difficult to provide timely and effective feedback guidance for trainees. These technical problems seriously restrict the practical effect of mixed reality surgical simulation systems. Summary of the Invention

[0004] This application provides a virtual surgery simulation method and system based on mixed reality technology, which is used to solve the problem of how to achieve high-precision registration of virtual and real scenes and real-time surgical process evaluation during the mixed reality surgical simulation. Through multi-source data fusion and real-time evaluation and feedback mechanism, the realism and accuracy of surgical simulation training are effectively improved.

[0005] In a first aspect, the present application provides a virtual surgery simulation method based on mixed reality technology. The virtual surgery simulation method based on mixed reality technology includes: extracting spatial features and performing registration and calibration processing on real surgery environment data and on-site video data based on the collected three-dimensional spatial information to generate mixed reality space mapping data and environment real-time monitoring data; performing virtual-real registration and physical property mapping processing on the patient's medical images according to the mixed reality space mapping data to obtain virtual surgery scene data and real-time correction parameters; performing time-series fusion processing on the multi-source sensing data of the surgical instruments according to the mixed reality space mapping data, the environment real-time monitoring data, and the real-time correction parameters to obtain mixed reality interaction data; simulating the surgical process through tissue deformation simulation calculation according to the virtual surgery scene data and the mixed reality interaction data to obtain surgical simulation data and process monitoring data; performing feature analysis and pattern recognition processing on the surgical operation trajectory based on the surgical simulation data, the process monitoring data, and the environment real-time monitoring data to generate surgical evaluation data and real-time warning information; and performing operation suggestion analysis on the surgical evaluation data and the real-time warning information to obtain a surgical simulation process analysis report and target suggestion data.

[0006] In a second aspect, the present application provides a virtual surgery simulation system based on mixed reality technology. The virtual surgery simulation system based on mixed reality technology includes:

[0007] An acquisition module, configured to extract spatial features and perform registration and calibration processing on real surgery environment data and on-site video data based on the collected three-dimensional spatial information to generate mixed reality space mapping data and environment real-time monitoring data;

[0008] A mapping module, configured to perform virtual-real registration and physical property mapping processing on the patient's medical images according to the mixed reality space mapping data to obtain virtual surgery scene data and real-time correction parameters;

[0009] A fusion module, configured to perform time-series fusion processing on the multi-source sensing data of the surgical instruments according to the mixed reality space mapping data, the environment real-time monitoring data, and the real-time correction parameters to obtain mixed reality interaction data;

[0010] A simulation module, configured to simulate the surgical process through tissue deformation simulation calculation according to the virtual surgery scene data and the mixed reality interaction data to obtain surgical simulation data and process monitoring data;

[0011] An identification module, configured to perform feature analysis and pattern recognition processing on the surgical operation trajectory based on the surgical simulation data, the process monitoring data, and the environment real-time monitoring data to generate surgical evaluation data and real-time warning information;

[0012] An analysis module for performing operation recommendation analysis on the surgical evaluation data and the real-time warning information to obtain a surgical simulation process analysis report and target recommendation data.

[0013] In the technical solution provided by this application, through spatial feature extraction and registration calibration processing of real surgical environment data and on-site video data, accurate reconstruction of the surgical scene is achieved, improving the realism and immersion of the mixed reality environment. And through virtual-real registration and physical property mapping processing, the virtual surgical scene is made closer to the real surgical environment, enhancing the authenticity of the simulation training. In terms of surgical instrument interaction, the temporal fusion processing of multi-source sensing data is adopted to improve the accuracy and stability of surgical instrument position tracking. At the same time, through tissue deformation simulation calculation, a realistic surgical process simulation effect is achieved. In the surgical operation evaluation link, feature analysis and pattern recognition processing are performed on the surgical operation trajectory, an objective and comprehensive evaluation system is established, and through a real-time warning mechanism, potential risks are discovered in a timely manner, improving the safety of surgical training. Finally, through in-depth analysis of the surgical evaluation data and warning information, a detailed analysis report and targeted improvement suggestions are generated, improving the effect and efficiency of surgical training. Brief Description of the Drawings

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0015] Figure 1 It is a schematic diagram of an embodiment of the virtual surgical simulation method based on mixed reality technology in the embodiments of this application;

[0016] Figure 2 It is a schematic diagram of an embodiment of the virtual surgical simulation system based on mixed reality technology in the embodiments of this application. Detailed Embodiments

[0017] The embodiments of the present application provide a virtual surgery simulation method and system based on mixed reality technology. Terms such as "first", "second", "third", "fourth", etc. (if any) in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the term "including" or "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0018] For ease of understanding, the specific process of the embodiments of the present application will be described below. Please refer to Figure 1 In an embodiment of the virtual surgery simulation method based on mixed reality technology in the embodiments of the present application, it includes:

[0019] Step S101: Based on the collected three-dimensional space information, perform spatial feature extraction and registration calibration processing on the real surgical environment data and on-site video data to generate mixed reality space mapping data and environmental real-time monitoring data;

[0020] Step S102: According to the mixed reality space mapping data, perform virtual-real registration and physical property mapping processing on the patient's medical images to obtain virtual surgery scene data and real-time correction parameters;

[0021] Step S103: Based on the mixed reality space mapping data, environmental real-time monitoring data and real-time correction parameters, perform temporal fusion processing on the multi-source sensing data of the surgical instruments to obtain mixed reality interaction data;

[0022] Step S104: According to the virtual surgery scene data and the mixed reality interaction data, perform surgical process simulation through tissue deformation simulation calculation to obtain surgical simulation data and process monitoring data;

[0023] Step S105: Based on the surgical simulation data, process monitoring data and environmental real-time monitoring data, perform feature analysis and pattern recognition processing on the surgical operation trajectory to generate surgical evaluation data and real-time warning information;

[0024] Step S106: Perform operation suggestion analysis on the surgical evaluation data and real-time warning information to obtain a surgical simulation process analysis report and target suggestion data.

[0025] It can be understood that the execution entity of this application can be a virtual surgery simulation system based on mixed reality technology, or it can also be a terminal or a server. Specifically, no limitation is made here. In the embodiments of this application, the server is taken as the execution entity for illustration.

[0026] Specifically, three-dimensional spatial information is collected to process real surgical environment data and on-site video data. The surgical space image data is collected through a three-dimensional camera, and the depth data of the surgical scene is obtained by performing depth feature segmentation. In this process, the point cloud data collected by the depth camera is denoised and registered to ensure the accurate extraction of scene features. For the collected video data, key feature points are extracted using feature point detection technology. These feature points include spatial reference information such as the edge of the operating table, the position of instruments, and anatomical landmark points. By performing temporal tracking and spatial registration on these feature points, a stable mixed reality space mapping relationship is established. In the virtual-real registration link, the medical image data of the patient is accurately aligned with the mixed reality space. The medical image data includes three-dimensional imaging data such as CT and MRI. First, multi-scale feature extraction is performed on these data to identify tissue boundaries and anatomical structures. The extracted features are registered with the reference marker points in the actual surgical environment through a feature matching algorithm, and the spatial transformation matrix is calculated. On this basis, corresponding physical property parameters, including elastic modulus, density, etc., are assigned to different tissue structures to construct a virtual surgical scene. At the same time, according to the environmental lighting conditions and viewing angle changes, the rendering parameters of the virtual scene are dynamically adjusted to ensure visual realism.

[0027] The real-time tracking of surgical instruments adopts a multi-source sensing data fusion method. Through the optical marker points and inertial sensors arranged on the surgical instruments, the position and attitude data of the instruments are collected. These data are spatio-temporally aligned with the mixed reality space mapping data to calculate the accurate position of the instruments in the virtual scene. The tracking results are dynamically corrected according to the real-time monitoring data of the environment to eliminate the interference caused by environmental factors. The real-time correction parameters are used to compensate for the cumulative errors in the tracking process to ensure the tracking accuracy during long-term operations. In the surgical process simulation stage, based on the obtained virtual surgical scene data and mixed reality interaction data, tissue deformation simulation calculations are performed. By establishing a mechanical model of the tissue, the tissue deformation response during the surgical operation is simulated. When the surgical instrument comes into contact with the virtual tissue, the deformation distribution is calculated according to the contact force and tissue physical properties. This process needs to consider the non-linear characteristics and boundary constraint conditions of the tissue to ensure the accuracy of the deformation calculation. At the same time, key data during the entire surgical process are recorded, including the amount of deformation, stress distribution, operation trajectory, etc.

[0028] In the surgical evaluation stage, in-depth analysis is carried out on the collected surgical data. The surgical operation trajectory is analyzed through feature extraction algorithms to identify key operation steps and surgical techniques. The extracted features are compared with the standard surgical specifications to evaluate the standardization and proficiency of the operations. Combining with environmental monitoring data, early warnings are given for potential risks during the surgical process. This includes real-time monitoring of parameters such as the position of instruments, operation force, and tissue stress, and generating warning messages when the parameters exceed the safety thresholds. Finally, based on the surgical evaluation data and real-time warning information, a detailed analysis report is generated. The report content includes the scoring of surgical operations, skill analysis, risk warnings, and improvement suggestions. Through the quantitative analysis of operation data, an objective evaluation basis is provided for surgical training. The suggested content specifically points out the operation details and precautions that need to be improved to help improve surgical skills.

[0029] For example, during the intracranial tumor resection operation in the neurosurgical simulation training, the mixed reality spatial mapping data shows the three-dimensional spatial position of the surgical approach, and the virtual surgical scene data contains the precise anatomical structure and physical property parameters of the patient's brain tissue. When operating with surgical instruments, the mixed reality interaction data records the movement trajectory and operation force of the instruments, and the tissue deformation simulation calculation shows the deformation response of the brain tissue in real time. The surgical evaluation system analyzes the deformation amount and boundary stress of the brain tissue during the operation, and issues early warnings in a timely manner when it is found that it is approaching important neurovascular structures. The analysis report points out the stability of instrument control and the appropriateness of tissue traction force during the operation process, and provides specific improvement suggestions.

[0030] In the embodiments of the present application, through the spatial feature extraction and registration and calibration processing of the real surgical environment data and on-site video data, the precise reconstruction of the surgical scene is realized, the realism and immersion of the mixed reality environment are improved, and through the virtual-real registration and physical property mapping processing, the virtual surgical scene is made closer to the real surgical environment, enhancing the authenticity of the simulation training. In terms of surgical instrument interaction, the sequential fusion processing of multi-source sensing data is adopted to improve the accuracy and stability of surgical instrument position tracking. At the same time, through the tissue deformation simulation calculation, a realistic surgical process simulation effect is realized. In the surgical operation evaluation link, feature analysis and pattern recognition processing are carried out on the surgical operation trajectory, an objective and comprehensive evaluation system is established, and through the real-time warning mechanism, potential risks are discovered in a timely manner, improving the safety of surgical training. Finally, through the in-depth analysis of the surgical evaluation data and warning information, a detailed analysis report and targeted improvement suggestions are generated, improving the effect and efficiency of surgical training.

[0031] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0032] (1) Collect surgical space image data, perform depth feature segmentation on the surgical space image data to obtain surgical scene depth data;

[0033] (2) Collect surgical environment positioning data through a spatial positioning sensor, and perform time-domain feature calibration on the surgical scene depth data and the surgical environment positioning data to obtain a scene feature matrix;

[0034] (3) Obtain a real-time video stream of the surgical scene according to the video acquisition unit, extract spatial feature points from the video stream to obtain video feature point data;

[0035] (4) Perform spatial registration calculation on the scene feature matrix and the video feature point data to obtain a spatial registration mapping relationship;

[0036] (5) Perform real-time monitoring calculation on the surgical environment parameters according to the spatial registration mapping relationship to obtain real-time environment monitoring data;

[0037] (6) Perform feature fusion processing on the spatial registration mapping relationship and the real-time environment monitoring data to generate mixed reality space mapping data and real-time environment monitoring data.

[0038] Specifically, collect surgical space image data, collect three-dimensional point cloud information of the surgical scene through a depth camera, and each point contains spatial coordinates and depth values. Perform depth feature segmentation on the collected point cloud data, use the region growing algorithm to cluster the point cloud, identify the spatial distribution of key objects such as the operating table, instruments, and markers, and form surgical scene depth data. During the depth feature segmentation process, according to the spatial continuity and depth similarity of the point cloud, adjacent points are classified into the same region to establish a hierarchical representation of the scene. During the process of the spatial positioning sensor collecting surgical environment positioning data, use infrared positioning markers and optical tracking devices to obtain the real-time three-dimensional coordinates of key reference points in the surgical scene. Perform time-domain feature calibration on the surgical scene depth data and the surgical environment positioning data, align the two data streams through timestamps, and establish a unified time reference system. On this basis, construct a scene feature matrix, and the matrix contains the corresponding relationship between spatial positions, depth values, and positioning reference points.

[0039] The video acquisition device obtains a real-time video stream of the surgical scene and extracts spatial feature points from the video data. The feature point extraction process uses a corner detection algorithm to identify significant features in the video frame, such as the boundaries of the surgical area, the contours of instruments, and anatomical marker points. Perform temporal tracking on the extracted feature points to ensure the correspondence of feature points between consecutive frames to obtain video feature point data.

[0040] The spatial registration calculation is a process of precisely aligning the scene feature matrix and the video feature point data. The mathematical model of spatial registration uses the following formula:

[0041]

[0042] Among them, represents the registration mapping matrix, n is the number of feature points, is the weight coefficient of the i-th feature point, is the rotation matrix, is the spatial coordinate of the feature point, is the translation vector. The optimal spatial registration mapping relationship is obtained by minimizing the registration error. The acquisition of environmental real-time monitoring data covers multiple aspects: the light intensity monitoring uses a photosensitive sensor to collect the environmental light value for adjusting the rendering parameters of virtual objects; the temperature and humidity monitoring uses a temperature and humidity sensor to collect environmental parameters for compensating sensor drift; the spatial noise monitoring evaluates the environmental interference intensity by analyzing the jitter degree of point cloud data. These monitoring data are feature-fused with the spatial registration mapping relationship to establish an association model between environmental parameters and spatial positioning accuracy.

[0043] The feature fusion processing adopts a multi-level data fusion method to integrate the spatial registration mapping relationship with the environmental real-time monitoring data. At the feature level, the key features of spatial position and environmental parameters are extracted; at the decision-making level, the spatial positioning results are dynamically corrected according to environmental parameters. The fused data forms mixed reality space mapping data, including spatial coordinate transformation relationships, environmental parameter distributions, and correction parameters.

[0044] For example, the point cloud data of the operating table is collected by a depth camera to identify the access points of surgical instruments and the position of the laparoscope. The spatial positioning sensor tracks the marker points on the surgical instrument to obtain the real-time position of the instrument. The video acquisition device records the surgical field of view and extracts the anatomical feature points in the laparoscope image. The environmental monitoring data shows that the light intensity in the operating room is 500 lux, the temperature is 24 °C, and the humidity is 50%. Through spatial registration calculation, the depth data, positioning data, and video feature points are aligned to the same coordinate system to achieve the precise fusion of the virtual surgical scene and the real environment. When the environmental light changes, the system adjusts the rendering parameters of virtual objects according to the change in light intensity; when the temperature change causes sensor drift, the spatial positioning data is corrected according to the temperature compensation model. This real-time fusion and dynamic correction of multi-source data ensure the stability and accuracy of the mixed reality surgical simulation system.

[0045] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0046] (1) Perform multi-scale recognition processing on the tissue interface information in the medical image data through feature analysis calculation to obtain anatomical feature distribution data;

[0047] (2) Using the mixed reality space mapping data as a reference benchmark, perform pose estimation and error compensation processing on the anatomical feature distribution data to obtain initial registration data;

[0048] (3) From the initial registration data, perform key point screening and spatial feature matching operations, and perform iterative optimization processing on the spatial correspondence relationship to obtain virtual-real fusion mapping data;

[0049] (4) Perform physical parameter calculation and hierarchical analysis processing on different tissue regions in the virtual-real fusion mapping data to obtain tissue parameter distribution data;

[0050] (5) According to the tissue parameter distribution data, perform dynamic correction calculation on the deformation response in the virtual environment to obtain real-time correction parameters;

[0051] (6) Perform fusion and superposition processing on the virtual-real fusion mapping data and the tissue parameter distribution data according to the topological structure to generate virtual surgery scene data and real-time correction parameters.

[0052] Specifically, process the medical image data through a multi-scale analysis method, perform multi-layer decomposition on the image using wavelet transform, and extract tissue boundary information at different resolutions. At each layer of the image pyramid, identify the tissue interface through an edge detection algorithm to form a multi-level feature description. These features include the boundary contour, internal structure, and spatial position relationship of the tissue, constituting the anatomical feature distribution data.

[0053] The pose estimation and error compensation processing adopt the following mathematical model:

[0054]

[0055] Among them, represents the pose estimation error, m is the number of feature points, is the weight factor of the k-th feature point, is the homography matrix, is the feature point coordinate, is the position offset, is the error compensation coefficient, n is the number of compensation terms, is the compensation weight, is the error term. By minimizing obtain the optimal initial registration data.

[0056] In the process of key point screening and spatial feature matching, evaluate the importance of feature points for the initial registration data, and select points with significant features as the matching benchmark. Use the iterative closest point algorithm to register the feature points, and continuously optimize the spatial transformation parameters until the convergence condition is reached to obtain the virtual-real fusion mapping data.

[0057] The physical parameter calculation for different tissue regions adopts the following model:

[0058]

[0059] Among them, is the tissue physical parameter, p is the number of tissue regions, is the regional weight coefficient, is the basic physical parameter matrix, is the correction coefficient, is the hierarchical adjustment factor, q is the number of hierarchies, is the hierarchical weight, is the hierarchical parameter. The tissue parameter distribution data is calculated through this model.

[0060] Based on the tissue parameter distribution data, the deformation response in the virtual environment is dynamically corrected to establish the corresponding relationship between tissue deformation and external force. By calculating the deviation between the tissue deformation amount and the theoretical predicted value in real time, real-time correction parameters are generated to adjust the mechanical response of the virtual tissue. Finally, the virtual-real fusion mapping data and the tissue parameter distribution data are fused and superimposed, and the corresponding relationship between the virtual and real scenarios is established based on the topological structure to generate virtual surgical scenario data including geometric information and physical characteristics.

[0061] For example, in liver surgery simulation, first, the CT image data of the patient is processed, and anatomical features such as the liver contour, blood vessel orientation, and tumor location are identified through multi-scale analysis. The pose estimation of these features is performed with the actual surgical area in the mixed reality space to establish the spatial corresponding relationship between the virtual liver model and the real surgical environment. By analyzing the relationship between the CT value and tissue density, the physical parameters of different regions such as the liver parenchyma, blood vessels, and tumors are calculated. When the surgeon operates in the virtual environment, according to the contact situation between the instrument and the tissue, the tissue deformation response is calculated in real time and dynamically corrected according to the deformation characteristics in the actual surgical environment. During the whole process, the physical parameters and deformation response of the tissue are dynamically updated to ensure the consistency between the virtual surgical scenario and the real surgical environment. For example, when the surgical instrument applies pressure on the liver surface, by measuring the difference between the deformation amount of the real tissue and the deformation amount of the virtual model, parameters such as the elastic modulus are dynamically adjusted to make the deformation behavior of the virtual tissue closer to the real situation.

[0062] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0063] (1) Extract and analyze the spatio-temporal distribution of the multi-source sensing data of the surgical instrument to obtain the initial state data of the instrument;

[0064] (2)Perform spatial transformation processing on the mixed reality space mapping data and the environmental real-time monitoring data, and obtain the instrument spatial coordinate data through projection mapping;

[0065] (3)Calibrate and synchronize the instrument spatial coordinate data based on the real-time correction parameters to obtain the corrected instrument trajectory data;

[0066] (4)Perform motion feature analysis and interaction state recognition processing on the corrected instrument trajectory data to obtain the instrument interaction feature data;

[0067] (5)Perform constraint condition calculation and spatial detection processing on the instrument interaction feature data to obtain the instrument interaction response data;

[0068] (6)Perform time-domain fusion and space-domain fusion processing on the instrument interaction feature data and the instrument interaction response data to generate the mixed reality interaction data.

[0069] Specifically, extract the spatio-temporal distribution of data from different sensors, including the spatial position information of optical marker points, the attitude data of inertial measurement units, and the contact force data of force sensors. Through feature analysis processing, extract feature parameters such as the position, attitude, and movement speed of the instrument, construct a six-degree-of-freedom state description of the instrument, and form the initial state data of the instrument. In the spatial transformation processing stage, use the mixed reality space mapping data as the reference coordinate system, and combine environmental parameters such as light conditions, temperature, and humidity in the environmental real-time monitoring data to perform unified transformation of the coordinate system. Convert the position information of the instrument in different coordinate systems to a unified reference coordinate system through the projection mapping method to obtain the instrument spatial coordinate data, which accurately describes the spatial position and attitude information of the instrument in the mixed reality environment.

[0070] In the calibration and synchronization processing link, dynamically correct the instrument spatial coordinate data based on the real-time correction parameters. The real-time correction parameters include the system error caused by environmental factors, the sensor drift error, and the error compensation value due to time sequence asynchronization. By applying these correction parameters to the original coordinate data, accurate calibration of the instrument position is achieved, and at the same time, the time sequence synchronization between different data sources is ensured, and finally the corrected instrument trajectory data is obtained. In the motion feature analysis and interaction state recognition processing stage, deeply analyze the corrected instrument trajectory data. First, extract the kinematic features of the instrument, including parameters such as position change, attitude change, speed, and acceleration. At the same time, identify the interaction state between the instrument and the virtual tissue, and judge whether operations such as contact, puncture, and cutting occur, so as to obtain the instrument interaction feature data. The instrument interaction feature data contains information such as motion trajectory features, interaction type identifiers, contact force magnitude and direction.

[0071] Constraint condition calculation and spatial detection processing are processes for further analyzing the instrument interaction feature data. Based on constraint conditions such as the safety boundary and avoidance area of the surgical operation, the legality of the instrument movement is calculated. Through the spatial collision detection algorithm, the positional relationship between the instrument and the virtual tissue is judged, and potential collision risks are predicted. These processing results form instrument interaction response data, which includes constraint violation flags, collision warning information, force feedback parameters, etc. The final time-domain fusion and space-domain fusion processing systematically integrate the instrument interaction feature data and the instrument interaction response data. Time-domain fusion ensures the continuity and consistency of data at different time scales, while space-domain fusion ensures the precise correspondence of spatial position and attitude information. The fused mixed reality interaction data contains a complete description of the interaction process, including the movement trajectory of the instrument, interaction state, force feedback information, and safety warning data.

[0072] For example, in laparoscopic surgery simulation training, the optical marker points installed on the surgical instrument collect position data through an infrared camera. At the same time, the inertial measurement unit records the attitude changes of the instrument, and the force sensor collects the contact force between the instrument and the tissue. Processing these multi-source data, first, the position and attitude information of the instrument are obtained, and then these information are converted to the unified coordinate system of the mixed reality environment through a spatial transformation matrix. During the calibration process, considering the sensor drift caused by temperature changes in the operating room, compensation is performed by real-time correcting parameters. When the doctor uses the instrument to perform tissue separation operations, the operation type is analyzed based on the instrument trajectory data, and the required force feedback magnitude is calculated in combination with the physical properties of the virtual tissue. At the same time, it is detected whether the instrument is approaching dangerous areas such as important blood vessels, and corresponding warning information is generated. The finally generated mixed reality interaction data not only reflects the precise position and operation state of the instrument but also includes force feedback control signals and safety warning information.

[0073] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0074] (1) Perform tissue layering and deformation parameter space construction on the virtual surgical scene data to obtain three-dimensional surgical scene analysis data;

[0075] (2) Extract the non-linear transfer characteristics and calculate the stress distribution of the mechanical parameters in the mixed reality interaction data to form surgical interaction mechanical data;

[0076] (3) Based on the three-dimensional surgical scene analysis data and the surgical interaction mechanical data, perform surgical cutting path planning and tissue deformation compensation calculation, and output surgical operation prediction data;

[0077] (4) Perform tissue interface deformation analysis on the surgical operation prediction data through rigid body transformation and elastic deformation coupling calculation to obtain tissue interface contact data;

[0078] (5) Analyze and process the tissue interface contact data through multi-level physical deformation and stress propagation to form deformation data during the surgical process;

[0079] (6) Perform state fusion and process recording on the deformation data and tissue interface contact data during the surgical process to generate surgical simulation data and process monitoring data.

[0080] Specifically, different types of tissues are stratified according to anatomical structure, including the epidermis, fat layer, muscle layer, and internal organs, etc. Corresponding physical parameters are set for each layer of tissue, such as elastic modulus, density, Poisson's ratio, etc., to construct a deformation parameter space. Three-dimensional surgical scene analysis data is formed through the combination of these parameters, providing a basis for subsequent deformation calculations.

[0081] The processing of mechanical parameters in the mixed reality interaction data adopts the following non-linear stress distribution calculation model:

[0082]

[0083] Among them, is the stress distribution, is the number of force application points, is the non-linear transfer coefficient, is the magnitude of the force, is the acting direction, is the material property factor, is the number of stress propagation layers, is the inter-layer transfer coefficient, is the inter-layer stress attenuation factor. The surgical interaction mechanical data is obtained through this calculation.

[0084] In surgical cutting path planning, combining the three-dimensional surgical scene analysis data and the surgical interaction mechanical data, perform tissue deformation compensation calculation:

[0085]

[0086] Among them, is the deformation compensation amount, is the number of compensation points, is the compensation weight, is the initial deformation amount, is the compensation coefficient, is the global compensation factor, s is the number of compensation layers, is the layer weight, is the layer compensation amount.

[0087] The tissue interface deformation analysis adopts the following calculation model:

[0088]

[0089] Among them, is the interface deformation amount, d is the number of interface points, is the interface weight, is the rigid body deformation component, is the elastic deformation component, is the coupling coefficient, g is the number of interface layers, is the interlayer coupling coefficient, is the interlayer deformation amount.

[0090] Perform multi-level physical deformation analysis on the tissue interface contact data, consider the mechanical conduction characteristics between different levels of tissues, and calculate the propagation law of stress inside the tissues. This process forms complete deformation data of the surgical process, including the instantaneous deformation state of the tissues, the internal stress distribution, and the deformation propagation process. The final state fusion integrates the deformation data of the surgical process and the tissue interface contact data to generate surgical simulation data and process monitoring data. These data contain a complete record of the surgical operation process and provide a basis for surgical simulation evaluation.

[0091] For example, in liver surgery simulation, the liver tissue is stratified, including the capsule layer, the parenchymal layer, and the vascular layer. Through the calculation of non-linear stress distribution, analyze the stress propagation characteristics when the surgical instrument applies pressure on the liver surface. When the instrument moves along the preset cutting path, the cutting path is adjusted in real time according to the tissue deformation compensation calculation. During the cutting process, calculate the deformation state of the tissue separation surface through interface deformation analysis, and record the tissue stress distribution and deformation amount during the cutting process. These data are finally integrated to form a complete surgical simulation record, which is used to evaluate the accuracy and safety of the surgical operation. For example, when the surgical instrument touches the liver surface, first calculate the stress distribution at the contact point, then analyze the propagation of the stress inside the tissue, and finally generate the tissue deformation effect according to the deformation response characteristics of different levels of tissues.

[0092] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0093] (1) Extract the trajectory features of the surgical simulation data and perform segmentation processing on the surgical operations to separate and obtain the spatial motion feature data;

[0094] (2) Perform node analysis and behavior feature recognition on the process monitoring data according to the key surgical steps to form surgical step sequence data;

[0095] (3) Perform operation space limitation and dynamic tracking processing on the surgical area boundary for the environmental real-time monitoring data to obtain surgical area constraint data;

[0096] (4) Compare and analyze the spatial motion feature data with the standard operation sequence through spatial trajectory similarity calculation to obtain operation specification degree evaluation data;

[0097] (5) Determine the safety boundary and calculate the warning for dangerous areas based on the surgical procedure sequence data and the surgical area constraint data, and generate surgical risk prediction data;

[0098] (6) Conduct comprehensive feature analysis and risk level determination based on the operation compliance evaluation data and the surgical risk prediction data, and generate surgical evaluation data and real-time warning information.

[0099] Specifically, extract trajectory features and segment surgical operations. Trajectory feature extraction includes information such as the position, attitude, speed, and acceleration of surgical instruments in three-dimensional space. Segment the continuous surgical operation trajectory according to operation characteristics to identify the features of different operation stages. During the segmentation process, mark the starting point, key turning points, and ending point of the operation by analyzing the continuity of instrument movement and feature changes, thus obtaining spatial motion feature data. The processing of process monitoring data focuses on the identification and analysis of surgical procedures. For each key step in the surgical process, extract its characteristic parameters, including operation duration, operation force, instrument movement range, etc. By analyzing these characteristic parameters, identify the key nodes in the standard surgical process, such as steps like surgical approach establishment, tissue dissection, and suturing. Organize these nodes according to the temporal sequence relationship to form the complete surgical procedure sequence data.

[0100] The processing of real-time environmental monitoring data first determines the spatial scope of the surgical operation. According to the surgical type and anatomical structure characteristics, delimit the boundaries of the safe operation area. During the surgical process, dynamically track the relative position relationship between the surgical instrument and the key anatomical structures, and update the boundary constraints of the operation area in real time. This information constitutes the surgical area constraint data, which is used to ensure the safety of surgical operations. The comparative analysis between the spatial motion feature data and the standard operation sequence uses the spatial trajectory similarity calculation method. By calculating the similarity between the actual operation trajectory and the standard trajectory, evaluate the compliance degree of the surgical operation. The similarity calculation considers multiple dimensions such as the spatial shape of the trajectory, movement speed, and operation smoothness, and comprehensively forms the operation compliance evaluation data.

[0101] In the determination of the safety boundary and the calculation of the warning for dangerous areas, combine the surgical procedure sequence data and the surgical area constraint data to analyze the risk degree of the surgical operation. By calculating the shortest distance between the instrument and the important anatomical structures, evaluate the potential risk level. When the instrument approaches the dangerous area or the operation trajectory deviates from the safe path, trigger the warning signal of the corresponding level. This information constitutes the surgical risk prediction data.

[0102] Integrate the operation standardization evaluation data and surgical risk prediction data. By establishing an evaluation index system, comprehensively evaluate the surgical operation from dimensions such as operation proficiency, safety, and efficiency. At the same time, according to the judgment result of the risk level, generate corresponding warning information to provide timely feedback and guidance for surgical training.

[0103] For example, in the simulation of endoscopic ventricular surgery, first analyze the instrument trajectories in the surgical simulation data to identify key operation stages such as entering the skull, passing through the brain tissue, and entering the ventricle. Through the analysis of process monitoring data, determine the standard operation steps for each stage, such as determining the entry point, controlling the entry angle, and adjusting the working channel. The environmental monitoring data shows the real-time status of the surgical area, including the distance from important blood vessels and nerves. When analyzing the doctor's operation trajectory, compare it with the standard operation sequence to evaluate whether parameters such as the entry angle and entry depth meet the specifications. If it is found that the instrument deviates from the preset path or approaches important structures, immediately calculate the risk level and issue a warning.

[0104] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0105] (1) Quantify the sub-indexes of the surgical evaluation data and perform correlation analysis to generate evaluation index matrix data;

[0106] (2) Process the real-time warning information through risk level classification and risk factor identification to obtain warning analysis data;

[0107] (3) Perform mapping transformation on the evaluation index matrix data according to the standard data in the expert knowledge base to form initial recommendation data;

[0108] (4) Perform hierarchical processing and countermeasure generation on the warning analysis data through risk prevention and control strategies, and output risk response data;

[0109] (5) Perform multi-layer optimization calculation and solution integration on the initial recommendation data and the risk response data to obtain target recommendation data;

[0110] (6) Integrate and analyze the evaluation index matrix data, warning analysis data, and target recommendation data according to the evaluation specifications to generate a surgical simulation process analysis report and target recommendation data.

[0111] Specifically, sub - item indicators are quantified and correlation analysis is carried out. The sub - item indicators include dimensions such as surgical operation precision, timing rhythm, movement fluency, instrument control ability, and tissue protection awareness. Precise quantification criteria are adopted for each indicator. For example, operation precision is quantified by parameters such as spatial position error and angle deviation; timing rhythm is evaluated by the time allocation ratio and transition connection of each surgical step; movement fluency is measured based on the continuity and volatility of the speed curve. These quantified indicators are organized into an evaluation index matrix data, and each element in the matrix represents the evaluation score of a specific dimension. The processing of real - time warning information involves multi - level risk level classification. First, the warning information is classified, including risks of anatomical structure damage, instrument operation risks, risks of surgical step disorder, etc. According to the urgency and potential harm degree of the risks, the risk levels are divided into three levels: mild warning, moderate warning, and severe warning. In - depth risk factor identification is carried out for each warning event, and the direct causes and potential factors leading to the risks are analyzed to form structured warning analysis data.

[0112] In the application of the expert knowledge base, standard surgical data is organized in the form of a knowledge graph. The evaluation index matrix data is mapped and compared with the expert experience data to identify key problems and improvement spaces in the evaluation results. Through the knowledge inference engine, expert experience is transformed into specific improvement suggestions, including optimization of operation skills, adjustment of surgical paths, key points of instrument use, etc., to form initial suggestion data. The formulation of risk prevention and control strategies adopts a hierarchical and classified processing method. For different types of risks identified in the warning analysis data, corresponding prevention and control measures are designed. For the risk of tissue damage, precise safety boundaries and buffer zones are formulated; for instrument operation risks, detailed operation specifications and correction schemes are designed; for the risk of step disorder, a complete process correction mechanism is established. These strategies are systematically sorted out to form risk response data.

[0113] The integration and optimization of the initial suggestion data and the risk response data is a multi - level process. First, at the strategic level, the feasibility and effectiveness of the suggestions are evaluated; second, at the tactical level, the specific implementation steps are refined; finally, at the operational level, a detailed implementation plan is formulated. Through multi - level optimization calculations, the optimal improvement schemes are selected and these schemes are integrated to form target suggestion data.

[0114] The evaluation index matrix data, warning analysis data, and target suggestion data are systematically integrated according to a unified evaluation specification. The content framework of the analysis report includes: overall evaluation results, detailed index analysis, risk warning summary, list of improvement suggestions, subsequent training plan, etc. Each part has clear data support and specific improvement measures.

[0115] For example, in the simulation evaluation of cardiac surgery, the evaluation index matrix data shows that during the suturing operation, the doctor's suture distance uniformity and suture depth control need improvement. The early warning analysis data indicates that at some suture points, the distance from the coronary artery is too close, posing a risk of injury. By comparing with the expert knowledge base, specific improvement suggestions for suture techniques are generated, including adjustment of the needle insertion angle and optimization of tension control. The risk prevention and control strategy formulates a detailed safety area division plan for the protection of the coronary artery. The final analysis report not only points out the specific aspects that need improvement but also provides a systematic training plan, such as increasing special exercises for suture depth control and strengthening safety awareness training for operations around the coronary artery.

[0116] The virtual surgery simulation method based on the mixed reality technology in the embodiments of the present application has been described above. Next, the virtual surgery simulation system based on the mixed reality technology in the embodiments of the present application will be described. Please refer to Figure 2 , an embodiment of the virtual surgery simulation system based on the mixed reality technology in the embodiments of the present application includes:

[0117] An acquisition module, configured to perform spatial feature extraction and registration and calibration processing on real surgery environment data and on-site video data based on the acquired three-dimensional space information, and generate mixed reality space mapping data and environment real-time monitoring data;

[0118] A mapping module, configured to perform virtual-real registration and physical property mapping processing on the patient's medical images according to the mixed reality space mapping data, and obtain virtual surgery scene data and real-time correction parameters;

[0119] A fusion module, configured to perform time-series fusion processing on multi-source sensing data of surgical instruments according to the mixed reality space mapping data, the environment real-time monitoring data, and the real-time correction parameters, and obtain mixed reality interaction data;

[0120] A simulation module, configured to perform surgical process simulation through tissue deformation simulation calculation according to the virtual surgery scene data and the mixed reality interaction data, and obtain surgical simulation data and process monitoring data;

[0121] An identification module, configured to perform feature analysis and pattern recognition processing on the surgical operation trajectory based on the surgical simulation data, the process monitoring data, and the environment real-time monitoring data, and generate surgical evaluation data and real-time warning information;

[0122] An analysis module, configured to perform operation suggestion analysis on the surgical evaluation data and the real-time warning information, and obtain a surgical simulation process analysis report and target suggestion data.

[0123] Through the collaborative cooperation of the above-mentioned various components, through the extraction of spatial features and registration and calibration processing of real surgical environment data and on-site video data, the accurate reconstruction of the surgical scene is achieved, improving the realism and immersion of the mixed reality environment. And through virtual-real registration and physical property mapping processing, the virtual surgical scene is made closer to the real surgical environment, enhancing the authenticity of the simulation training. In terms of surgical instrument interaction, the temporal fusion processing of multi-source sensing data is adopted to improve the accuracy and stability of surgical instrument position tracking. At the same time, through tissue deformation simulation calculation, a realistic surgical process simulation effect is achieved. In the surgical operation evaluation link, the feature analysis and pattern recognition processing of the surgical operation trajectory are carried out, an objective and comprehensive evaluation system is established, and through a real-time warning mechanism, potential risks are detected in a timely manner, improving the safety of surgical training. Finally, through in-depth analysis of surgical evaluation data and warning information, detailed analysis reports and targeted improvement suggestions are generated, improving the effect and efficiency of surgical training.

[0124] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A virtual surgery simulation method based on mixed reality technology, characterized in that: The virtual surgery simulation method based on mixed reality technology includes: based on the collected three-dimensional spatial information, performing spatial feature extraction and registration and calibration processing on the real surgical environment data and the on-site video data to generate mixed reality space mapping data and real-time environmental monitoring data; According to the mixed reality space mapping data, the patient's medical image is processed by virtual-real registration and physical property mapping to obtain virtual surgical scene data and real-time correction parameters; According to the mixed reality space mapping data, the real-time environment monitoring data and the real-time correction parameters, time-series fusion processing is performed on the multi-source sensor data of the surgical instrument to obtain mixed reality interaction data; According to the virtual surgical scene data and the mixed reality interaction data, the surgical process is simulated through tissue deformation simulation calculation to obtain surgical simulation data and process monitoring data. This step specifically includes: performing tissue stratification and deformation parameter space construction on the virtual surgical scene data to obtain three-dimensional surgical scene analysis data; performing nonlinear transfer characteristic extraction and stress distribution calculation on the mechanical parameters in the mixed reality interaction data to form surgical interaction mechanical data; performing surgical cutting path planning and tissue deformation compensation calculation based on the three-dimensional surgical scene analysis data and the surgical interaction mechanical data, and outputting surgical operation prediction data; performing tissue interface deformation analysis on the surgical operation prediction data through rigid body transformation and elastic deformation coupling calculation to obtain tissue interface contact data; performing multi-level physical deformation and stress propagation analysis processing on the tissue interface contact data to form surgical process deformation data; performing state fusion and process recording processing on the surgical process deformation data and the tissue interface contact data to generate the surgical simulation data and the process monitoring data; Based on the surgical simulation data, the process monitoring data and the real-time environmental monitoring data, feature analysis and pattern recognition processing are performed on the surgical operation trajectory to generate surgical evaluation data and real-time warning information; The surgical evaluation data and the real-time warning information are analyzed for operation suggestions to obtain a surgical simulation process analysis report and target suggestion data.

2. The virtual surgery simulation method based on mixed reality technology according to claim 1, characterized in that: Based on the collected three-dimensional spatial information, the real surgical environment data and the on-site video data are subjected to spatial feature extraction and registration and calibration processing to generate mixed reality space mapping data and real-time environmental monitoring data, including: collecting surgical space image data, performing depth feature segmentation on the surgical space image data, and obtaining surgical scene depth data; Collecting surgical environment positioning data through a spatial positioning sensor, and performing time domain feature calibration on the surgical scene depth data and the surgical environment positioning data to obtain a scene feature matrix; Acquire a real-time video stream of the surgical scene according to the video acquisition unit, extract spatial feature points from the video stream, and obtain video feature point data; Performing spatial registration calculation on the scene feature matrix and the video feature point data to obtain a spatial registration mapping relationship; Performing real-time monitoring and calculation of surgical environment parameters according to the spatial registration mapping relationship to obtain real-time environmental monitoring data; The spatial registration mapping relationship and the environmental real-time monitoring data are subjected to feature fusion processing to generate the mixed reality spatial mapping data and the environmental real-time monitoring data.

3. The virtual surgery simulation method based on mixed reality technology according to claim 1, characterized in that: The method of performing virtual-real registration and physical property mapping processing on the patient's medical image according to the mixed reality space mapping data to obtain virtual surgical scene data and real-time correction parameters includes: performing multi-scale recognition processing on tissue interface information in the medical image data through feature analysis calculation to obtain anatomical feature distribution data; Using the mixed reality space mapping data as a reference benchmark to perform pose estimation and error compensation processing on the anatomical feature distribution data to obtain initial registration data; The initial registration data is used to perform key point screening and spatial feature matching operations, and the spatial correspondence is iteratively optimized to obtain virtual-real fusion mapping data; Performing physical parameter calculation and hierarchical analysis processing on different tissue regions in the virtual-reality fusion mapping data to obtain tissue parameter distribution data; Performing dynamic correction calculation on the deformation response in the virtual environment according to the tissue parameter distribution data to obtain real-time correction parameters; The virtual-reality fusion mapping data and the tissue parameter distribution data are fused and superimposed according to the topological structure to generate the virtual surgery scene data and the real-time correction parameters.

4. The virtual surgery simulation method based on mixed reality technology according to claim 1, characterized in that: The method performs time-series fusion processing on the multi-source sensor data of the surgical instrument according to the mixed reality space mapping data, the real-time environmental monitoring data and the real-time correction parameters to obtain mixed reality interaction data, including: performing spatiotemporal distribution extraction and feature analysis processing on the multi-source sensor data of the surgical instrument to obtain initial state data of the instrument; Performing spatial transformation processing on the mixed reality space mapping data and the real-time environmental monitoring data, and obtaining the instrument space coordinate data through projection mapping; Calibrate and synchronize the instrument spatial coordinate data based on the real-time correction parameters to obtain corrected instrument trajectory data; Performing motion feature analysis and interaction state recognition processing on the corrected instrument trajectory data to obtain instrument interaction feature data; Performing constraint condition calculation and space detection processing on the device interaction feature data to obtain device interaction response data; The device interaction feature data and the device interaction response data are subjected to time domain fusion and space domain fusion processing to generate the mixed reality interaction data.

5. The virtual surgery simulation method based on mixed reality technology according to claim 1, characterized in that: The method of performing feature analysis and pattern recognition processing on the surgical operation trajectory based on the surgical simulation data, the process monitoring data and the real-time environmental monitoring data to generate surgical evaluation data and real-time warning information includes: performing trajectory feature extraction and surgical operation segmentation processing on the surgical simulation data to separate and obtain spatial motion feature data; Perform node analysis and behavior feature recognition on the process monitoring data according to key surgical steps to form surgical step sequence data; Performing operation space limitation and surgical area boundary dynamic tracking processing on the real-time environmental monitoring data to obtain surgical area constraint data; Comparing and analyzing the spatial motion feature data with the standard operation sequence through spatial trajectory similarity calculation to obtain operation standardization evaluation data; Performing safety boundary determination and dangerous area warning calculation according to the surgical step sequence data and the surgical area constraint data to form surgical risk prediction data; Comprehensive feature analysis and risk level determination are performed based on the operation standardization assessment data and the surgical risk prediction data to generate the surgical assessment data and the real-time warning information.

6. The virtual surgery simulation method based on mixed reality technology according to claim 1, characterized in that: The step of performing operation suggestion analysis on the surgical evaluation data and the real-time warning information to obtain a surgical simulation process analysis report and target suggestion data includes: performing sub-item index quantification and correlation analysis on the surgical evaluation data to generate evaluation index matrix data; The real-time warning information is processed through risk level classification and risk factor identification to obtain warning analysis data; Mapping and transforming the evaluation index matrix data according to the standard data in the expert knowledge base to form initial recommendation data; The early warning analysis data is graded and countermeasures are generated through risk prevention and control strategies, and risk response data is output; Perform multi-layer optimization calculation and solution integration on the initial suggestion data and the risk response data to obtain target suggestion data; The evaluation index matrix data, the early warning analysis data and the target recommendation data are integrated and analyzed according to the evaluation specifications to generate the surgical simulation process analysis report and the target recommendation data.

7. A virtual surgery simulation system based on mixed reality technology, used to implement the virtual surgery simulation method based on mixed reality technology as described in any one of claims 1 to 6, characterized in that: The virtual surgery simulation system based on mixed reality technology includes: an acquisition module, which is used to extract spatial features and perform registration and calibration processing on real surgical environment data and on-site video data based on the acquired three-dimensional spatial information, and generate mixed reality space mapping data and real-time environmental monitoring data; A mapping module, used to perform virtual-real registration and physical property mapping processing on the patient's medical image according to the mixed reality space mapping data, so as to obtain virtual surgical scene data and real-time correction parameters; A fusion module, configured to perform time-series fusion processing on multi-source sensor data of the surgical instrument according to the mixed reality space mapping data, the real-time environmental monitoring data and the real-time correction parameters, so as to obtain mixed reality interaction data; A simulation module, used to simulate the surgical process through tissue deformation simulation calculation according to the virtual surgical scene data and the mixed reality interaction data, so as to obtain surgical simulation data and process monitoring data; An identification module, configured to perform feature analysis and pattern recognition processing on the surgical operation trajectory based on the surgical simulation data, the process monitoring data and the environmental real-time monitoring data, and generate surgical evaluation data and real-time warning information; The analysis module is used to perform operation suggestion analysis on the surgery evaluation data and the real-time warning information to obtain a surgery simulation process analysis report and target suggestion data.

Images