Precise positioning method and system for minimally invasive hepatobiliary surgery based on AR technology

Abstract

The present application relates to the field of image processing technology, and discloses a method and system for precise positioning of minimally invasive hepatobiliary surgery based on AR technology. The method includes: the present invention extracts vascular network data through medical image sequences and generates a three-dimensional structure diagram, marks surgical target points to establish a coordinate system; collects dynamic data for mapping and matching, and establishes a real-time tracking model; constructs an augmented reality scene and implants a navigation path; collects instrument position calculation trajectories, and generates instructions by comparing with navigation maps; constructs a projection interface to display key areas; records the entire process and evaluates the quality of the surgery. The present application effectively improves the accuracy and reliability of surgical navigation by establishing a real-time tissue deformation tracking model, a precise positioning mechanism of multi-source data fusion, and an intelligent surgical quality evaluation system.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an AR-based method and system for precise positioning of minimally invasive hepatobiliary surgery. Background Art

[0002] In the modern medical field, minimally invasive hepatobiliary surgery is widely used due to its advantages such as small trauma and fast recovery. Traditional minimally invasive hepatobiliary surgery mainly relies on doctors to perform surgical operations through endoscopic two-dimensional images, and combines preoperative CT, MRI and other imaging examinations for surgical planning. During the operation, the doctor needs to spatially map the three-dimensional information of the preoperative plan with the two-dimensional endoscopic images during the operation in his mind. This method heavily relies on the doctor's personal experience and spatial imagination. In addition, since the liver tissue will deform during the operation, the accuracy of the preoperative planning will be affected, which will bring certain risks to the operation. In recent years, with the development of augmented reality (AR) technology, the application of AR technology to surgical navigation has gradually become a research hotspot. The current surgical navigation system provides surgical guidance to doctors by superimposing the preoperative planning information on the surgical field of view.

[0003] However, the existing AR surgical navigation system still has many shortcomings: first, due to the lack of real-time tissue deformation tracking and compensation mechanism, there is an error between the actual anatomical structure during surgery and the AR display information; second, the spatial position tracking accuracy of surgical instruments is insufficient, making it difficult to provide accurate navigation guidance for doctors; third, the existing system lacks intelligent evaluation and quality control mechanisms for the surgical process, and cannot provide an objective evaluation of the surgical quality. These problems have seriously restricted the clinical application of AR technology in minimally invasive hepatobiliary surgery. Summary of the invention

[0004] In response to the problems of insufficient tissue deformation compensation, low instrument positioning accuracy and lack of surgical quality assessment in existing AR surgical navigation systems, this application provides a precise positioning method and system for minimally invasive hepatobiliary surgery based on AR technology. This method effectively improves the accuracy and reliability of surgical navigation by establishing a real-time tissue deformation tracking model, a precise positioning mechanism of multi-source data fusion, and an intelligent surgical quality assessment system.

[0005] In the first aspect, the present application provides a method for precise positioning of minimally invasive hepatobiliary surgery based on AR technology, and the method for precise positioning of minimally invasive hepatobiliary surgery based on AR technology includes: acquiring a medical image sequence through a scanner, extracting liver vascular network data from the medical image sequence, generating a three-dimensional anatomical structure map based on the liver vascular network data, marking surgical target points on the three-dimensional anatomical structure map, and obtaining a surgical space coordinate system; using a depth camera to collect dynamic data of the surgical area, mapping and matching the dynamic data of the surgical area with the surgical space coordinate system, calculating spatial deformation parameters according to the mapping and matching results, and establishing a real-time tracking model through the spatial deformation parameters; constructing an augmented reality scene based on the real-time tracking model, and in the augmented reality A surgical navigation path is implanted in a real scene, and the surgical navigation path is rendered by ray tracing to generate a surgical navigation map; surgical instrument position data is collected, and the tool motion trajectory is calculated based on the surgical instrument position data, and the tool motion trajectory is spatially compared with the surgical navigation map to form an instrument navigation instruction; a stereoscopic projection interface is constructed based on the instrument navigation instruction and the surgical navigation map, and a surgical key area identifier is loaded in the stereoscopic projection interface, and the transparency of the surgical key area identifier is adjusted to establish a real-time surgical guidance field of view; a data acquisition module is used to record the entire process information of the real-time surgical guidance field of view, and operation key frames are extracted from the entire process information, and the operation key frames are intelligently classified to generate a surgical quality assessment report.

[0006] In a second aspect, the present application provides a precise positioning system for minimally invasive hepatobiliary surgery based on AR technology, and the precise positioning system for minimally invasive hepatobiliary surgery based on AR technology includes:

[0007] an acquisition module, configured to acquire a medical image sequence through a scanner, extract liver vascular network data from the medical image sequence, generate a three-dimensional anatomical structure map based on the liver vascular network data, mark surgical target points on the three-dimensional anatomical structure map, and obtain a surgical space coordinate system;

[0008] A matching module, used to collect dynamic data of the surgical area using a depth camera, map and match the dynamic data of the surgical area with the surgical space coordinate system, calculate spatial deformation parameters according to the mapping and matching results, and establish a real-time tracking model through the spatial deformation parameters;

[0009] A construction module, used to construct an augmented reality scene based on the real-time tracking model, implant a surgical navigation path in the augmented reality scene, perform ray tracing rendering on the surgical navigation path, and generate a surgical navigation map;

[0010] A calculation module, used for collecting surgical instrument position data, calculating a tool motion trajectory according to the surgical instrument position data, and spatially comparing the tool motion trajectory with the surgical navigation map to form an instrument navigation instruction;

[0011] A loading module, used to construct a stereoscopic projection interface according to the instrument navigation instruction and the surgical navigation map, load the surgical key area identifier in the stereoscopic projection interface, adjust the transparency of the surgical key area identifier, and establish a real-time surgical guidance field of view;

[0012] The classification module is used to use the data acquisition module to record the whole process information of the real-time guidance field of view of the operation, extract the operation key frames from the whole process information, intelligently classify the operation key frames, and generate a surgery quality evaluation report.

[0013] In the technical solution provided by the present application, the process of acquiring medical image sequences and extracting liver vascular network data through a scanner ensures the integrity and accuracy of preoperative data collection, laying a data foundation for subsequent precise navigation. On this basis, the technical feature of using a depth camera to collect dynamic data of the surgical area and map and match it with the surgical space coordinate system solves the positioning deviation problem caused by tissue deformation during surgery and improves the accuracy of real-time tracking. The technical feature of building an augmented reality scene based on a real-time tracking model realizes the precise fusion of virtual information and actual surgical scenes, and provides intuitive and clear visual guidance by implanting surgical navigation paths and performing ray tracing rendering. The technical feature of collecting surgical instrument position data and calculating tool motion trajectories enables the spatial position of surgical instruments to be accurately tracked in real time. Combined with the spatial comparison with the surgical navigation map, the instrument navigation instructions formed provide accurate operation guidance for doctors. The technical feature of constructing a stereoscopic projection interface based on instrument navigation instructions and surgical navigation maps creates a hierarchical real-time surgical guidance field of view by loading key surgical area identifiers and adjusting transparency, effectively reducing the difficulty of surgical operations. Finally, the data acquisition module is used to record the entire process of the real-time surgical guidance field of view, and the technical features of the surgical quality assessment report are generated by extracting operation key frames and intelligent classification, thus achieving objective evaluation and quality control of the surgical process. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0015] Figure 1This is a schematic diagram of an embodiment of a method for accurate positioning of minimally invasive hepatobiliary surgery based on AR technology in an embodiment of the present application;

[0016] Figure 2 This is a schematic diagram of an embodiment of a precise positioning system for minimally invasive hepatobiliary surgery based on AR technology in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The embodiments of the present application provide a method and system for precise positioning of minimally invasive hepatobiliary surgery based on AR technology. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0018] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In the embodiments of the present application, an embodiment of the method for accurate positioning of minimally invasive hepatobiliary surgery based on AR technology includes:

[0019] Step S101, acquiring a medical image sequence through a scanner, extracting liver vascular network data from the medical image sequence, generating a three-dimensional anatomical structure map based on the liver vascular network data, marking surgical target points on the three-dimensional anatomical structure map, and obtaining a surgical space coordinate system;

[0020] Step S102: using a depth camera to collect dynamic data of the surgical area, mapping and matching the dynamic data of the surgical area with the surgical space coordinate system, calculating spatial deformation parameters according to the mapping and matching results, and establishing a real-time tracking model through the spatial deformation parameters;

[0021] Step S103: constructing an augmented reality scene based on the real-time tracking model, implanting a surgical navigation path in the augmented reality scene, performing ray tracing rendering on the surgical navigation path, and generating a surgical navigation map;

[0022] Step S104, collecting surgical instrument position data, calculating the tool motion trajectory according to the surgical instrument position data, and spatially comparing the tool motion trajectory with the surgical navigation map to form an instrument navigation instruction;

[0023] Step S105: construct a stereoscopic projection interface according to the instrument navigation instruction and the surgical navigation map, load the surgical key area identifier in the stereoscopic projection interface, adjust the transparency of the surgical key area identifier, and establish a real-time surgical guidance field of view;

[0024] Step S106: Use the data acquisition module to record the entire process information of the real-time guidance field of view of the surgery, extract operation key frames from the entire process information, intelligently classify the operation key frames, and generate a surgery quality assessment report.

[0025] It is understandable that the execution subject of the present application can be a precise positioning system for minimally invasive hepatobiliary surgery based on AR technology, or a terminal or a server, which is not limited here. The present application embodiment is described by taking the server as the execution subject as an example.

[0026] Specifically, a medical image sequence of the patient's hepatobiliary system is obtained by a CT scanning device. The scanning process uses multi-slice spiral CT technology to perform continuous tomographic scanning of the hepatobiliary system. The original scan data is processed by grayscale value analysis to convert tissue density information into standardized grayscale distribution data. The distribution range of grayscale values ​​is usually between 0-4095. By setting different grayscale thresholds, different tissue types such as blood vessels and liver parenchyma can be distinguished. In particular, the grayscale value distribution of liver blood vessels is relatively concentrated, which is helpful for extracting vascular networks. Based on the grayscale value distribution characteristics, the liver vascular network is extracted by a regional growing algorithm. The algorithm sets an initial seed point and gradually expands the region according to the grayscale value similarity principle to finally obtain a complete vascular network structure. After obtaining the vascular network data, an anatomical structure diagram is generated by three-dimensional reconstruction technology. The reconstruction process first converts the vascular network data into a point cloud form, each point containing spatial position and grayscale value information. The point cloud data is spatially aligned and denoised by a point cloud registration algorithm to remove noise points generated by the scanning process. The denoised point cloud data is used to generate a continuous vascular surface model through a surface reconstruction algorithm to form a three-dimensional anatomical structure diagram. On the three-dimensional structure diagram, surgical target points are marked according to characteristic positions such as vascular branch points and confluence points. The spatial coordinates of these points are standardized to form a surgical space coordinate system.

[0027] During the operation, a depth camera is used to collect dynamic data of the surgical area in real time. The depth camera obtains the depth information of the scene through structured light or time of flight principle, and generates point cloud data with depth values. These point cloud data collected in real time are registered with the surgical space coordinate system established before the operation, and the spatial transformation relationship between the two sets of data is calculated. The registration process uses an iterative closest point algorithm to reduce the registration error through continuous iterative optimization, and finally obtain an accurate mapping matrix. The tissue deformation parameters are calculated based on the mapping matrix, and a real-time tracking model is established. The real-time tracking model provides a basis for the construction of augmented reality scenes. The virtual information is aligned with the real surgical scene through spatial transformation, and the pre-planned surgical navigation path is implanted in the augmented reality scene. The display of the navigation path uses ray tracing rendering technology, taking into account the physical properties of light reflection, refraction, etc., to generate realistic visual effects. The rendered navigation path is integrated with the real surgical scene to form a surgical navigation map.

[0028] The position tracking of surgical instruments is achieved through electromagnetic sensors, which collect the position and posture information of the instruments in three-dimensional space. The continuously collected position data is analyzed in time series to calculate the movement trajectory of the instrument. The trajectory data is compared with the navigation map in real time. When the instrument deviates from the predetermined path, the system generates corresponding navigation instructions. The navigation instructions contain information such as direction adjustment and distance control to guide doctors to perform precise operations.

[0029] Based on navigation instructions and navigation maps, a stereoscopic projection interface is constructed. The interface adopts a layered display strategy to overlay the key surgical area logos on the actual surgical field of view. By adjusting the transparency of the logos, it is ensured that the virtual information does not obstruct important anatomical structures. This dynamic adjustment mechanism forms an intuitive real-time surgical guidance field of view. During the entire operation, the data acquisition module continuously records the surgical field of view information. Through scene analysis, key surgical frames are extracted, and each frame contains information such as instrument position and tissue status. These key frames are classified and analyzed to evaluate the standardization and effectiveness of surgical operations, and finally generate a comprehensive quality assessment report.

[0030] Taking vascular network extraction as an example, grayscale value analysis can identify micro-vessels with a diameter of less than 1 mm, providing detailed anatomical information for surgical planning. In terms of tissue deformation tracking, the real-time tracking model can capture micron-level tissue displacement and update navigation information in a timely manner. Analysis of surgical records shows that after adopting this solution, the accuracy of surgical positioning has been significantly improved, the operation time has been shortened, and the amount of intraoperative bleeding has been reduced, reflecting the advantages of this technical solution in improving surgical accuracy and safety.

[0031] In the embodiment of the present application, the process of acquiring medical image sequences and extracting liver vascular network data by scanner ensures the integrity and accuracy of preoperative data collection, laying a data foundation for subsequent precise navigation. On this basis, the technical feature of using a depth camera to collect dynamic data of the surgical area and map and match it with the surgical space coordinate system solves the positioning deviation problem caused by tissue deformation during surgery and improves the accuracy of real-time tracking. The technical feature of building an augmented reality scene based on a real-time tracking model realizes the precise fusion of virtual information and actual surgical scenes, and provides intuitive and clear visual guidance by implanting surgical navigation paths and performing ray tracing rendering. The technical feature of collecting surgical instrument position data and calculating tool motion trajectories enables the spatial position of surgical instruments to be accurately tracked in real time. Combined with the spatial comparison with the surgical navigation map, the instrument navigation instructions formed provide accurate operation guidance for doctors. The technical feature of constructing a stereoscopic projection interface based on instrument navigation instructions and surgical navigation maps creates a hierarchical real-time surgical guidance field of view by loading key surgical area identifiers and adjusting transparency, effectively reducing the difficulty of surgical operations. Finally, the data acquisition module is used to record the entire process of the real-time surgical guidance field of view, and the technical features of the surgical quality assessment report are generated by extracting operation key frames and intelligent classification, thus achieving objective evaluation and quality control of the surgical process.

[0032] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0033] (1) Collect tomographic images of the hepatobiliary system through a CT scanning device, perform grayscale value analysis on the tomographic images to obtain grayscale distribution data; classify tissues based on the grayscale distribution data to obtain a tissue type atlas; extract vascular density parameters from the tissue type atlas to generate a medical image sequence;

[0034] (2) Input the medical image sequence into the neural network for feature extraction, classify and screen the extracted features to obtain a vascular feature point set; perform morphological processing based on the vascular feature point set to extract liver vascular network data;

[0035] (3) Reconstruct a three-dimensional point cloud based on the liver vascular network data, perform noise reduction on the reconstructed point cloud data to form a spatial grid structure; perform surface fitting on the spatial grid structure to generate a three-dimensional anatomical structure diagram;

[0036] (4) Identify vascular branches on the three-dimensional anatomical structure diagram, extract node location information from the vascular branches, and obtain branch point data; use the branch point data to determine key anatomical landmarks and mark surgical target points at the key anatomical landmarks;

[0037] (5) Perform spatial geometric transformation on the surgical target points, convert the coordinate system of the transformed point set, and obtain standardized coordinate values; use the standardized coordinate values ​​to establish a spatial transformation matrix to generate the initial surgical space coordinate system;

[0038] (6) Verify the accuracy of the initial surgical space coordinate system, and use the verification data to calibrate and compensate the coordinate system to obtain the surgical space coordinate system.

[0039] Specifically, the patient's hepatobiliary system was tomographically scanned by a CT scanning device. The scanning process used multi-slice spiral CT technology, the scanning layer thickness was set to 1mm, and the scanning interval was 0.5mm to ensure the acquisition of high-resolution tomographic images. The grayscale analysis of the tomographic image was processed by histogram equalization to map the original CT value into a standardized grayscale range. The grayscale distribution data reflects the density characteristics of different tissues. The tissue classification is performed by setting the grayscale threshold interval to form a distribution map of different tissue types such as blood vessels, liver parenchyma, and bile ducts. On the basis of the tissue type map, the voxel density is calculated for the vascular tissue area, the vascular density parameters are extracted, and finally a medical image sequence containing spatial position and density information is generated. After preprocessing, the medical image sequence is input into a deep convolutional neural network, and the network structure includes a feature extraction layer and a classification layer. The feature extraction layer extracts local and global features of the image through multi-layer convolution operations, and the classification layer screens and classifies the extracted features. The vascular feature point set obtained after network processing includes key feature points such as vascular centerline points and branch points. Morphological processing algorithms, including skeletonization and connectivity analysis, are applied to the feature point set to reconstruct the topological structure of the vascular network and form complete liver vascular network data.

[0040] Based on the acquired vascular network data, a point cloud model is generated using 3D reconstruction technology. The point cloud data contains a large number of discrete spatial sampling points, each of which has a 3D coordinate value and density attribute. The point cloud is denoised using a statistical filtering algorithm to remove noise points that deviate from the main structure. The denoised point cloud data is constructed using the Delaunay triangulation algorithm to construct an initial grid structure, and then the grid quality is adjusted using a grid optimization algorithm to form a regular spatial grid structure. The NURBS surface fitting algorithm is applied to the grid structure to generate a smooth and continuous 3D anatomical structure map.

[0041] In the stage of vascular branch identification, the branch identification formula is introduced:

[0042]

[0043] in, represents the metric value of blood vessel branches, n is the number of neighborhood points, is the weight coefficient, is the spatial point coordinate, x, y, z are the three-dimensional spatial coordinate axes. The node position information is extracted using the following formula:

[0044]

[0045] in, is the node eigenvalue, is the gradient coefficient, is the curl coefficient, is the local velocity field, represents the gradient operator, which is used to calculate the rate of change of the vascular branch metric value B(v) in three-dimensional space. Represents the curl operator, which is used to calculate the rotational properties of the local velocity field.

[0046] The node position information obtained after the above processing constitutes a branch point data set, and key anatomical landmarks such as vascular intersections and confluences are determined based on the branch point data. The surgical target points are marked at these landmarks as key reference points for surgical navigation. The marked target points are subjected to spatial geometric transformation, including rotation, translation and other operations, to meet the requirements of the standard coordinate system. The transformed point set is converted to a standardized coordinate value through a coordinate system conversion, and a spatial transformation matrix is ​​constructed based on these coordinate values ​​to form the initial surgical space coordinate system. The final accuracy verification step is to analyze the registration error by comparing it to the standard model, calculate the spatial deviation of the coordinate points, and compensate and correct the coordinate system according to the deviation data, and finally obtain an accurate surgical space coordinate system.

[0047] For example, in a liver tumor surgery, the grayscale value analysis of the tomographic images obtained by CT scan successfully identified blood vessel structures with a diameter of more than 0.5 mm, and the feature points extracted by the neural network accurately reflected the direction and branching of the blood vessels. The anatomical structure diagram after three-dimensional reconstruction clearly showed the distribution of blood vessels around the tumor, and multiple surgical target points were accurately marked through the identification of blood vessel branches.

[0048] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0049] (1) Use a depth camera to perform multi-angle imaging of the surgical area, extract spatial depth information from the multi-angle images, generate a point cloud of the surgical area, and reconstruct the point cloud of the surgical area into dynamic data of the surgical area;

[0050] (2) Perform feature detection on the dynamic data of the surgical area, extract spatial geometric features, and obtain a feature vector group; pair the feature vector group with the marker points in the surgical space coordinate system to form mapping relationship data;

[0051] (3) Perform distance calculation based on the mapping relationship data, construct a feature point correspondence matrix, optimize and screen the feature point correspondence matrix, and obtain the mapping matching result;

[0052] (4) Perform tissue deformation analysis based on the mapping matching results, establish a deformation variable calculation equation, and solve the deformation variable calculation equation to obtain the spatial deformation parameters;

[0053] (5) Input the spatial deformation parameters into the biomechanical equation, calculate the tissue stress distribution, and obtain the deformation field data; perform real-time position correction based on the deformation field data to generate a real-time tracking model.

[0054] Specifically, real-time monitoring of the surgical area is achieved through a depth camera. The depth camera uses the principle of structured light to continuously scan the surgical area from different angles. The coded grating pattern projected by the structured light is mapped to the surface of the surgical area, and the surface depth value is calculated by analyzing the grating deformation. The depth information collected from multiple angles is aligned to generate high-density point cloud data, which contains three-dimensional spatial coordinates and depth value information. The point cloud data is reorganized into a dynamic data stream through time-series correlation analysis to reflect the real-time deformation state of the surgical area.

[0055] SIFT feature extraction is performed on the dynamic data of the surgical area. The SIFT algorithm constructs a scale space, detects local extreme points at different scales, and extracts feature points with scale invariance. Each feature point contains information such as position, scale, and direction, forming a 128-dimensional feature descriptor. The feature descriptor is processed by principal component analysis and dimensionality reduction to form a feature vector group. The feature vector group is matched with the marked points in the surgical space coordinate system marked before surgery, and the initial correspondence relationship is established using the nearest neighbor search algorithm to generate mapping relationship data. Based on the mapping relationship data, the Euclidean distance and Mahalanobis distance between the feature points are calculated to construct the feature point correspondence matrix. Each element in the correspondence matrix represents the similarity measure between a pair of feature points. The RANSAC algorithm is used to optimize the correspondence matrix, eliminate incorrect matching points, retain stable correspondence relationships, and obtain accurate mapping matching results.

[0056] The deformation calculation uses the following equation:

[0057]

[0058] in, Representing a spatial point The deformation at time t, m is the number of control points, is the deformation weight coefficient, is the spatial attenuation coefficient, r is the coordinate of the current observation point, are the coordinates of the control point positions, is a time-varying basis function.

[0059] The biomechanical properties of tissue deformation are described by the following equation:

[0060]

[0061] in, is the displacement field, is the first Lamé constant and is the second Lamé constant, is the pressure coupling coefficient, The pressure within the organization, is the external force, Represents the displacement field The acceleration over time The second-order rate of change of .

[0062] The deformation calculation results are combined with the biomechanical equations to obtain spatial deformation parameters, including displacement field, stress field and other information. Based on these parameters, the stress distribution state of each point in the tissue is calculated to generate complete deformation field data. The deformation field data is used to update the tissue position information in real time, and finally form a dynamically updated real-time tracking model.

[0063] For example, the depth camera captures images of the surgical area at a frequency of 30 frames per second, and extracts about 1,000 SIFT feature points from each frame. In the feature matching process, a rough matching correspondence is first established, and then a stable feature correspondence is obtained through screening using the RANSAC algorithm. When calculating the deformation variable, 50 key control points are selected to construct the deformation field, and the tissue deformation parameters are obtained by iteratively solving the biomechanical equations.

[0064] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0065] (1) Transform the spatial data information in the real-time tracking model, establish a visual space reference system, and generate an augmented reality scene through spatial calibration calculation;

[0066] (2) Extract spatial feature points from the augmented reality scene, perform regional connection calculations on the spatial feature points, and determine the boundary of the surgical safety area; plan the surgical navigation path based on the positional relationship between the boundary of the surgical safety area and the three-dimensional anatomical structure diagram;

[0067] (3) Decompose the surgical navigation path into a sequence of key control points, perform three-dimensional curve fitting on the key control point sequence, and generate path node data; construct a path guidance curve based on the path node data to form a path geometry description;

[0068] (4) Perform spatial illumination analysis based on the path geometry description, calculate the surface reflection coefficient distribution, and obtain illumination propagation data; use the illumination propagation data to perform ray tracing rendering on the surgical navigation path;

[0069] (5) The rendering result of the surgical navigation path is integrated with the augmented reality scene, a depth cache relationship is established, and a scene rendering image is generated; the perspective transformation calculation is performed based on the scene rendering image to form a surgical navigation map, wherein the surgical navigation map includes multiple navigation key points.

[0070] Specifically, the spatial point data in the real-time tracking model is converted from the camera coordinate system to the world coordinate system through the rigid body transformation matrix to establish a unified visual space reference system. The spatial calibration process uses the binocular vision calibration method to calculate the camera intrinsic parameter matrix and distortion parameters to ensure the precise alignment of the real scene with the virtual content, and finally generate the initial augmented reality scene. In the augmented reality scene, the Harris corner detection algorithm is used to extract spatial feature points. The detection process focuses on the significant features at the edge of the surgical area and the junction of tissues. The detected feature point set is processed by the region growing algorithm, and the adjacent feature points are connected into a continuous region by setting the growth threshold and connectivity constraints. The regional connection calculation is based on the dual constraints of spatial distance and feature similarity to ensure the rationality of the connection. The boundary contour of the surgical safety area is determined according to the connection results, and the boundary information is spatially registered with the pre-constructed three-dimensional anatomical structure map. Combined with the direction of blood vessels and the distribution of important anatomical structures, the optimal surgical navigation path is planned.

[0071] The refinement process of the surgical navigation path first decomposes the overall path into a sequence of key control points. The control points are selected based on the change of path curvature and the distribution of anatomical structures, and control points are set at the path turning points and key anatomical positions. The cubic spline interpolation algorithm is applied to the control point sequence for three-dimensional curve fitting to generate continuous and smooth path node data. Based on the node data, a Bezier curve is constructed as the path guide curve, and the curve shape is optimized by adjusting the control point weight parameters to form a complete path geometry description. A physically based lighting analysis is performed on the path geometry description, and the bidirectional reflectance distribution function (BRDF) model is used to calculate the surface reflection characteristics. Considering the effects of multiple light sources in the surgical scene, including surgical lights, ambient light and other factors, the surface reflection coefficient distribution is calculated. Ray tracing rendering is performed based on the reflection coefficient distribution, and the path tracing algorithm is used to simulate the propagation of light in the scene to generate a realistic navigation path visual effect.

[0072] The perspective transformation is calculated using the following formula:

[0073]

[0074] in, represents the viewing angle transformation matrix, is the transformation order, is the weight coefficient, is the translation transformation matrix, is the rotation transformation matrix, is the scaling matrix, is the view weight function, is the viewpoint distance. The rendering result of the surgical navigation path is fused with the augmented reality scene through the depth buffer algorithm. The depth buffer relationship ensures the correct occlusion relationship between the virtual navigation information and the real tissue, and generates a scene rendering with a clear sense of hierarchy. The scene rendering is calculated by perspective transformation, considering the dynamic changes of the doctor's viewpoint position, and the visual effect is updated in real time to form a surgical navigation map containing multiple navigation key points.

[0075] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0076] (1) Collecting the real-time position signal of the surgical instrument through an electromagnetic sensor, performing coordinate conversion processing on the real-time position signal, and obtaining the position data of the surgical instrument;

[0077] (2) Perform time series analysis on the surgical instrument position data, extract the position changes at consecutive time points, and generate a motion feature sequence; calculate the velocity and acceleration based on the motion feature sequence to form the tool motion trajectory;

[0078] (3) Map the tool motion trajectory to the surgical navigation map, calculate the trajectory deviation parameters, and establish spatial position correlation data; perform collision detection analysis on the spatial position correlation data to obtain safe distance data;

[0079] (4) Construct a risk warning threshold based on the safety distance data, compare the risk warning threshold with the spatial location associated data, and generate risk warning information;

[0080] (5) By performing correlation analysis between risk warning information and tool motion trajectory, key operation points are extracted, operation guidance data is established, and instrument navigation instructions are generated based on the operation guidance data. The instrument navigation instructions include direction guidance sub-instructions and navigation control data streams.

[0081] Specifically, the real-time position monitoring of surgical instruments is performed by electromagnetic sensors. The electromagnetic sensors include a transmitter and a receiver. The transmitter generates an electromagnetic field, and the receiver is installed on the surgical instrument to sense the change of the electromagnetic field. The real-time position signal collected by the receiver contains three-dimensional space coordinates and posture angle information. After the coordinate system conversion processing, the signal is converted from the local coordinate system of the sensor to the global coordinate system of the surgical space to obtain standardized surgical instrument position data. When performing time series analysis on the surgical instrument position data, the sliding time window method is used, the window length is set to 100ms, and the window overlap rate is 50%. In each time window, the position difference between adjacent sampling points is calculated to obtain the position change. After the position change sequence is subjected to median filtering to remove noise, a continuous motion feature sequence is formed. Based on the motion feature sequence, the instantaneous velocity and acceleration of the instrument are calculated. The numerical differentiation method is used. The velocity is obtained by the position difference, and the acceleration is obtained by the velocity difference, and finally a complete tool motion trajectory description is formed.

[0082] The nearest neighbor search algorithm is used to map the tool motion trajectory to the surgical navigation map. For each sampling point on the trajectory, the nearest reference point is found in the navigation map, and the spatial distance and direction deviation between the two points are calculated to obtain the trajectory deviation parameter. The trajectory deviation parameter is used to establish spatial position association data to describe the correspondence between the actual motion trajectory of the instrument and the planned path. The spatial position association data is subjected to collision detection analysis. The hierarchical bounding box algorithm is used to construct a collision detection model for surgical instruments and surrounding tissues, and the safety distance data between the instrument and important anatomical structures are calculated. Based on the safety distance data, a multi-level risk warning mechanism is established. The risk warning threshold is divided into three levels: safe, warning, and dangerous, corresponding to different safety distance ranges. The spatial position association data calculated in real time is compared with the preset risk warning threshold. When the safety distance is less than the warning threshold, risk warning information is generated. The warning information includes distance value, trend judgment, and risk level identification.

[0083] Based on the correlation analysis of risk warning information and tool motion trajectory, the key moments and positions during the surgical operation are identified. The extraction of key operation points is based on multiple factors, including trajectory curvature changes, speed changes, and risk level changes. For each key operation point, the corresponding operation guidance data is established. The guidance data contains the recommended movement direction, speed control parameters, and safety boundary information. Finally, the instrument navigation instructions are generated based on the operation guidance data. The navigation instructions are divided into two parts: direction guidance sub-instructions and navigation control data streams. The direction guidance sub-instructions give specific spatial adjustment suggestions, and the navigation control data stream provides continuous position and posture control parameters.

[0084] For example, the electromagnetic sensor installed on the surgical instrument collects 200 position sampling points per second, and each sampling point contains position and posture information with 6 degrees of freedom. In the timing analysis stage, the motion characteristics of the instrument are extracted through 100ms time window processing, and the characteristic changes of the instrument when approaching important blood vessels are found. The trajectory mapping results show that the maximum deviation between the actual motion trajectory of the instrument and the planned path occurs at the edge of the resection area. During the collision detection process, when the instrument is less than 5mm away from the portal vein branch, the risk warning mechanism is triggered, and warning information containing trend prediction is generated. Based on the warning information and operation analysis, precise navigation instructions are generated at this location to guide the doctor to adjust the direction and feed speed of the instrument to ensure the safety and accuracy of the surgical operation.

[0085] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0086] (1) Perform data fusion processing on the instrument navigation instructions and surgical navigation map, extract spatial projection parameters, and construct a three-dimensional projection interface;

[0087] (2) Obtain spatial positioning parameters from the stereoscopic projection interface, perform regional division calculations on the spatial positioning parameters, and establish a regional feature map; mark the boundaries of the dangerous area according to the regional feature map to form a critical surgical area identification;

[0088] (3) Calculate the depth coordinates of the surgical key area markers, generate layered display data, sort the layered display data, and obtain display priority parameters;

[0089] (4) Calculating the transparency value based on the display priority parameter, performing nonlinear mapping on the transparency value, and forming gradient display data; superimposing the gradient display data with the stereoscopic projection interface to generate a layered view;

[0090] (5) Establish a visual guidance hierarchy through layered views, extract surgical key point marking information, and construct a real-time surgical guidance field of view.

[0091] Specifically, data fusion processing is performed on instrument navigation instructions and surgical navigation maps. Data fusion adopts a multi-level fusion strategy to match direction guidance instructions, control parameters and spatial structure information of navigation maps. The two sets of data are unified into the same coordinate system through spatial transformation, and spatial projection parameters such as projection transformation matrix and viewpoint parameters are extracted. Based on these parameters, a stereoscopic projection interface is constructed, which serves as the basic platform for augmented reality display. The spatial positioning parameters in the stereoscopic projection interface include depth value, viewing angle information and spatial position coordinates. These parameters are subjected to regional segmentation and cluster analysis, and the K-means clustering algorithm is used to group spatial points according to distance and feature similarity. Based on the clustering results, a regional feature map is established, which contains the boundary information, spatial distribution characteristics and adjacent relationships of each region. According to the feature map combined with the safety boundary information of preoperative planning, the boundaries of dangerous areas such as vascular dense areas and tumor edges are marked to form a warning key surgical area logo.

[0092] The depth coordinates of the surgical critical area markers are calculated using the following formula:

[0093]

[0094] Among them, Z(x,y) is the depth value of point (x,y), is the depth scaling factor, is the number of reference points, is the weight coefficient, is the spatial attenuation factor, is the current point coordinate, is the reference point coordinate, is the depth correction factor, is the original depth map.

[0095] Prioritize the display content at different depth levels to generate layered display data. The priority is determined based on the importance of the anatomical structure, the warning level, and the operational relevance. The transparency is calculated using the following formula:

[0096]

[0097] in, is the transparency value at depth d, is the basic transparency coefficient, is the depth attenuation rate, is the number of impact factors, is the factor weight, is a quality parameter. The transparency value is converted into a gradient display effect through a nonlinear mapping function to ensure that important structures are clearly visible while avoiding blocking important surgical fields. The gradient display data is layered with the stereoscopic projection interface, and the Alpha blending algorithm is used to achieve a smooth transition display effect and generate a multi-level hierarchical view. The visual guidance level in the hierarchical view contains information such as surgical path instructions, warning marks, and operation prompts. The key mark point information is extracted in combination with the spatial position relationship, and finally a dynamically updated real-time surgical guidance field of view is constructed.

[0098] For example, the stereoscopic projection interface first integrates the preoperatively planned navigation path and real-time instrument position information. The area division process divides the surgical area into three levels: core resection area, warning area, and safe operation area, each with unique display characteristics. Depth calculation is based on the weighted results of multiple reference points to ensure the spatial hierarchy of the displayed content. Transparency calculation takes into account multiple factors such as structural importance, observation angle, and operation stage, so that the direction of blood vessels is clearly distinguishable against the tissue background. The surgical guidance field of view displays clear hierarchical information, and doctors can intuitively observe the spatial relationship between instruments and important structures, ensuring the accuracy and safety of surgical operations.

[0099] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0100] (1) The real-time surgical guidance field of view is continuously recorded through the data acquisition module, image sequences are extracted from the continuous records, and the image sequences are timestamped to generate full-process information;

[0101] (2) Analyze the scene changes of the whole process information, extract the scene transition points, segment the whole process information according to the scene transition points, and obtain the surgical stage sequence; calculate the inter-frame difference of the surgical stage sequence and generate the operation key frame;

[0102] (3) Extract features from the instrument trajectory data in the operation key frame, classify the extracted features, and establish an operation type database; perform action recognition through the operation type database to form surgical step data;

[0103] (4) Perform time series analysis on surgical step data, extract operation time parameters, calculate step completion indicators, and generate quality assessment parameters; compare the quality assessment parameters with standard operating specifications to form an evaluation data set;

[0104] (5) Extract key surgical indicators based on the evaluation data set, conduct statistical analysis on the key surgical indicators, and establish an evaluation indicator system; generate a surgical quality evaluation report based on the evaluation indicator system.

[0105] Specifically, a dual-channel data recording method is used to simultaneously collect augmented reality display content and actual surgical field of view. The continuous recording process uses high-speed image acquisition technology to collect 30 frames of image data per second. Each frame of the image is attached with accurate timestamp information, including millisecond-level time accuracy. The image sequence is stored using lossless compression to retain all the details of the original image, and combined with the timestamp data to form a complete record of the entire surgical process. The scene segmentation algorithm is used to analyze the scene changes of the entire process information, and the scene transition points are identified by calculating the image differences and feature changes between adjacent frames. The judgment criteria for scene transition points include multiple dimensions such as image content changes, instrument position changes, and surgical stage characteristics. According to the identified scene transition points, the entire process information is divided into multiple surgical stages, and each stage corresponds to a specific surgical operation link. In each surgical stage, the key action frames are extracted by calculating the inter-frame difference value. The selection of key frames is based on the amplitude of image content changes, instrument movement characteristics, and the importance of surgical operations.

[0106] The motion trajectory features of the instrument are extracted from the operation keyframes. The feature extraction process focuses on the spatial position, motion direction and speed change of the instrument. The extracted features are classified by the action classifier, and different operation actions are divided into basic types such as cutting, suturing and clamping. Based on the classification results, an operation type database is established, which contains standard action templates and corresponding feature descriptions. By matching with the standard actions in the database, the specific action type of each operation segment is identified to form surgical step data containing timing information. The timing analysis of surgical step data focuses on the operation time distribution and step connection relationship. The extraction of time parameters includes the duration of each step, the transition time between steps and the time interval of key operation points. The calculation of step completion indicators takes into account the accuracy, stability and time efficiency of the operation. These indicators are compared with the preset standard operation specifications to generate quality assessment parameters containing multi-dimensional evaluation information. The comparison process of quality assessment parameters with standard specifications adopts a multi-level evaluation system, taking into account factors such as surgical difficulty and individual differences of patients, and finally forms an objective evaluation data set.

[0107] The key indicator extraction process in the evaluation data set is based on statistical analysis methods, including multiple dimensions such as operation accuracy, time efficiency, and risk control. A comprehensive statistical analysis is performed on these indicators to establish a multidimensional evaluation indicator system that includes quantitative and qualitative evaluations. A detailed surgical quality evaluation report is generated based on this indicator system, which includes the advantages of the surgical operation, areas that need improvement, and specific improvement suggestions. For example, the data acquisition module records the entire process from surgical approach to the end of suturing. The scenario analysis divides the surgical process into four main stages: instrument placement, vascular ligation, tumor resection, and suture hemostasis. In the tumor resection stage, the two main types of operations, fine separation and blunt dissection, are identified by analyzing the characteristics of the instrument motion trajectory. The time series analysis shows that the operation time distribution in the vascular ligation stage is the most concentrated, while the time distribution in the tumor resection stage is more dispersed, which is directly related to the complexity of the surgical operation.

[0108] The above describes the precise positioning method for minimally invasive hepatobiliary surgery based on AR technology in the embodiment of the present application. The following describes the precise positioning system for minimally invasive hepatobiliary surgery based on AR technology in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of the hepatobiliary surgery minimally invasive surgery precise positioning system based on AR technology includes:

[0109] an acquisition module, configured to acquire a medical image sequence through a scanner, extract liver vascular network data from the medical image sequence, generate a three-dimensional anatomical structure map based on the liver vascular network data, mark surgical target points on the three-dimensional anatomical structure map, and obtain a surgical space coordinate system;

[0110] A matching module, used to collect dynamic data of the surgical area using a depth camera, map and match the dynamic data of the surgical area with the surgical space coordinate system, calculate spatial deformation parameters according to the mapping and matching results, and establish a real-time tracking model through the spatial deformation parameters;

[0111] A construction module, used to construct an augmented reality scene based on the real-time tracking model, implant a surgical navigation path in the augmented reality scene, perform ray tracing rendering on the surgical navigation path, and generate a surgical navigation map;

[0112] A calculation module, used for collecting surgical instrument position data, calculating a tool motion trajectory according to the surgical instrument position data, and spatially comparing the tool motion trajectory with the surgical navigation map to form an instrument navigation instruction;

[0113] A loading module, used to construct a stereoscopic projection interface according to the instrument navigation instruction and the surgical navigation map, load the surgical key area identifier in the stereoscopic projection interface, adjust the transparency of the surgical key area identifier, and establish a real-time surgical guidance field of view;

[0114] The classification module is used to use the data acquisition module to record the whole process information of the real-time guidance field of view of the operation, extract the operation key frames from the whole process information, intelligently classify the operation key frames, and generate a surgery quality evaluation report.

[0115] Through the collaborative cooperation of the above components, the process of acquiring medical image sequences and extracting liver vascular network data through scanners ensures the integrity and accuracy of pre-operative data collection, laying a data foundation for subsequent precise navigation. On this basis, the technical feature of using a depth camera to collect dynamic data of the surgical area and map and match it with the surgical space coordinate system solves the positioning deviation problem caused by tissue deformation during surgery and improves the accuracy of real-time tracking. The technical feature of building an augmented reality scene based on a real-time tracking model realizes the precise fusion of virtual information and actual surgical scenes, and provides intuitive and clear visual guidance by implanting surgical navigation paths and performing ray tracing rendering. The technical feature of collecting surgical instrument position data and calculating tool motion trajectories enables the spatial position of surgical instruments to be accurately tracked in real time. Combined with the spatial comparison with the surgical navigation map, the instrument navigation instructions formed provide accurate operation guidance for doctors. The technical feature of constructing a stereoscopic projection interface based on instrument navigation instructions and surgical navigation maps creates a hierarchical real-time surgical guidance field of view by loading key surgical area identifiers and adjusting transparency, effectively reducing the difficulty of surgical operations. Finally, the data acquisition module is used to record the entire process of the real-time surgical guidance field of view, and the technical features of the surgical quality assessment report are generated by extracting operation key frames and intelligent classification, thus achieving objective evaluation and quality control of the surgical process.

[0116] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A precise positioning method for minimally invasive hepatobiliary surgery based on AR technology, characterized in that: The AR-based hepatobiliary minimally invasive surgery precise positioning method comprises: Acquire a medical image sequence through a scanner, extract liver vascular network data from the medical image sequence, generate a three-dimensional anatomical structure map based on the liver vascular network data, mark surgical target points on the three-dimensional anatomical structure map, and obtain a surgical space coordinate system; Using a depth camera to collect dynamic data of the surgical area, mapping and matching the dynamic data of the surgical area with the surgical space coordinate system, calculating spatial deformation parameters according to the mapping and matching results, and establishing a real-time tracking model through the spatial deformation parameters; Building an augmented reality scene based on the real-time tracking model, implanting a surgical navigation path in the augmented reality scene, performing ray tracing rendering on the surgical navigation path, and generating a surgical navigation map; Collecting surgical instrument position data, calculating tool motion trajectory according to the surgical instrument position data, and spatially comparing the tool motion trajectory with the surgical navigation map to form instrument navigation instructions; Constructing a stereoscopic projection interface according to the instrument navigation instruction and the surgical navigation map, loading surgical key area identifiers in the stereoscopic projection interface, adjusting the transparency of the surgical key area identifiers, and establishing a real-time surgical guidance field of view; The data acquisition module is used to record the whole process information of the real-time guidance field of view of the operation, the operation key frames are extracted from the whole process information, the operation key frames are intelligently classified, and the operation quality evaluation report is generated.

2. The AR-based precise positioning method for minimally invasive hepatobiliary surgery according to claim 1, characterized in that: The method of acquiring a medical image sequence by a scanner, extracting liver vascular network data from the medical image sequence, generating a three-dimensional anatomical structure diagram based on the liver vascular network data, marking surgical target points on the three-dimensional anatomical structure diagram, and obtaining a surgical space coordinate system includes: Acquire hepatobiliary system tomographic images by a CT scanning device, perform grayscale value analysis on the tomographic images to obtain grayscale distribution data; perform tissue classification based on the grayscale distribution data to obtain a tissue type atlas; extract vascular density parameters from the tissue type atlas to generate the medical image sequence; Inputting the medical image sequence into a neural network for feature extraction, classifying and screening the extracted features to obtain a vascular feature point set; performing morphological processing based on the vascular feature point set to extract the liver vascular network data; Reconstructing a three-dimensional point cloud based on the liver vascular network data, performing noise reduction processing on the reconstructed point cloud data to form a spatial grid structure; performing surface fitting on the spatial grid structure to generate the three-dimensional anatomical structure diagram; Performing vascular branch identification on the three-dimensional anatomical structure diagram, extracting node position information from the vascular branches, and obtaining branch point data; determining key anatomical landmarks using the branch point data, and marking the surgical target point at the key anatomical landmark; Performing spatial geometric transformation on the surgical target points, performing coordinate system conversion on the transformed point set to obtain standardized coordinate values; using the standardized coordinate values ​​to establish a spatial transformation matrix to generate an initial surgical spatial coordinate system; The accuracy of the initial surgical space coordinate system is verified, and the coordinate system is corrected and compensated through verification data to obtain the surgical space coordinate system.

3. The AR-based precise positioning method for minimally invasive hepatobiliary surgery according to claim 1 is characterized in that: The method of collecting dynamic data of the surgical area using a depth camera, mapping and matching the dynamic data of the surgical area with the surgical space coordinate system, calculating spatial deformation parameters according to the mapping and matching results, and establishing a real-time tracking model through the spatial deformation parameters includes: Perform multi-angle imaging of the surgical area through a depth camera, extract spatial depth information from the multi-angle images, generate a point cloud of the surgical area, and reorganize the point cloud of the surgical area into dynamic data of the surgical area; Performing feature detection on the dynamic data of the surgical area, extracting spatial geometric features, and obtaining a feature vector group; pairing the feature vector group with the marking points in the surgical space coordinate system to form mapping relationship data; Perform distance calculation based on the mapping relationship data, construct a feature point correspondence matrix, optimize and screen the feature point correspondence matrix, and obtain the mapping matching result; Perform tissue deformation analysis based on the mapping matching results, establish a deformation variable calculation equation, and solve the deformation variable calculation equation to obtain the spatial deformation parameter; The spatial deformation parameters are input into the biomechanical equation, the tissue stress distribution is calculated, and the deformation field data is obtained; the real-time position correction is performed according to the deformation field data to generate the real-time tracking model.

4. The AR-based precise positioning method for minimally invasive hepatobiliary surgery according to claim 1, characterized in that: The method of constructing an augmented reality scene based on the real-time tracking model, implanting a surgical navigation path in the augmented reality scene, performing ray tracing rendering on the surgical navigation path, and generating a surgical navigation map includes: Performing coordinate transformation on the spatial data information in the real-time tracking model, establishing a visual space reference system, and generating the augmented reality scene through spatial calibration calculation; Extracting spatial feature points from the augmented reality scene, performing regional connection calculation on the spatial feature points, and determining the boundary of the surgical safety area; planning the surgical navigation path according to the positional relationship between the boundary of the surgical safety area and the three-dimensional anatomical structure diagram; Decomposing the surgical navigation path into a key control point sequence, performing three-dimensional curve fitting on the key control point sequence to generate path node data; constructing a path guidance curve according to the path node data to form a path geometry description; Performing spatial illumination analysis on the path geometry description, calculating the surface reflection coefficient distribution, and obtaining illumination propagation data; and performing ray tracing rendering on the surgical navigation path using the illumination propagation data; The rendering result of the surgical navigation path is integrated with the augmented reality scene, a depth cache relationship is established, and a scene rendering image is generated; and a perspective transformation calculation is performed based on the scene rendering image to form the surgical navigation map, wherein the surgical navigation map includes a plurality of navigation key points.

5. The AR-based precise positioning method for minimally invasive hepatobiliary surgery according to claim 1, characterized in that: The collecting of surgical instrument position data, calculating a tool motion trajectory according to the surgical instrument position data, and spatially comparing the tool motion trajectory with the surgical navigation map to form an instrument navigation instruction includes: Collecting the real-time position signal of the surgical instrument through an electromagnetic sensor, performing coordinate conversion processing on the real-time position signal, and obtaining the position data of the surgical instrument; Performing time series analysis on the surgical instrument position data, extracting position changes at consecutive time points, and generating a motion feature sequence; performing velocity acceleration calculation based on the motion feature sequence to form the tool motion trajectory; Mapping the tool motion trajectory to the surgical navigation map, calculating trajectory deviation parameters, and establishing spatial position association data; performing collision detection analysis on the spatial position association data to obtain safety distance data; Constructing a risk warning threshold based on the safety distance data, comparing the risk warning threshold with the spatial position associated data, and generating risk warning information; By performing correlation analysis between the risk warning information and the tool motion trajectory, key operation points are extracted, operation guidance data are established, and the instrument navigation instructions are generated according to the operation guidance data, wherein the instrument navigation instructions include direction guidance sub-instructions and navigation control data streams.

6. The AR-based precise positioning method for minimally invasive hepatobiliary surgery according to claim 1, characterized in that: The method of constructing a stereoscopic projection interface according to the instrument navigation instruction and the surgical navigation map, loading a surgical key area identifier in the stereoscopic projection interface, adjusting the transparency of the surgical key area identifier, and establishing a real-time surgical guidance field of view includes: Performing data fusion processing on the instrument navigation instruction and the surgical navigation map, extracting spatial projection parameters, and constructing the stereoscopic projection interface; Acquire spatial positioning parameters from the stereoscopic projection interface, perform regional division calculation on the spatial positioning parameters, and establish a regional feature map; mark the boundary of the dangerous area according to the regional feature map to form the surgical key area identification; Calculating the depth coordinates of the surgical key area markers to generate hierarchical display data, sorting the hierarchical display data, and obtaining display priority parameters; Calculating a transparency value based on the display priority parameter, performing nonlinear mapping on the transparency value to form gradient display data; superimposing the gradient display data with the stereoscopic projection interface to generate a layered view; The visual guidance hierarchy is established through the layered view, the surgical key point marking information is extracted, and the real-time surgical guidance field of view is constructed.

7. The AR-based precise positioning method for minimally invasive hepatobiliary surgery according to claim 1, characterized in that: The data acquisition module is used to record the whole process information of the real-time guided visual field of the operation, extract the operation key frames from the whole process information, intelligently classify the operation key frames, and generate a surgery quality assessment report, including: Continuously recording the real-time guided visual field of the surgery through a data acquisition module, extracting an image sequence from the continuous recording, annotating the image sequence with a timestamp, and generating the whole process information; Performing scene change analysis on the whole process information, extracting scene transition points, segmenting the whole process information according to the scene transition points, and obtaining a surgical stage sequence; performing frame difference calculation on the surgical stage sequence, and generating the operation key frame; Extracting features from the instrument trajectory data in the operation key frame, classifying the extracted features, and establishing an operation type database; performing action recognition through the operation type database to form surgical step data; Performing time series analysis on the surgical step data, extracting operation time parameters, calculating step completion indicators, and generating quality assessment parameters; comparing the quality assessment parameters with standard operating specifications to form an assessment data set; Extract key surgical indicators according to the evaluation data set, perform statistical analysis on the key surgical indicators, and establish an evaluation indicator system; and generate the surgical quality evaluation report based on the evaluation indicator system.

8. A precise positioning system for minimally invasive hepatobiliary surgery based on AR technology, used to implement the precise positioning method for minimally invasive hepatobiliary surgery based on AR technology as described in any one of claims 1 to 7, characterized in that: The AR technology-based hepatobiliary minimally invasive surgery precise positioning system includes: an acquisition module, configured to acquire a medical image sequence through a scanner, extract liver vascular network data from the medical image sequence, generate a three-dimensional anatomical structure map based on the liver vascular network data, mark surgical target points on the three-dimensional anatomical structure map, and obtain a surgical space coordinate system; A matching module, used to collect dynamic data of the surgical area using a depth camera, map and match the dynamic data of the surgical area with the surgical space coordinate system, calculate spatial deformation parameters according to the mapping and matching results, and establish a real-time tracking model through the spatial deformation parameters; A construction module, used to construct an augmented reality scene based on the real-time tracking model, implant a surgical navigation path in the augmented reality scene, perform ray tracing rendering on the surgical navigation path, and generate a surgical navigation map; A calculation module, used for collecting surgical instrument position data, calculating a tool motion trajectory according to the surgical instrument position data, and spatially comparing the tool motion trajectory with the surgical navigation map to form an instrument navigation instruction; A loading module, used to construct a stereoscopic projection interface according to the instrument navigation instruction and the surgical navigation map, load the surgical key area identifier in the stereoscopic projection interface, adjust the transparency of the surgical key area identifier, and establish a real-time surgical guidance field of view; The classification module is used to use the data acquisition module to record the whole process information of the real-time guidance field of view of the operation, extract the operation key frames from the whole process information, intelligently classify the operation key frames, and generate a surgery quality evaluation report.

Images