Method and system for processing minimally invasive surgical data based on AR technology

Abstract

This application relates to the field of image processing technology and discloses a method and system for processing surgical minimally invasive surgery data based on AR technology. The method includes: performing three-dimensional reconstruction and lesion segmentation on medical imaging data and extracting feature points; using feature point data to acquire the position of surgical instruments and perform spatial mapping; reconstructing the light field based on feature point and position data and generating an augmented reality scene; performing trajectory analysis and path planning based on augmented reality; performing real-time monitoring of the surgical path and risk warning; and finally extracting surgical rules and establishing knowledge mapping relationships. This application implements the correlation analysis and fusion processing of multi-source heterogeneous data during surgery, establishes a complete technical chain from data acquisition to knowledge extraction, and improves the safety and intelligence level of surgery.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method and system for processing minimally invasive surgical data based on AR technology. Background Art

[0002] In existing minimally invasive surgical procedures, the application of AR technology is primarily focused on surgical navigation and real-time intraoperative imaging. Traditional AR surgical navigation technology provides intuitive surgical guidance by overlaying preoperative imaging data with the real-time surgical scene. Existing technologies also include methods such as optical tracking and electromagnetic tracking to spatially locate surgical instruments and achieve real-time navigation of the surgical path. In terms of surgical data processing, existing methods primarily utilize image registration, 3D reconstruction, and trajectory planning to collect and analyze various types of data during the surgical process.

[0003] However, existing technologies have significant shortcomings in data processing and risk warning. First, existing data processing methods often fragment various types of data and lack systematic analysis and correlation mining of data from the entire surgical process. Second, in terms of surgical risk warning, existing technologies mainly rely on simple threshold judgments and are unable to effectively identify and predict complex risk evolution processes. In addition, existing technologies are also deficient in knowledge accumulation and experience conversion, making it difficult to effectively transform surgical data into a reusable knowledge system. Summary of the Invention

[0004] This application provides a surgical minimally invasive surgery data processing method and system based on AR technology, which is used to realize the correlation analysis and fusion processing of multi-source heterogeneous data during the operation, establish a complete technical link from data acquisition to knowledge extraction, and improve the safety and intelligence level of surgery.

[0005] In the first aspect, the present application provides a method for processing surgical minimally invasive surgery data based on AR technology, which includes: performing three-dimensional reconstruction processing on medical imaging data, performing lesion segmentation on the reconstructed data, processing the segmented data through feature extraction, and obtaining three-dimensional feature point data; using the three-dimensional feature point data to collect and process the position of surgical instruments, spatially mapping the collected position data, and aligning and fusing the mapped data to form spatial positioning data; reconstructing light field data based on the three-dimensional feature point data and the spatial positioning data, deforming the reconstructed data, and combining the processed data with real-scene data to obtain augmented reality data; performing operation trajectory analysis based on the augmented reality data, performing path planning on the analysis data, and obtaining optimal path data through optimization processing; monitoring the optimal path data in real time, performing risk analysis on the monitoring data, and performing early warning processing on the analysis results through threshold judgment to obtain risk data; extracting surgical rules using the risk data, performing multi-dimensional analysis on the rule data, establishing a mapping relationship between data through association processing, and forming target knowledge data.

[0006] In a second aspect, the present application provides a minimally invasive surgical data processing system based on AR technology, the minimally invasive surgical data processing system based on AR technology comprising:

[0007] The reconstruction module is used to perform three-dimensional reconstruction on medical image data, segment lesions on the reconstructed data, process the segmented data through feature extraction, and obtain three-dimensional feature point data;

[0008] An acquisition module is used to acquire and process the position of the surgical instrument using the three-dimensional feature point data, spatially map the acquired position data, and register and fuse the mapped data to form spatial positioning data;

[0009] a processing module, configured to reconstruct light field data based on the three-dimensional feature point data and the spatial positioning data, perform deformation processing on the reconstructed data, and combine the processed data with the real scene data to obtain augmented reality data;

[0010] An analysis module is used to analyze the operation trajectory based on the augmented reality data, perform path planning on the analyzed data, and obtain optimal path data through optimization processing;

[0011] A monitoring module is used to monitor the optimal path data in real time, perform risk analysis on the monitoring data, perform early warning processing on the analysis results through threshold judgment, and obtain risk data;

[0012] A module is established to extract surgical rules using the risk data, perform multi-dimensional analysis on the rule data, establish a mapping relationship between the data through association processing, and form target knowledge data.

[0013] The technical solution provided by this application achieves significant technical effects through data processing in multiple stages: By performing 3D reconstruction and lesion segmentation on medical imaging data, and combining feature extraction technology to obtain 3D feature point data, precise positioning and feature expression of the surgical target area are achieved; 3D feature point data is used to collect and spatially map the position of surgical instruments in real time, and spatial positioning data is generated through registration and fusion, ensuring the accuracy and real-time positioning of surgical instruments; light field data is reconstructed based on 3D feature point data and spatial positioning data, combined with deformation processing and scene fusion technology, to achieve a natural presentation of augmented reality scenes; operation trajectory analysis and path planning are performed based on augmented reality data, and optimal path data is obtained through optimization processing, improving the accuracy and safety of surgical operations; real-time monitoring and risk analysis of the optimal path data are carried out, and early warning processing is implemented through threshold judgment to generate risk data, enhancing the safety of the surgical process; finally, the risk data is used to extract surgical rules, conduct multi-dimensional analysis, and establish data mapping relationships to form target knowledge data, achieving effective accumulation of surgical experience and knowledge transformation. This complete technical chain from data acquisition, processing, analysis to knowledge extraction is realized, significantly improving the accuracy, safety, and intelligent level of minimally invasive surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 This is a schematic diagram of an embodiment of a method for processing minimally invasive surgical data based on AR technology in an embodiment of the present application;

[0016] Figure 2 This is a schematic diagram of an embodiment of a minimally invasive surgical data processing system based on AR technology in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The embodiments of the present application provide a method and system for processing surgical minimally invasive surgery data based on AR technology. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0018] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In the embodiment of the present application, an embodiment of the method for processing minimally invasive surgical data based on AR technology includes:

[0019] Step S101: Perform three-dimensional reconstruction on the medical image data, perform lesion segmentation on the reconstructed data, process the segmented data through feature extraction, and obtain three-dimensional feature point data;

[0020] Step S102: using the three-dimensional feature point data to collect and process the position of the surgical instrument, spatially mapping the collected position data, and registering and fusing the mapped data to form spatial positioning data;

[0021] Step S103: reconstructing light field data based on the three-dimensional feature point data and the spatial positioning data, performing deformation processing on the reconstructed data, and combining the processed data with the real scene data to obtain augmented reality data;

[0022] Step S104: Analyze the operation trajectory based on the augmented reality data, perform path planning on the analyzed data, and obtain the optimal path data through optimization processing;

[0023] Step S105: Monitor the optimal path data in real time, perform risk analysis on the monitored data, perform early warning processing on the analysis results through threshold judgment, and obtain risk data;

[0024] Step S106: extract surgical rules using risk data, perform multi-dimensional analysis on the rule data, establish mapping relationships between data through association processing, and form target knowledge data.

[0025] It is understandable that the execution subject of this application can be a surgical minimally invasive surgery data processing system based on AR technology, or a terminal or server, which is not limited here. The embodiment of this application is described by taking the server as the execution subject as an example.

[0026] Specifically, a three-dimensional reconstruction process is performed on medical imaging data. Medical imaging data generated by CT or MRI is received and grayscale processing is performed on the original two-dimensional slice images. The grayscale processing stage converts the brightness value of each pixel into a standardized value. This converted data is then used for subsequent three-dimensional reconstruction. During the three-dimensional reconstruction process, the processed two-dimensional slice data is stacked in space and spatially reconstructed based on voxel density. After reconstruction, the data is segmented to identify the target area from surrounding normal tissue based on differences in image density. During segmentation, the boundary points of the lesion area are first marked, and then a region growing algorithm is used to completely delineate the entire lesion area. Feature extraction is performed on the segmented data, primarily including edge features, texture features, and morphological features. Edge features reflect the contour information of the lesion area, texture features characterize the internal structural characteristics of the lesion tissue, and morphological features describe the geometric shape of the lesion area. Through combined analysis of these features, three-dimensional feature point data of the lesion area is obtained. The acquired three-dimensional feature point data is used to acquire and process the position of the surgical instrument. During the acquisition process, marker points with a specific geometric distribution are first placed on the surface of the surgical instrument. During the acquisition and processing phase, optical sensors track markers in real time, recording their coordinates in three-dimensional space. This acquired position data then undergoes spatial mapping, converting the positional information from the instrument coordinate system to the patient coordinate system. This mapping process requires establishing a transformation relationship between the two coordinate systems, including the calculation of rotation matrices and translation vectors. The mapped data undergoes registration and fusion, which minimizes the spatial distance error between the two sets of data to achieve precise alignment. The fusion process integrates the registered data to form unified spatial positioning data.

[0027] Light field data is reconstructed based on 3D feature point data and spatial positioning data. The light field data reconstruction process aligns the two sets of data within the same coordinate system, then constructs the light field distribution of the surgical scene based on light propagation theory. The reconstruction process considers the emission, propagation, and reception characteristics of light, as well as the reflective properties of various surfaces in the scene. The reconstructed data undergoes deformation processing, primarily targeting the deformation characteristics of soft tissue during surgical procedures. This process includes building a deformation model, calculating deformation parameters, and predicting tissue deformation trends. The processed data is combined with real-world data, taking into account factors such as viewpoint changes, lighting conditions, and occlusions. Image fusion algorithms enable seamless integration of virtual information and the real scene, generating augmented reality data. Surgical trajectory analysis is performed based on the augmented reality data. Trajectory analysis first decomposes the motion path of the surgical instrument and extracts key motion features. Path planning is then performed on the resulting data, taking into account the constraints of the surgical space, including avoiding critical anatomical structures and maintaining a safe operating distance. Path planning utilizes a multi-objective optimization approach, considering multiple objectives such as path length, operational safety, and energy consumption. The optimal path data is obtained through optimization processing. The optimization process adopts an iterative calculation method to continuously adjust the path parameters until all constraints are met and the optimal goal is achieved.

[0028] Optimal path data is monitored in real time. The monitoring process includes real-time collection and analysis of parameters such as position deviation, velocity variation, and acceleration fluctuations during path execution. Risk analysis is performed on the monitored data, primarily examining the relationship between the current operational status and pre-set safety thresholds. Threshold judgment is used to assess the analysis results for early warning. When a monitored parameter exceeds a pre-set threshold, the system generates a corresponding level of warning information and presents a risk assessment result. This early warning process utilizes a multi-level warning mechanism, generating warning signals of varying levels based on the degree of risk, thereby generating risk data. Finally, surgical patterns are extracted from this risk data. Pattern extraction begins with time series analysis of the risk data to identify the occurrence patterns and temporal characteristics of risk events. The extracted pattern data undergoes multi-dimensional analysis, encompassing spatial, temporal, and operational dimensions. Data mapping is established through association processing, which examines the interactions and dependencies between different data features. This generates target knowledge data, encompassing multiple aspects of surgical experience, risk prevention measures, and emergency response plans.

[0029] For example, consider processing patient CT imaging data. The raw CT data consists of multiple slice images, each with a resolution of 512×512 pixels and a grayscale value range of 0–4095. Using a 3D reconstruction algorithm, these slices are reconstructed into a 3D model. Approximately 2,000 feature points in the tumor region are extracted during the reconstruction process, accurately describing the tumor's spatial morphology and location. During the surgical procedure, position data is obtained by tracking surgical instruments at a sampling frequency of 60 Hz, achieving submillimeter accuracy. Combining the patient's anatomy and the spatial position of surgical instruments, the system generates an augmented reality view, overlaying virtual navigation information onto the real-time surgical field of view. Based on this information, path planning is performed. The optimal path obtained avoids critical blood vessels while ensuring the shortest possible surgical path. During the procedure, a real-time monitoring system tracks the instrument's position and issues a risk warning if deviation from the pre-set path exceeds a safety threshold. By analyzing data from the entire surgical process, surgical patterns are extracted and a knowledge base is generated.

[0030] In the embodiments of the present application, significant technical effects are achieved through data processing in multiple stages: By performing three-dimensional reconstruction and lesion segmentation on medical image data, and combining feature extraction technology to obtain three-dimensional feature point data, precise positioning and feature expression of the surgical target area are achieved; the three-dimensional feature point data is used to collect and spatially map the position of surgical instruments in real time, and spatial positioning data is generated through registration and fusion, ensuring the accuracy and real-time positioning of surgical instruments; light field data is reconstructed based on the three-dimensional feature point data and spatial positioning data, and deformation processing and scene fusion technology are combined to achieve a natural presentation of augmented reality scenes; operation trajectory analysis and path planning are performed based on augmented reality data, and optimal path data is obtained through optimization processing, thereby improving the accuracy and safety of surgical operations; real-time monitoring and risk analysis of the optimal path data are carried out, and early warning processing is implemented through threshold judgment to generate risk data, thereby enhancing the safety of the surgical process; finally, the risk data is used to extract surgical rules, conduct multi-dimensional analysis, and establish data mapping relationships to form target knowledge data, thereby effectively accumulating surgical experience and transforming knowledge. This complete technical chain from data acquisition, processing, analysis to knowledge extraction is achieved, significantly improving the accuracy, safety, and intelligent level of minimally invasive surgery.

[0031] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0032] (1) Project the grayscale values in the medical imaging data into three-dimensional space and obtain the initial voxel data through pixel density threshold processing;

[0033] (2) Perform region growing processing on the initial voxel data, separate the target area according to the grayscale gradient value, and obtain separated and reconstructed data;

[0034] (3) The separated and reconstructed data are analyzed for connected domains through region marking, and the boundaries of the connected domains are extracted to form reconstructed data;

[0035] (4) The reconstructed data is subjected to regional segmentation to extract the lesion boundary, and feature enhancement processing is performed based on the boundary curvature to obtain lesion segmentation data;

[0036] (5) Perform edge detection on the lesion segmentation data, extract key point coordinates based on the detection results, and obtain segmentation data;

[0037] (6) Calculate the feature point density of the segmented data, filter the feature points through spatial distance constraints, and obtain three-dimensional feature point data.

[0038] Specifically, two-dimensional slice images generated by CT or MRI are processed. Each pixel in the slice image contains a specific grayscale value, which reflects the density characteristics of different tissues. The grayscale value information of these two-dimensional slices is projected into three-dimensional space, taking into account the spacing between slices and the actual physical size of the pixels. The projected data is filtered by setting a density threshold. The density threshold is selected based on the attenuation characteristics of different tissues. For example, bone tissue, soft tissue, and pathological tissue have different density ranges. After threshold processing, the resulting initial voxel data retains the basic characteristics of the tissue structure. Region growing is performed on this initial voxel data. Region growing is a process that starts from a seed point and gradually incorporates surrounding voxels with similar characteristics into the target region. This process first calculates the grayscale gradient value around each voxel point. The gradient value reflects the severity of the grayscale change. Based on the changing characteristics of the gradient value, different tissue regions are separated. During the separation process, growth criteria are set, including grayscale value similarity and spatial continuity. When adjacent voxels meet the growing criteria, they are incorporated into the current region, thus obtaining separated and reconstructed data.

[0039] Connected domain analysis is performed on the separated reconstructed data. Connected domain analysis identifies sets of interconnected voxels in three-dimensional space and numbers each independent connected region using a labeling algorithm. During the labeling process, a recursive scan is used to examine each voxel's 26 neighboring points to determine whether they belong to the same connected region. Boundary extraction is performed on the labeled connected regions. Boundary extraction determines whether a voxel is a boundary point by examining its neighborhood features, thus generating the reconstructed data.

[0040] The reconstructed data is subjected to lesion boundary extraction. The data is processed using a regional segmentation method, which is based on the grayscale features, texture features, and morphological features of the image. During the boundary extraction process, the curvature value of the boundary point is calculated, and the curvature value reflects the local geometric features of the boundary. The boundary curvature is subjected to feature enhancement processing, which includes normalization and highlighting of the curvature value to obtain lesion segmentation data. Edge detection is performed on the lesion segmentation data. Edge detection is mainly based on the gradient features of the grayscale value, and edges are detected by calculating the gradient operator in three-dimensional space. During the detection process, the gradient amplitude and direction of each point are first calculated, and then non-edge points are removed by non-maximum suppression. Finally, the edge points are determined by double threshold processing. The coordinates of key points are extracted based on the detected edge points. The selection of key points is based on the curvature, connectivity and other features of the edge points to form segmentation data.

[0041] The process of feature point density calculation can be expressed by the following formula:

[0042]

[0043] in: Indicates the density value of the feature point at the coordinate point (x, y, z); Represents the weight coefficient of the i-th feature point; Represents the position vector of the current calculation point; Represents the position vector of the i-th feature point; represents the standard deviation parameter of the Gaussian kernel function; n represents the total number of feature points.

[0044] For example, when processing CT image sequences, the grayscale value distribution of each CT slice reflects the attenuation characteristics of different tissues. The grayscale value range of liver tissue is significantly different from that of surrounding tissues, while the tumor area exhibits grayscale characteristics different from normal liver tissue. The initial three-dimensional voxel model is constructed through grayscale value projection. During the region growth process, the feature points of the tumor area are selected as seed points, and the region is separated based on the grayscale gradient value, which can accurately delineate the boundaries of the tumor. Connected domain analysis ensures the integrity of the segmented area, and the precise contour of the tumor is obtained through boundary extraction. Feature enhancement processing highlights the features of the tumor boundary, and the most representative three-dimensional feature points are screened out through feature point density calculation. These feature points accurately describe the spatial morphological characteristics of the tumor.

[0045] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0046] (1) Project the three-dimensional feature point data into the surgical space, calibrate the projection results, and obtain the calibrated coordinate data;

[0047] (2) Optically track the position of the surgical instrument and perform spatial fixed-point processing on the tracking data to obtain fixed-point data;

[0048] (3) Convert the position parameters in the fixed-point data into three-dimensional space coordinates, perform noise filtering on the coordinate data, and form the collected position data;

[0049] (4) Perform coordinate system transformation on the collected position data, project the transformed data into the calibration space, and obtain mapping data;

[0050] (5) Calculate the overlap between the mapping data and the calibration coordinate data, perform error compensation on the calculation results, and form the registration data;

[0051] (6) Perform spatial registration processing on the registration data, perform multi-dimensional fusion of the registration parameters, and obtain spatial positioning data.

[0052] Specifically, the process of projecting 3D feature point data into the surgical space involves coordinate transformation and calibration. 3D feature point data contains key location information for the surgical target area, and this data needs to be transformed from image space into the actual surgical operating space. The projection process uses ray tracing to project each feature point along a specific direction into the surgical space. To calibrate the projection results, a calibration plate with an array of markers at known spacings is placed in the surgical space. By identifying the correspondence between the actual and theoretical positions of these markers, a coordinate transformation matrix is established. The calibration process utilizes repeated measurements at multiple angles and locations, optimizing the calibration parameters using the least squares method to generate the calibration coordinate data. Optical tracking of surgical instrument positions is achieved by attaching reflective markers to the instrument surface. These markers have a specific geometric distribution pattern, making them easier for tracking cameras to identify. During optical tracking, multiple high-speed cameras simultaneously capture the spatial positions of the markers, and their 3D coordinates are calculated using triangulation. The tracking data undergoes spatial point processing, including outlier removal, trajectory smoothing, and motion delay compensation. Fixed-point processing uses a median filter to remove transient noise and a Kalman filter to smooth the motion trajectory, obtaining stable fixed-point data. To convert fixed-point data into three-dimensional coordinates, the local coordinate system of the markers must first be determined. The position parameters of each marker include its relative position and orientation in the instrument's local coordinate system. These local coordinates are then transformed into the global surgical space coordinate system using a rigid body transformation matrix. The transformed data often contain measurement noise, necessitating noise filtering. This filtering process utilizes a Butterworth low-pass filter with a cutoff frequency set based on the characteristic frequency of the surgical procedure to eliminate high-frequency noise while retaining valid motion information. The filtered data serves as the acquired position data for subsequent processing. When performing coordinate system transformation on the acquired position data, the mapping relationship between different reference frames must be considered. While surgical instrument motion is typically described in its native coordinate system, surgical navigation must be performed in the patient coordinate system. Therefore, a transformation matrix must be established between the two coordinate systems, encompassing both rotation and translation. Quaternions are used in this transformation to avoid gimbal lock issues that can occur with Euler angles. The transformed data is mapped to the calibration space through projection transformation to form mapping data.

[0053] Overlap calculation is a key step in evaluating the accuracy of spatial registration, and its mathematical expression is as follows:

[0054]

[0055] in: Indicates the degree of overlap between two sets of point cloud data A and B; Represents the weight coefficient between the i-th point and the j-th point; Represents the position vector of the i-th point in point cloud A; Represents the position vector of the jth point in point cloud B; represents the bandwidth parameter of the Gaussian kernel function; N represents the number of points in the point cloud.

[0056] The overlap calculation results are used to assess the registration quality and are then compensated for errors. This compensation process considers both systematic and random errors. Systematic errors primarily arise from measurement deviations within the device itself, for which compensation parameters are obtained through pre-calibration. Random errors are estimated and corrected using statistical methods. The error-compensated data form the registered data. Spatial registration is then performed on the registered data, using an iterative closest point algorithm to precisely align the two sets of data. The registration process considers both position and orientation, and the iterations are controlled by setting a convergence threshold. Multidimensional fusion of registration parameters utilizes a weighted average method, with weights dynamically adjusted based on the reliability of the data in each dimension. The fused data becomes the spatial positioning data.

[0057] For example, three-dimensional feature points of the tumor region were extracted from preoperative CT reconstruction data and projected into the actual surgical space using the operating room's optical positioning equipment. During the calibration process, a calibration plate with 64 markers was used, achieving submillimeter calibration accuracy through multi-angle acquisition. Four reflective ball markers were installed on the surgical instrument to form a specific geometric configuration. During instrument movement, the optical tracking system captured the positions of the markers at a frequency of 60 frames per second, and the real-time position data of the instrument was obtained using a spatial fixed-point algorithm. After noise filtering of the raw tracking data, the position accuracy was improved to within 0.3 mm. The processed position data was projected into the patient's coordinate system and aligned with the preoperatively planned target. The alignment process evaluated the matching quality through overlap calculation. After multiple rounds of iterative optimization, the precise spatial positioning required for real-time intraoperative navigation was achieved.

[0058] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0059] (1) Perform spatial registration on the 3D feature point data and the spatial positioning data, and uniformly process the registration results through coordinate transformation to obtain unified coordinate data;

[0060] (2) Perform ray tracing on the unified coordinate data, convert the tracing results into brightness value distribution, and obtain the initial light field data;

[0061] (3) Perform density calculation on the initial light field data and interpolate the calculation results to form interpolated light field data;

[0062] (4) Calculate the tissue deformation of the interpolated light field data, perform numerical iteration on the deformation parameters, and obtain the reconstructed data;

[0063] (5) Perform surface fitting on the reconstructed data and process the fitting results by calculating the shape variables to form processed data;

[0064] (6) Perform depth alignment on the processed data and spatially fuse the alignment results with the real scene data to obtain augmented reality data.

[0065] Specifically, the 3D feature point data and spatial positioning data are first processed. The 3D feature point data is derived from preoperative medical image reconstruction, while the spatial positioning data represents the position of the surgical instruments in real space. The registration process uses a rigid body transformation to convert both sets of data into a common coordinate system. This coordinate transformation involves rotation and translation, optimizing the transformation parameters by minimizing the distance between corresponding points in the two sets. The transformation also accounts for differences in coordinate systems across different devices, unifying all data into the global coordinate system of the operating room to generate unified coordinate data. Ray tracing is a key step in converting this unified coordinate data into a light field representation. Starting from the virtual camera position, the tracing process sends rays along a preset direction, calculating intersections with object surfaces in the scene. Each ray's intersection with an object surface records information such as its position, normal, and material properties. This information is used to calculate the propagation characteristics of the light, including effects such as reflection, refraction, and scattering. The interaction of light with the object surface generates brightness values, which, according to their spatial distribution, form the initial light field data.

[0066] When calculating the density of the initial light field data, it is necessary to evaluate the light field density distribution at each sampling point in space. This density calculation takes into account the spatial distribution characteristics of light and the energy attenuation law. The calculated density distribution may exhibit uneven sampling, necessitating interpolation to compensate for missing data points. This interpolation process uses radial basis functions to continuum the sparse data points in space, generating interpolated light field data.

[0067] The mathematical expression of tissue deformation calculation is as follows:

[0068]

[0069] in: Represents the deformation field at a spatial point (x, y, z); Represents the weight coefficient of the kth control point; represents radial basis function; Represents the position vector of the current calculation point; represents the position vector of the kth control point; represents the affine transformation parameters; Represents the spatial coordinate component; represents the translation parameter; K represents the total number of control points.

[0070] Deformation calculations are based on physical models, taking into account the elastic properties of the tissue and the effects of external forces. Deformation parameters are solved using an iterative optimization method, with each iteration updating the node positions and internal stress distribution until an equilibrium state is reached and reconstructed data is obtained. A three-dimensional spline interpolation method is used for surface fitting of the reconstructed data. The fitting process first constructs a control point grid and then optimizes the fitting parameters by minimizing the surface energy functional. Deformation calculations are based on the local deformation characteristics of the fitted surface, including indicators such as curvature change and area change. The processed data retains the key features of tissue deformation.

[0071] The processed data is then registered with the real-world image. Depth alignment, based on the principle of parallax matching, adjusts the spatial relationship by comparing the depth information of virtual and real-world data. The spatial fusion process considers factors such as lighting conditions and occlusion, using a physically based rendering method to seamlessly integrate virtual information and real-world scenes to generate augmented reality data.

[0072] For example, during liver tumor surgery, the 3D tumor model reconstructed using preoperative CT scans is spatially registered with the real-time position data of surgical instruments. The registration process optimizes the transformation matrix to keep the registration error within the millimeter range. Ray tracing is then performed, emitting virtual rays from different angles in the surgical field of view, intersecting them with the reconstructed model to obtain the light field distribution. The density calculation stage evaluates the spatial distribution characteristics of the light field and supplements data from sparsely sampled areas through interpolation. When processing soft tissue deformation, a deformation field containing hundreds of control points is established, and iterative calculations are used to simulate the deformation characteristics of liver tissue under respiratory motion and surgical manipulation. The surface fitting stage uses high-order spline functions to describe the tissue surface, accurately capturing changes in local details during deformation. Finally, through depth alignment and spatial fusion, the virtual navigation information is superimposed on the real-time surgical field of view, achieving precise surgical navigation.

[0073] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0074] (1) Segment the augmented reality data into time series, track the feature points of the segmented data, and obtain the spatiotemporal state data;

[0075] (2) Perform multi-level trajectory decomposition on the spatiotemporal state data, extract trajectory points from the decomposition results, and form a trajectory point sequence;

[0076] (3) Fit the trajectory point sequence with a polynomial curve, perform tangent analysis on the fitting results, and obtain analysis data;

[0077] (4) Perform collision detection calculations on the analyzed data, filter the detection results according to the safety distance, and form candidate path data;

[0078] (5) Calculate the path length of the path candidate data, solve the energy function of the calculation results, and obtain the path score data;

[0079] (6) Calculate multi-objective weights on the path scoring data, perform multiple iterative comparisons on the path weights, and form the optimal path data;

[0080] (7) Perform speed planning on the optimal path data, calculate the acceleration constraints on the planning results, and obtain motion control data;

[0081] (8) Smooth the trajectory of the motion control data, perform gradient optimization on the smoothing results, and update the optimal path data.

[0082] Specifically, the augmented reality data is segmented and processed temporally. Augmented reality data contains a fusion of virtual information and real scenes and requires chronological segmentation. Temporal segmentation uses a fixed time window approach to divide the continuous data stream into several time segments. Within each time segment, a feature point detection algorithm extracts key points. These feature points primarily include surgical instrument markers, anatomical structure features, and the location information of virtual markers. A tracking algorithm tracks the motion of these feature points in adjacent time segments, recording changes in their position and velocity to generate spatiotemporal state data. This spatiotemporal state data contains kinematic information such as the feature points' three-dimensional coordinates, velocity, and acceleration. Multi-level trajectory decomposition of the spatiotemporal state data is performed to analyze the hierarchical structure of surgical operations. The decomposition process utilizes wavelet transforms to decompose complex surgical operation trajectories into motion components at different scales. The large scale reflects the main surgical operation trajectory, the medium scale captures detailed movements, and the small scale corresponds to fine-tuning of the hand. Feature point extraction is performed on the decomposition results at each scale. Key points in the motion trajectory are selected based on features such as curvature and velocity changes. These key points are connected in chronological order to form a trajectory point sequence, which reflects the complete process of the surgical operation.

[0083] When fitting a curve to a sequence of trajectory points, a polynomial approximation method is used. A polynomial function of appropriate order is selected in the fitting process, and the polynomial coefficients are determined by the least squares method. The fitted curve needs to meet the requirements of smoothness and continuity. Tangent analysis is performed on the fitting curve, and the tangent direction and curvature at each point are calculated. These parameters reflect the change in direction and smoothness of the movement, forming analysis data. Collision detection calculation is an important part of ensuring surgical safety. For each trajectory point in the analysis data, it is necessary to check its spatial relationship with the surrounding anatomical structures. The detection process uses a hierarchical bounding box method, first performing a rough detection, and then accurately calculating the area where collisions may occur. The detection results are screened according to the preset safety distance, and path points that are too close to important structures are eliminated. Path points that meet safety requirements are retained to form path candidate data.

[0084] Path length calculation is based on the cumulative distances between adjacent points in the candidate path data. The calculation process considers the smoothness and continuity of the path to avoid sharp turns. The results are evaluated using an energy function that incorporates multiple evaluation metrics, including path length, curvature change, and operational difficulty. Numerical optimization methods are used to find the optimal value, resulting in path score data. Multi-objective weight calculation is a comprehensive evaluation of path score data. The calculation requires balancing multiple objectives, including minimizing path length, maximizing safety distance, and minimizing operational difficulty. The weight coefficients of each objective are adjusted iteratively. Each iteration compares the comprehensive scores of different paths, selecting the path with the best overall performance and generating the optimal path data.

[0085] Velocity planning is the process of dynamically optimizing optimal path data. The planning process considers the motion characteristics of the surgical instrument, including constraints such as maximum velocity and acceleration limits. Acceleration constraints are calculated on the planning results to ensure smooth and controllable motion, generating motion control data that meets dynamic requirements.

[0086] Trajectory smoothing uses cubic spline interpolation to make discrete points in the motion control data continuous. The interpolation process must ensure the continuity of position, velocity, and acceleration at the nodes to avoid sudden changes during motion. A sliding window method is used during smoothing to optimize local trajectory segments, and the window size is adjusted according to the precision requirements of the surgical operation. After obtaining the smoothed trajectory, the trajectory is further refined using a gradient optimization method. The gradient optimization process considers multiple aspects of the trajectory, including its spatial distribution, motion characteristics, and energy consumption. Optimization objectives include minimizing the rate of change of the trajectory's curvature, maintaining smoothness of motion, and reducing mechanical vibration. The optimization process uses an iterative approach. Each iteration calculates the gradient information of the current trajectory and makes fine adjustments along the gradient direction until convergence conditions are reached. The optimization results are used to update the optimal path data. The update process ensures that the new path data meets both smoothness and dynamic constraints, providing a more accurate navigation reference for surgical operations.

[0087] For example, augmented reality scenes are processed. The scene includes a 3D reconstructed model of the lesion area, the real-time position of the surgical instruments, and planned approach markers. This data is time-series segmented, with data collected every 33 milliseconds to form a continuous time series. During feature point tracking, the motion trajectory of the surgical instrument tip and important anatomical landmarks is tracked simultaneously. In the multi-level trajectory decomposition stage, different levels of operational actions, including instrument introduction, fine manipulation, and instrument withdrawal, are identified. The curve fitting process generates a smooth operational trajectory, with each trajectory point containing position and direction information. During collision detection, a safety distance of 5 mm from important blood vessels and a safety distance of 3 mm from brain tissue are set as constraints. Path planning takes into account multiple factors such as the risk of tissue damage and operational convenience, and the surgical path is obtained through multiple iterative optimizations. During the speed planning stage, the maximum movement speed and acceleration of the instrument are limited to ensure the smoothness and safety of the surgical operation.

[0088] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0089] (1) Sample the optimal path data according to the time window, calculate the position deviation of the sampling points, and form real-time deviation data;

[0090] (2) Perform cumulative error statistics on real-time deviation data, perform trend prediction on the statistical results, and obtain monitoring data;

[0091] (3) Decompose the monitoring data into multidimensional features, perform correlation analysis on the decomposition results, and obtain the risk feature sequence;

[0092] (4) Score the risk level of the risk feature sequence and add up the scores to form the analysis results;

[0093] (5) Compare the analysis results with the preset threshold sequence, classify the risk level of the exceeded data, and obtain early warning data;

[0094] (6) Conduct spatiotemporal correlation analysis on the early warning data, perform comprehensive probability calculation on the correlation results, and form risk data.

[0095] Specifically, the optimal path data is sampled and processed. The optimal path data contains the ideal motion trajectory of the surgical instrument. The sampling process discretizes this data according to a fixed time window. The size of the time window is set according to the precision requirements of the surgical operation and is usually in the millisecond level. At each sampling point, two sets of data, the actual position and the ideal position, are recorded simultaneously. The position deviation is calculated by calculating the Euclidean distance between these two sets of data. The calculation of the position deviation takes into account both translation and rotation in three-dimensional space to form real-time deviation data. Cumulative error statistics of real-time deviation data are an important means of monitoring long-term trends. The statistical process uses a sliding window method to calculate statistical characteristics such as the mean, variance, and maximum value of the deviation within a certain time range. These statistics reflect the stability and accuracy of the surgical operation. Trend prediction uses time series analysis methods to predict the deviation change trend over a period of time in the future by establishing an autoregressive model. The prediction results are combined with historical data to form monitoring data.

[0096] The decomposition process uses principal component analysis to project the raw data into different feature spaces. Feature spaces include multiple dimensions such as position deviation, velocity fluctuation, and acceleration change. Correlation analysis is performed on the decomposition results, and correlation coefficients between different dimensions are calculated to identify the main risk factors. The correlation analysis results are ranked by importance, and the most representative feature combinations are extracted to form a risk feature sequence. A multi-index evaluation method is used to score the risk feature sequence. The scoring process considers multiple aspects such as the severity, duration, and rate of change of the feature. A weight coefficient is assigned to each feature, and a weighted sum is used to obtain a comprehensive score. The scoring results are normalized to facilitate comparison across different situations. The numerical accumulation process takes into account time decay, with more recent risk events being given greater weight to form the analysis results.

[0097] A multi-level threshold judgment method is used to compare the analysis results with a preset threshold sequence. The threshold sequence contains critical values corresponding to multiple risk levels, which are determined based on surgical safety standards and expert experience. Data exceeding the threshold is classified into risk levels. This classification process takes into account the urgency and degree of harm of the risk, classifying the risks into different levels and generating early warning data. When performing spatiotemporal correlation analysis on early warning data, it is necessary to consider the distribution characteristics of risk events in both time and space. Temporal correlation analysis focuses on the occurrence sequence and time interval of risk events, while spatial correlation analysis examines the distribution patterns of risk events in the surgical area. Correlation analysis uses graph theory to construct a correlation network of risk events. By calculating the importance indicators and connection strengths of nodes in the network, the propagation characteristics of risks are evaluated. Finally, through comprehensive probability calculation, the influence of multiple risk factors is integrated to form risk data.

[0098] For example, during a spinal surgery, the trajectory of the surgical instrument is first monitored in real time. Position sampling is performed every 10 milliseconds, recording the deviation between the instrument's position and the preset path. During the cumulative error statistics phase, a 60-second sliding window is used to calculate statistical features, while also predicting deviation trends over the next 10 seconds. The feature decomposition process identifies key risk factors, including position deviation, posture error, and speed fluctuation. During the risk scoring phase, different weights are assigned to different types of deviations: position deviation is weighted at 0.4, posture error at 0.3, and speed fluctuation at 0.3. Three levels of warning thresholds are set: a comprehensive score exceeding 0.7 indicates high risk, between 0.4 and 0.7 indicates medium risk, and below 0.4 indicates low risk. Spatiotemporal correlation analysis revealed a significant correlation between position deviation and speed fluctuation when the instrument approaches the spine, directly influencing the determination of risk levels.

[0099] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0100] (1) Risk data is divided into time series according to the surgical stage, and the division results are subjected to multi-scale time window sliding. Variance analysis and peak detection are performed based on the data volatility within the sliding window. The risk change pattern is extracted through the data distribution characteristics, and the change trend is quantified to obtain risk characteristic data;

[0101] (2) Perform multi-level decomposition on the risk characteristic data, conduct periodic analysis on the decomposed characteristic data through the autocorrelation function, divide the data into groups based on the periodic results, perform pattern matching based on the similarity between data groups, perform numerical statistics and normalization on the matching results to form regular data;

[0102] (3) Construct a multidimensional feature space for regular data, perform density distribution analysis on the data points in the feature space, cluster the data according to the density distribution results, calculate the dimension correlation based on the distribution characteristics of the cluster centers, sort the calculation results by feature importance, and obtain dimension correlation data;

[0103] (4) Perform multi-layer cross-validation on dimension correlation data, perform uncertainty analysis on the verification results through information entropy calculation, evaluate the reliability of the data according to the uncertainty level, quantitatively describe the data correlation strength based on the evaluation results, and numerically normalize the description results to obtain correlation strength data;

[0104] (5) Construct a relational network based on the correlation strength data, calculate the weight of the connection strength between network nodes, extract the key paths based on the weight distribution, characterize the data transmission characteristics through path analysis, perform mathematical modeling on the characterization results, and form mapping relational data;

[0105] (6) Perform semantic structuring on the mapping relationship data, organize the structured results through hierarchical analysis, extract knowledge units according to the organizational structure, construct rules based on the logical relationship between knowledge units, verify the validity of the construction results, and form the target knowledge data.

[0106] Specifically, the risk data is segmented and features extracted from the time series. The surgical stages are divided based on the different nature of the surgical procedure, including key phases such as incision, dissection, and suturing. Within each stage, sliding analysis is performed using multi-scale time windows, with window sizes ranging from seconds to minutes, to capture the characteristics of risk changes at different time scales. Within each sliding window, the variance of the data is calculated to measure the degree of risk fluctuation, while peak detection is performed to identify risk emergencies. Data distribution analysis includes the calculation of statistics such as mean, standard deviation, skewness, and kurtosis, which collectively describe the regularity of risk changes. Numerical fitting methods are used to quantify the changing trends and generate risk signature data. The risk signature data is then subjected to a multi-level decomposition process. The wavelet transform is used to decompose the data into different frequency components. Autocorrelation analysis is performed on the decomposed signature data, calculating the autocorrelation coefficients at different time delays to identify periodic patterns in the data. Based on the results of the periodicity analysis, the data is divided into groups, with data with similar periods grouped together. Within each data group, pattern matching is performed by calculating the similarity between data sequences using a dynamic time warping algorithm. The matching results are statistically analyzed and normalized using maximum and minimum values to generate regular data.

[0107] Construct a multidimensional feature space. In the feature space, each dimension represents a risk feature, and the distribution of data points reflects the relationship between different features. Perform density distribution analysis on the data points, and use the kernel density estimation method to calculate the data density in the space. Cluster the data according to the density distribution, and use the density peak clustering algorithm to determine the cluster center. By analyzing the distribution characteristics of the cluster centers, calculate the correlation between different dimensions. The correlation calculation is based on the mutual information criterion. Finally, sort the feature importance to obtain dimension correlation data. Divide the data set into multiple subsets, and use the K-fold cross-validation method to evaluate the stability of the correlation relationship. Calculate the information entropy of the verification results to evaluate the uncertainty level of the data. Based on the results of the uncertainty analysis, quantitatively evaluate the reliability of the data and quantitatively describe the data association strength. The description results are normalized to ensure comparability between different data and form association strength data.

[0108] Nodes in the network represent different risk factors, and edges represent the relationships between factors. The connection strengths between network nodes are weighted, with the weight values reflecting the importance of the relationships. By analyzing the weight distribution, critical paths in the network are extracted; these paths reflect the main channels for risk transmission. Data transmission characteristics are mathematically modeled, taking into account both direct and indirect interactions between nodes to form mapping relationship data. Semantic structuring is performed. Mapping relationship data is converted into understandable knowledge representations, and knowledge is organized using an ontology model. Hierarchical analysis is used to determine the subordinate relationships between knowledge units and establish a knowledge system. Based on the logical relationships between knowledge units, inference rules are constructed, encompassing causal relationships, constraints, and other aspects. The validity of the constructed rules is verified, including logical consistency checks and instance testing, to form the target knowledge data.

[0109] For example, in liver tumor surgery, the entire surgical process is divided into preoperative planning, navigation positioning, instrument operation, and suturing and closing stages. During the instrument operation phase, risk monitoring is performed using sliding windows of 30 seconds, 1 minute, and 5 minutes. Variance analysis revealed significant risk fluctuations during vascular separation procedures, and peak detection identified several high-risk events near critical vessels. Periodic analysis of these risk data revealed a cyclical risk variation pattern associated with respiratory motion. Feature space analysis, correlation analysis of features such as operation position, instrument posture, and movement speed, revealed a strong correlation between instrument posture and operation risk. Relationship network analysis identified the risk propagation chain of "improper posture-position deviation-tissue damage." These findings are organized into structured knowledge, forming risk prevention and control guidelines for liver surgery, including specific rules such as instrument posture control requirements and safe operation boundaries.

[0110] In a specific embodiment, the process of performing the multi-dimensional feature decomposition step on the monitoring data may specifically include the following steps:

[0111] (1) Decompose the monitoring data spatially through orthogonal transformation, calculate the main direction component based on the decomposed data, and obtain the characteristic component data;

[0112] (2) Using the time window to reconstruct the characteristic component data into a sequence, and extracting the fluctuation law to obtain the fluctuation characteristic data;

[0113] (3) Calculate the spectrum distribution based on the fluctuation characteristic data, classify the frequency bands according to the energy size, and form frequency domain characteristic data;

[0114] (4) Perform correlation analysis on the frequency domain feature data through the cross-correlation function, select relevant parameters based on the analysis results, and obtain correlation data;

[0115] (5) Establish feature significance index based on correlation data, extract key features through statistical testing, and obtain feature index data;

[0116] (6) Organize the feature index data based on time sequence, and form a risk feature sequence after reordering the data.

[0117] Specifically, spatial decomposition is performed. Orthogonal transformation projects high-dimensional data onto an orthogonal basis. Common methods include principal component analysis (PCA) and singular value decomposition (SVD). Monitoring data typically contains information in multiple dimensions, such as position, velocity, and attitude. Orthogonal transformation can identify the main directions of change in the data. During the decomposition process, the data is first centered and normalized. Then, the covariance matrix is calculated, and the principal directions are determined through eigenvalue decomposition. The principal directions are sorted according to the size of their corresponding eigenvalues. The eigenvectors corresponding to the larger eigenvalues represent the main directions of change in the data, thereby obtaining the characteristic component data. The time window reconstruction process uses a sliding window method. The characteristic component data is segmented into fixed time segments. There is overlap between adjacent windows, and the degree of overlap affects the continuity of the data. Within each time window, the data trend is analyzed to extract fluctuation patterns. Fluctuation patterns are extracted through multiple aspects, such as mean shift, variance change, and mutation point detection. Statistical analysis of these features generates fluctuation characteristic data.

[0118] Spectral distribution calculations utilize the Fourier transform method. Fluctuation feature data is converted from the time domain to the frequency domain, yielding the energy distribution of different frequency components. Spectral analysis identifies periodic characteristics and primary frequency components within the data. Frequency bands are classified based on energy, dividing the spectrum into high-energy, medium-energy, and low-energy regions corresponding to fluctuation features of varying scales, generating frequency domain feature data. Cross-correlation analysis assesses the correlation between different features. Cross-correlation functions are calculated pairwise on the frequency domain feature data to yield parameters such as correlation coefficients and time delays. Correlation analysis results reflect the dependencies and influence strength between features. Parameters are screened based on the magnitude of the correlation coefficient, retaining significantly correlated feature pairs to generate correlation data. Feature significance indices are developed based on statistical testing methods. Hypothesis tests are performed on the correlation data to assess the statistical significance of feature correlations. Significance tests employ t-tests or F-tests, with the significance of features determined based on the test results. Key features are identified by setting significance levels, generating feature index data.

[0119] Finally, the data is organized temporally. Feature index data is arranged chronologically to establish temporal relationships between features. The reordering process considers the causal relationships and logical order of features to ensure data coherence. After reordering, a risk feature sequence is formed.

[0120] For example, in a neurosurgery procedure, monitoring data includes information such as the three-dimensional position coordinates, posture angle, and operating force of surgical instruments. Orthogonal transformation revealed that instrument movement is primarily concentrated within a plane that correlates with the anatomical structure of the surgical target area. Time window analysis revealed significant periodic fluctuations in instrument movement, which are related to the surgeon's operating habits and the patient's circadian rhythm. Spectral analysis further confirmed this, revealing a significant energy peak in the 0.2-0.5 Hz frequency band, corresponding to the primary rhythm of the surgical procedure. Cross-correlation analysis revealed a significant positive correlation between instrument posture and operating force, with a time delay of approximately 200 milliseconds, reflecting the relationship between force control and posture adjustment. Statistical testing identified posture angle, movement speed, and operating force as key monitoring indicators. These indicators are organized chronologically to form a risk monitoring sequence.

[0121] In a specific embodiment, the process of performing correlation analysis on the frequency domain feature data through the cross-correlation function may specifically include the following steps:

[0122] (1) Establish a cross-correlation matrix based on the frequency domain feature data, normalize the matrix elements, and form correlation distribution data;

[0123] (2) Extract the eigenvalue sequence for the associated distribution data, sort the eigenvalues according to their numerical values, and obtain the feature sorting data;

[0124] (3) Perform threshold segmentation on the feature sorting data, calculate statistics through the segmentation interval, and obtain statistical feature data;

[0125] (4) Project the statistical feature data into the feature space, perform cluster analysis through distance measurement, and form feature cluster data;

[0126] (5) Construct a correlation coefficient matrix based on the feature clustering data, extract key parameters based on the matrix elements, and obtain parameter correlation data;

[0127] (6) Carry out correlation evaluation based on parameter correlation data, and comprehensively calculate the evaluation indicators to form correlation data.

[0128] Specifically, frequency-domain feature data contains energy distribution information across different frequency bands. Correlation coefficients are calculated for each pair of these data to form a symmetric matrix. Each element of the matrix represents the strength of the correlation between two frequency bands. Normalization of the matrix elements uses a maximum-minimum normalization method to normalize the values to the range [0, 1]. The normalized matrix reflects the relative strength of the correlation between the frequency bands, forming correlation distribution data.

[0129] Eigenvalue extraction is the process of performing eigendecomposition on correlated distribution data. By solving the characteristic equation, the eigenvalues and eigenvectors of the matrix are obtained. The eigenvalues reflect the variance of the data in each principal direction, while the eigenvectors represent these principal directions. The eigenvalues are arranged from large to small, and the eigenvectors corresponding to the larger eigenvalues represent the main pattern of change. The sorted eigenvalue sequence constitutes the feature-sorted data. The threshold segmentation process is to perform stratified processing on the feature-sorted data. Based on the distribution characteristics of the eigenvalues, appropriate thresholds are set to divide the data into different levels. Within each segmentation interval, statistics such as the mean, standard deviation, and kurtosis are calculated. These statistics describe the distribution characteristics of the data within that interval. By combining multiple statistics, statistical feature data is formed.

[0130] Feature space projection is the process of mapping statistical feature data into a multidimensional feature space. In feature space, each dimension represents a statistical feature, and the position of a data point reflects its feature combination. Cluster analysis is performed by calculating the Euclidean or Mahalanobis distance between data points. The clustering process uses K-means or hierarchical clustering algorithms to group similar data points into the same category, forming feature clusters.

[0131] The correlation matrix is constructed based on the feature clustering results. Within each cluster, the correlation coefficients between features are calculated to form a local correlation matrix. Meanwhile, the relationships between different clusters are considered to construct a global correlation matrix. Key parameters, including the magnitude, sign, and significance level of the correlation coefficient, are extracted from the matrix to obtain parameter association data.

[0132] The parameter correlation data is used as input and comprehensively evaluated using multiple evaluation metrics. These metrics include correlation strength, stability, and interpretability. These metrics are weighted, with the weights reflecting the importance of each metric. The result of this comprehensive calculation is the correlation data.

[0133] For example, in liver tumor surgery, frequency domain feature data includes the spectral characteristics of surgical instrument motion. When constructing a cross-correlation matrix, a significant correlation was found between the 0.2Hz-0.5Hz and 1Hz-2Hz frequency bands, reflecting the coordinated relationship between slow movement and rapid adjustment during surgical operations. Eigenvalue analysis revealed that the first three principal components explained the vast majority of the data variation, indicating that surgical operations exhibit distinct pattern characteristics. Cluster analysis identified three typical operation modes: fine dissection, tissue separation, and suturing. Within each operation mode, the instrument motion characteristics exhibited distinct correlation structures.

[0134] The above describes the surgical minimally invasive surgery data processing method based on AR technology in the embodiment of the present application. The following describes the surgical minimally invasive surgery data processing system based on AR technology in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of the minimally invasive surgical data processing system based on AR technology includes:

[0135] The reconstruction module 201 is used to perform three-dimensional reconstruction processing on the medical image data, perform lesion segmentation on the reconstructed data, process the segmented data through feature extraction, and obtain three-dimensional feature point data;

[0136] An acquisition module 202 is configured to acquire and process the position of the surgical instrument using the three-dimensional feature point data, spatially map the acquired position data, and register and fuse the mapped data to form spatial positioning data;

[0137] A processing module 203 is configured to reconstruct light field data based on the three-dimensional feature point data and the spatial positioning data, perform deformation processing on the reconstructed data, and combine the processed data with the real scene data to obtain augmented reality data;

[0138] An analysis module 204 is configured to analyze the operation trajectory based on the augmented reality data, perform path planning on the analyzed data, and obtain optimal path data through optimization processing;

[0139] The monitoring module 205 is used to monitor the optimal path data in real time, perform risk analysis on the monitoring data, and perform early warning processing on the analysis results through threshold judgment to obtain risk data;

[0140] The establishment module 206 is used to extract surgical rules using the risk data, perform multi-dimensional analysis on the rule data, establish a mapping relationship between the data through association processing, and form target knowledge data.

[0141] Through the collaborative efforts of the aforementioned components and the data processing of multiple links, significant technical results were achieved: By performing 3D reconstruction and lesion segmentation on medical imaging data, combined with feature extraction technology to obtain 3D feature point data, precise positioning and feature expression of the surgical target area were achieved; 3D feature point data was used to acquire and spatially map the position of surgical instruments in real time, and spatial positioning data was generated through registration and fusion, ensuring the accuracy and real-time performance of surgical instrument positioning; light field data was reconstructed based on 3D feature point data and spatial positioning data, combined with deformation processing and scene fusion technology, to achieve a natural presentation of augmented reality scenes; operation trajectory analysis and path planning were performed based on augmented reality data, and optimal path data was obtained through optimization processing, improving the accuracy and safety of surgical operations; real-time monitoring and risk analysis of the optimal path data were carried out, and early warning processing was implemented through threshold judgment to generate risk data, enhancing the safety of the surgical process; finally, risk data was used to extract surgical rules, conduct multi-dimensional analysis, and establish data mapping relationships to form target knowledge data, achieving the effective accumulation of surgical experience and knowledge transformation. This complete technical chain from data acquisition, processing, analysis to knowledge extraction has been realized, significantly improving the accuracy, safety, and intelligent level of minimally invasive surgery.

[0142] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for processing surgical minimally invasive surgery data based on AR technology, characterized in that: The method for processing minimally invasive surgical data based on AR technology includes: Perform three-dimensional reconstruction on medical imaging data, segment lesions on the reconstructed data, process the segmented data through feature extraction, and obtain three-dimensional feature point data; The three-dimensional feature point data is used to collect and process the position of the surgical instrument, the collected position data is spatially mapped, and the mapped data is registered and fused to form spatial positioning data; Reconstructing light field data based on the three-dimensional feature point data and the spatial positioning data, performing deformation processing on the reconstructed data, and combining the processed data with the real scene data to obtain augmented reality data; Performing operation trajectory analysis on the augmented reality data, performing path planning on the analyzed data, and obtaining optimal path data through optimization processing, including: performing time series segmentation on the augmented reality data, tracking feature points on the segmented data, and obtaining spatiotemporal state data; performing multi-level trajectory decomposition on the spatiotemporal state data, extracting trajectory points from the decomposition results, and forming a trajectory point sequence; performing polynomial curve fitting on the trajectory point sequence, and performing tangent analysis on the fitting results, and obtaining the analyzed data; performing collision detection calculation on the analyzed data, screening the detection results according to the safety distance, and forming path candidate data; performing path length calculation on the path candidate data, solving the calculation result by energy function, and obtaining path scoring data; performing multi-target weight calculation on the path scoring data, and performing multiple iterative comparisons on the path weights, and forming the optimal path data; performing speed planning on the optimal path data, and performing acceleration constraint calculation on the planning result, and obtaining motion control data; performing trajectory smoothing processing on the motion control data, and performing gradient optimization on the smoothing result, and updating the optimal path data; The optimal path data is monitored in real time, risk analysis is performed on the monitoring data, and early warning processing is performed on the analysis results through threshold judgment to obtain risk data, including: sampling the optimal path data according to a time window, calculating the position deviation of the sampling points, and forming real-time deviation data; performing cumulative error statistics on the real-time deviation data, and performing trend prediction on the statistical results to obtain the monitoring data; performing multi-dimensional feature decomposition on the monitoring data, performing correlation analysis on the decomposition results, and obtaining a risk feature sequence; performing a risk score on the risk feature sequence, and numerically accumulating the score results to form the analysis result; numerically comparing the analysis result with a preset threshold sequence, and classifying the over-limit data into risk levels to obtain early warning data; performing spatiotemporal correlation analysis on the early warning data, and performing comprehensive probability calculation on the correlation results to form the risk data; The risk data is used to extract surgical rules, and multi-dimensional analysis is performed on the rule data. A mapping relationship between the data is established through association processing to form target knowledge data.

2. The method for processing minimally invasive surgical data based on AR technology according to claim 1, characterized in that: The three-dimensional reconstruction of medical image data, segmentation of lesions on the reconstructed data, and processing of the segmented data by feature extraction to obtain three-dimensional feature point data include: Projecting the grayscale values in the medical image data into a three-dimensional space and obtaining initial voxel data by pixel density threshold processing; Performing region growing processing on the initial voxel data, separating the target region according to the grayscale gradient value, and obtaining separated and reconstructed data; Performing connected domain analysis on the separated and reconstructed data through region marking, and performing boundary extraction processing on the connected domain to form reconstructed data; extracting the lesion boundary from the reconstructed data using a regional segmentation method, performing feature enhancement processing based on the boundary curvature, and obtaining lesion segmentation data; Performing edge detection on the lesion segmentation data, extracting key point coordinates based on the detection results, and obtaining segmentation data; Feature point density calculation is performed on the segmented data, and feature points are screened using spatial distance constraints to obtain the three-dimensional feature point data.

3. The method for processing minimally invasive surgical data based on AR technology according to claim 1, characterized in that: The method of collecting and processing the position of the surgical instrument using the three-dimensional feature point data, spatially mapping the collected position data, and registering and fusing the mapped data to form spatial positioning data includes: Projecting the three-dimensional feature point data into the surgical space, and performing coordinate calibration on the projection result to obtain calibration coordinate data; Optically tracking the position of the surgical instrument, performing spatial fixed-point processing on the tracking data, and obtaining fixed-point data; Converting the position parameters in the fixed-point data into three-dimensional space coordinates, and performing noise filtering on the coordinate data to form the collected position data; Performing coordinate system transformation on the collected position data, and projecting the transformed data into a calibration space to obtain the mapping data; Calculating the overlap between the mapping data and the calibration coordinate data, and performing error compensation processing on the calculation results to form registration data; The registration data is spatially registered, and registration parameters are multi-dimensionally fused to obtain the spatial positioning data.

4. The method for processing minimally invasive surgical data based on AR technology according to claim 1, characterized in that: The light field data is reconstructed based on the three-dimensional feature point data and the spatial positioning data, the reconstructed data is deformed, and the processed data is combined with the real scene data to obtain augmented reality data, including: Performing spatial registration on the three-dimensional feature point data and the spatial positioning data, and uniformly processing the registration results through coordinate transformation to obtain unified coordinate data; Performing ray tracing on the unified coordinate data, converting the tracing results into a brightness value distribution, and obtaining initial light field data; Performing density calculation on the initial light field data, and performing interpolation processing on the calculation results to form interpolated light field data; Performing tissue deformation calculation on the interpolated light field data, and numerically iterating the deformation parameters to obtain the reconstructed data; Performing surface fitting on the reconstructed data, and processing the fitting results by shape variable calculation to form the processed data; Depth alignment is performed on the processed data, and the alignment result is spatially fused with the real scene data to obtain the augmented reality data.

5. The method for processing minimally invasive surgical data based on AR technology according to claim 1, characterized in that: The method of extracting surgical rules using the risk data, performing multi-dimensional analysis on the rule data, establishing a mapping relationship between the data through association processing, and forming target knowledge data includes: The risk data is divided into time series according to the surgical stage, and a multi-scale time window sliding is performed on the division results. Variance analysis and peak detection are performed based on the data volatility within the sliding window. The risk change pattern is extracted through the data distribution characteristics, and the change trend is quantified to obtain risk characteristic data; Performing multi-level decomposition processing on the risk characteristic data, performing periodic analysis on the decomposed characteristic data through an autocorrelation function, dividing the data into groups based on the periodic results, performing pattern matching based on the similarity between data groups, and performing numerical statistics and normalization processing on the matching results to form the regular data; Constructing a multidimensional feature space for the regular data, performing density distribution analysis on the data points in the feature space, clustering the data according to the density distribution results, calculating dimension correlation based on the distribution characteristics of the cluster centers, and ranking the calculation results by feature importance to obtain dimension correlation data; Perform multi-layer cross-validation on the dimension association data, perform uncertainty analysis on the verification results through information entropy calculation, evaluate the reliability of the data according to the uncertainty level, quantitatively describe the data association strength based on the evaluation results, and numerically normalize the description results to obtain association strength data; Construct a relational network using the association strength data, calculate the weights of the connection strengths between network nodes, extract key paths based on the weight distribution, characterize the data transfer characteristics through path analysis, perform mathematical modeling on the characterization results, and form mapping relationship data; The mapping relationship data is semantically structured, the structured results are organized through hierarchical analysis, knowledge units are extracted according to the organizational structure, rules are constructed through the logical relationships between knowledge units, the construction results are verified for validity, and the target knowledge data is formed.

6. The method for processing minimally invasive surgical data based on AR technology according to claim 5, characterized in that: The monitoring data is subjected to multi-dimensional feature decomposition, and correlation analysis is performed on the decomposition results to obtain a risk feature sequence, including: Decomposing the monitoring data spatially by orthogonal transformation, calculating the main direction component according to the decomposed data, and obtaining characteristic component data; Reconstructing the characteristic component data into a sequence using a time window, and extracting the fluctuation law to obtain the fluctuation characteristic data; Calculating the spectrum distribution based on the fluctuation characteristic data, classifying the frequency bands according to energy size, and forming frequency domain characteristic data; Performing correlation analysis on the frequency domain feature data using a cross-correlation function, screening relevant parameters based on the analysis results, and obtaining correlation data; Establishing a feature significance index based on the correlation data, extracting key features through statistical testing, and obtaining feature index data; The feature index data is organized based on time sequence, and the risk feature sequence is formed after the data is reordered.

7. The method for processing minimally invasive surgical data based on AR technology according to claim 6, characterized in that: The method of performing correlation analysis on the frequency domain feature data by using a cross-correlation function and screening relevant parameters according to the analysis results to obtain correlation data includes: Establish a cross-correlation matrix based on the frequency domain feature data, normalize the matrix elements, and form correlation distribution data; Extracting a sequence of characteristic values from the associated distribution data, sorting the characteristic values according to their numerical values, and obtaining characteristic sorting data; Performing threshold segmentation on the feature sorting data, calculating statistics over the segmentation intervals, and obtaining statistical feature data; Projecting the statistical feature data into a feature space, performing cluster analysis by distance measurement, and forming feature cluster data; Constructing a correlation coefficient matrix based on the characteristic clustering data, extracting key parameters based on matrix elements, and obtaining parameter correlation data; A correlation evaluation is performed based on the parameter association data, and the evaluation indicators are comprehensively calculated to form the correlation data.

8. A minimally invasive surgical data processing system based on AR technology, used to implement the minimally invasive surgical data processing method based on AR technology as described in any one of claims 1 to 7, characterized in that: The minimally invasive surgery data processing system based on AR technology includes: The reconstruction module is used to perform three-dimensional reconstruction on medical image data, segment lesions on the reconstructed data, process the segmented data through feature extraction, and obtain three-dimensional feature point data; An acquisition module is used to acquire and process the position of the surgical instrument using the three-dimensional feature point data, spatially map the acquired position data, and register and fuse the mapped data to form spatial positioning data; a processing module, configured to reconstruct light field data based on the three-dimensional feature point data and the spatial positioning data, perform deformation processing on the reconstructed data, and combine the processed data with the real scene data to obtain augmented reality data; an analysis module for performing operation trajectory analysis based on the augmented reality data, performing path planning on the analysis data, and obtaining optimal path data through optimization processing, including: performing time-series segmentation on the augmented reality data, tracking feature points on the segmented data, and obtaining spatiotemporal state data; performing multi-level trajectory decomposition on the spatiotemporal state data, extracting trajectory points from the decomposition results, and forming a trajectory point sequence; performing polynomial curve fitting on the trajectory point sequence, and performing tangent analysis on the fitting results, and obtaining the analysis data; performing collision detection calculation on the analysis data, screening the detection results according to safety distances, and forming path candidate data; performing path length calculation on the path candidate data, and solving the calculation results with an energy function to obtain path scoring data; performing multi-objective weight calculation on the path scoring data, and performing multiple iterative comparisons on the path weights to form the optimal path data; performing velocity planning on the optimal path data, and performing acceleration constraint calculation on the planning results to obtain motion control data; performing trajectory smoothing processing on the motion control data, and performing gradient optimization on the smoothing results to update the optimal path data; A monitoring module is used to monitor the optimal path data in real time, perform risk analysis on the monitoring data, and perform early warning processing on the analysis results through threshold judgment to obtain risk data, including: sampling the optimal path data according to a time window, calculating the position deviation of the sampling points, and forming real-time deviation data; performing cumulative error statistics on the real-time deviation data, and performing trend prediction on the statistical results to obtain the monitoring data; performing multi-dimensional feature decomposition on the monitoring data, performing correlation analysis on the decomposition results, and obtaining a risk feature sequence; performing risk score on the risk feature sequence, and numerically accumulating the score results to form the analysis result; numerically comparing the analysis result with a preset threshold sequence, and classifying the over-limit data into risk levels to obtain early warning data; performing spatiotemporal correlation analysis on the early warning data, and performing comprehensive probability calculation on the correlation results to form the risk data; A module is established to extract surgical rules using the risk data, perform multi-dimensional analysis on the rule data, establish a mapping relationship between the data through association processing, and form target knowledge data.

Images