Augmented reality blood vessel fusion navigation system for transcatheter occlusion

Abstract

The invention relates to the technical field of medical navigation, in particular to an augmented reality blood vessel fusion navigation system for transcatheter occlusion, which comprises a real-time image acquisition module, a blood vessel three-dimensional model construction module, an image enhancement module, a navigation module and a fusion module, and is characterized in that the image acquisition module is connected with an ultrasonic catheter to acquire a blood vessel three-dimensional image. The model construction module creates a three-dimensional blood vessel model based on the images, the image enhancement module enhances the blood vessel image through spatial adaptive transformation, key point recognition, region adaptive segmentation and edge feature extraction, and the navigation module determines the position of the tip of the catheter by using the three-dimensional blood vessel model and the enhanced image. The fusion module fuses the enhanced image and the blood vessel model to generate fusion display data, and the AR equipment displays the blood vessel three-dimensional model at the tip position of the catheter in an overlapping manner based on the tip position of the catheter and the fusion data, so that the visualization effect of a complex blood vessel structure is remarkably improved, and an accurate space reference is provided for catheter navigation.

Description

Technical Field

[0001] The present invention relates to the field of medical navigation technology, and in particular to an augmented reality vascular fusion navigation system for transcatheter occlusion, which is applied in the interventional treatment of cardiovascular diseases such as atrial septal defect and ventricular septal defect. Background Art

[0002] Transcatheter closure is an important minimally invasive surgical method for treating congenital heart disease, such as atrial septal defect and ventricular septal defect. Traditional transcatheter closure relies primarily on ultrasound and X-ray imaging for navigation, but these methods suffer from poor image quality, lack of depth information, and excessive radiation exposure.

[0003] Existing navigation systems mostly use two-dimensional image displays, which struggle to accurately represent the three-dimensional structure of blood vessels. Furthermore, the integration of image data with vascular models is poor, limiting surgical navigation accuracy. Furthermore, traditional systems lack effective enhancement processing for vascular images, making it difficult to identify complex vascular structures, hindering the precise positioning and deployment of occluders.

[0004] With the development of augmented reality (AR) technology, it has become possible to apply AR technology to transcatheter occlusion navigation. However, there is currently a lack of high-precision AR vascular fusion navigation systems specifically for transcatheter occlusion, especially the lack of effective image enhancement technology and accurate spatial registration methods, which limits the application of AR technology in this field. Summary of the Invention

[0005] The purpose of the present invention is to provide an augmented reality vascular fusion navigation system for transcatheter occlusion surgery, which realizes precise enhancement of vascular images, accurate construction of three-dimensional models and precise catheter navigation through the collaborative work of multiple modules, thereby improving the safety and success rate of transcatheter occlusion surgery.

[0006] The present invention proposes an augmented reality vascular fusion navigation system for transcatheter occlusion, comprising:

[0007] A real-time image acquisition module, connected to the ultrasound catheter, for acquiring three-dimensional images of blood vessels;

[0008] a blood vessel three-dimensional model construction module, connected to the real-time image acquisition module, and configured to construct a blood vessel three-dimensional model based on the blood vessel three-dimensional spatial image;

[0009] An image enhancement module is connected to the real-time image acquisition module and is used to enhance the three-dimensional image of the blood vessel. The image enhancement module includes:

[0010] a spatial adaptive transformation unit, configured to perform spatial adaptive transformation processing on the three-dimensional spatial image of the blood vessel to generate a grayscale image;

[0011] a key point recognition and target tracking unit, connected to the spatial adaptive transformation unit, for recognizing blood vessel key points on the grayscale image and constructing a blood vessel connectivity structure;

[0012] a region adaptive segmentation unit, connected to the key point recognition and target tracking unit, for calculating the vascular region information entropy based on the vascular connectivity structure and performing adaptive segmentation;

[0013] An edge feature extraction and splicing unit, connected to the region adaptive segmentation unit, for extracting the edge of the segmented blood vessel region and generating a region feature image;

[0014] A multi-feature fusion optimization unit, connected to the edge feature extraction and splicing unit, for performing feature selection, weighted fusion and optimization on the regional feature image;

[0015] a navigation module, connected to the vascular three-dimensional model construction module and the image enhancement module, respectively, for performing fusion comparison based on the vascular three-dimensional model and the enhanced vascular three-dimensional spatial image to determine the position of the catheter tip;

[0016] a fusion module connected to the image enhancement module and the vascular three-dimensional model construction module, and configured to fuse the enhanced vascular three-dimensional spatial image with the vascular three-dimensional model to generate fused display data;

[0017] An AR device is connected to the navigation module and the fusion module, and is used to overlay and display the three-dimensional model of the blood vessel at the position of the catheter tip based on the catheter tip position and the fusion display data.

[0018] Preferably, the spatial adaptive transformation unit includes:

[0019] An image preprocessing component, configured to perform noise reduction and equalization processing on the three-dimensional image of the blood vessel;

[0020] A local characteristic analysis component, connected to the image preprocessing component, for calculating the local regional statistical characteristics of the three-dimensional spatial image of the blood vessel;

[0021] A transformation function construction component, connected to the local characteristic analysis component, for constructing a spatially adaptive transformation function based on the local area statistical characteristics;

[0022] a transformation execution component connected to the transformation function construction component, for applying the spatially adaptive transformation function to process the three-dimensional spatial image of the blood vessel;

[0023] The grayscale conversion component is connected to the transformation execution component and is used to convert the transformed three-dimensional image of the blood vessel into a grayscale image.

[0024] Preferably, the key point recognition and target tracking unit includes:

[0025] A feature point detection component, configured to detect blood vessel feature points on the grayscale image;

[0026] A multi-scale analysis component, connected to the feature point detection component, for analyzing the blood vessel feature points in different scale spaces;

[0027] a connectivity structure building component, connected to the multi-scale analysis component, for establishing a vascular network connectivity graph based on the vascular feature points;

[0028] A key point marking component, connected to the connectivity structure building component, for classifying and marking the blood vessel feature points;

[0029] The dynamic target tracking component is connected to the key point marking component and is used to establish a tracking model to realize the dynamic tracking of the blood vessel feature points.

[0030] Preferably, the region adaptive segmentation unit includes:

[0031] An information entropy calculation component, used to calculate the information entropy of each blood vessel unit in the blood vessel connection structure;

[0032] a sub-region division component, connected to the information entropy calculation component, for dividing each blood vessel unit into a plurality of sub-regions;

[0033] A regional characteristic analysis component, connected to the sub-region division component, for analyzing the complexity of different regions based on information entropy;

[0034] An adaptive threshold generation component, connected to the regional characteristic analysis component, for automatically calculating the segmentation threshold according to regional information entropy;

[0035] The region marking component is connected to the adaptive threshold generation component and is used to mark the segmented region.

[0036] Preferably, the edge feature extraction and splicing unit includes:

[0037] An edge detection component for detecting blood vessel edges using gradient and direction information;

[0038] an edge enhancement component, connected to the edge detection component, for enhancing the detected blood vessel edge;

[0039] A regional component extraction component, connected to the edge enhancement component, for segmenting the image into multiple independent regional components based on blood vessel edge information;

[0040] A feature image generating component, connected to the region component extracting component, for generating a feature image for each region component;

[0041] The edge-guided stitching component is connected to the feature image generation component and is used to guide the precise stitching of regional components using edge information.

[0042] Preferably, the multi-feature fusion optimization unit includes:

[0043] A feature analysis component for analyzing the feature image of each regional component;

[0044] A feature screening component, connected to the feature analysis component, for screening representative features based on information value assessment;

[0045] A weight calculation component, connected to the feature screening component, for calculating a fusion weight based on feature importance and reliability;

[0046] A weighted fusion component, connected to the weight calculation component, for integrating multiple features using a weighted fusion strategy to generate an enhanced image;

[0047] The quality assessment component is connected to the weighted fusion component and is used to evaluate the quality of the fusion result and feed the evaluation result back to the aforementioned processing link.

[0048] Preferably, the blood vessel three-dimensional model building module includes:

[0049] A three-dimensional data acquisition unit, used for reading three-dimensional image data and performing three-dimensional spatial reconstruction;

[0050] a blood vessel structure recognition unit, connected to the three-dimensional data acquisition unit, for acquiring the main blood vessel trunk and blood vessel branches as well as the starting and ending points of the branches;

[0051] a blood vessel layer division unit, connected to the blood vessel structure identification unit, for dividing the blood vessel trunk and blood vessel branches into blood vessel segments and blood vessel layers;

[0052] a blood vessel connection processing unit, connected to the blood vessel layer division unit, for establishing a blood vessel segment connecting a blood vessel branch with a blood vessel trunk according to the starting point and end point of the blood vessel;

[0053] The vascular tree construction unit is connected to the vascular connection processing unit and is used to merge the connected vascular segments to form a complete vascular tree structure.

[0054] Preferably, the navigation module includes:

[0055] A registration processing unit, used for calculating the spatial transformation relationship between the three-dimensional blood vessel model and the enhanced three-dimensional blood vessel image;

[0056] a position tracking unit, connected to the registration processing unit, for tracking the real-time position of the catheter tip in the three-dimensional space of the blood vessel;

[0057] a trajectory prediction unit, connected to the position tracking unit, for predicting the future movement trajectory of the catheter tip based on the historical position data of the catheter tip;

[0058] a spatial mapping unit, connected to the trajectory prediction unit and the registration processing unit, for mapping the position of the catheter tip to a corresponding position of the 3D model of the blood vessel;

[0059] A navigation information generating unit is connected to the space mapping unit and is used to generate catheter operation suggestions and path planning information.

[0060] Preferably, the fusion module includes:

[0061] A data synchronization unit, used to ensure the synchronization of the enhanced vascular three-dimensional spatial image and the vascular three-dimensional model in time and space;

[0062] a feature matching unit, connected to the data synchronization unit, for identifying corresponding feature points in two types of data;

[0063] a transparency adjustment unit, connected to the feature matching unit, for dynamically adjusting the model transparency according to clinical needs and image quality;

[0064] A viewing angle optimization unit, connected to the transparency adjustment unit, for optimizing the model display viewing angle according to the operator's line of sight and surgical needs;

[0065] The fusion rendering unit is connected to the viewing angle optimization unit and is used to fuse and render the adjusted data into a final display image.

[0066] Preferably, the AR device includes:

[0067] A display component, used for displaying the fused three-dimensional blood vessel model in a three-dimensional form;

[0068] Camera component, used to collect real-time images of surgical scenes;

[0069] a processing component connected to the display component and the camera component, and configured to integrate the surgical scene image with the 3D blood vessel model;

[0070] An interactive component, connected to the processing component, for receiving gestures or voice commands from the doctor and adjusting display content;

[0071] a communication component connected to the processing component and configured to exchange data with the navigation module and the fusion module;

[0072] The positioning component is connected to the processing component and is used to obtain the position and posture of the AR device in space in real time.

[0073] The present invention has the following beneficial effects:

[0074] 1. Through the innovative image enhancement module, adaptive enhancement of vascular images is achieved, significantly improving the visualization of complex vascular structures;

[0075] 2. Through the construction of precise 3D vascular models, 3D reconstruction of vascular trunks and branches is achieved, providing accurate spatial reference for catheter navigation;

[0076] 3. By combining AR technology with precise navigation, intuitive real-time catheter positioning and path planning are achieved, reducing surgical risks;

[0077] 4. The close collaboration between the system modules forms a closed-loop feedback mechanism, improving navigation accuracy and system reliability;

[0078] 5. Reduces dependence on contrast agents and X-ray radiation exposure, improving surgical safety and doctor's operating comfort. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 This is a block diagram of the overall structure of the augmented reality vascular fusion navigation system for transcatheter occlusion surgery of the present invention;

[0080] Figure 2 is a functional structure diagram of the image enhancement module of the present invention;

[0081] Figure 3 It is a workflow diagram of the space adaptive transformation unit of the present invention;

[0082] Figure 4 It is a processing schematic diagram of the key point identification and target tracking unit of the present invention;

[0083] Figure 5 It is a processing principle diagram of the regional adaptive segmentation unit of the present invention;

[0084] Figure 6 Schematic diagram of the processing effect of the edge feature extraction and splicing unit of the present invention;

[0085] Figure 7 It is a processing flow chart of the multi-feature fusion optimization unit of the present invention;

[0086] Figure 8 This is a workflow diagram of the vascular three-dimensional model construction module of the present invention;

[0087] Figure 9 is a functional structure diagram of the navigation module of the present invention;

[0088] Figure 10 It is a processing diagram of the fusion module of the present invention. DETAILED DESCRIPTION

[0089] Please refer to the attached Figure 1-10 , the specific implementation of the present invention is further described in detail below with reference to the accompanying drawings.

[0090] See also Figure 1 The augmented reality vascular fusion navigation system for transcatheter occlusion provided by the present invention includes a real-time image acquisition module 1, a vascular three-dimensional model construction module 2, an image enhancement module 3, a navigation module 4, a fusion module 5 and an AR device 6.

[0091] The real-time image acquisition module 1 is connected to the ultrasound catheter and is used to capture three-dimensional images of blood vessels. Specifically, the real-time image acquisition module 1 uses a high-frequency ultrasound probe to capture real-time images of the heart's internal vascular structures. The acquisition frequency is preferably 25-50 frames per second and the resolution is preferably 512×512 pixels, ensuring that the temporal and spatial resolutions of the images meet clinical requirements.

[0092] The 3D vascular model construction module 2 is connected to the real-time image acquisition module 1 and is used to construct a 3D vascular model based on the 3D spatial images of the vessels. By processing the real-time vascular image sequence and combining it with pre-acquired CT or MRI image data, the 3D vascular model construction module 2 constructs an accurate 3D vascular model, providing a spatial reference for navigation.

[0093] Image enhancement module 3 is connected to real-time image acquisition module 1 and is used to enhance the three-dimensional spatial image of blood vessels. Image enhancement module 3 is the key innovation of this system. It improves the quality of blood vessel images through multi-level processing and enhances the visualization of blood vessel structures.

[0094] Navigation Module 4 is connected to 3D Vascular Modeling Module 2 and Image Enhancement Module 3, respectively. It determines the catheter tip position by fusing and comparing the 3D vascular model with the enhanced 3D vascular image. Using advanced registration algorithms and position tracking technology, Navigation Module 4 precisely positions the catheter within the vessel.

[0095] Fusion module 5 is connected to image enhancement module 3 and vascular 3D model construction module 2 to fuse the enhanced vascular 3D spatial image with the vascular 3D model to generate fused display data. Fusion module 5 seamlessly integrates multi-source data to generate unified visualization data.

[0096] AR device 6 is connected to navigation module 4 and fusion module 5. It is used to overlay a 3D vascular model at the catheter tip location based on the catheter tip location and fused display data. Using perspective display technology, AR device 6 overlays the virtual vascular model on the patient's actual anatomical location, providing the physician with a "perspective" view.

[0097] See also Figure 2 The image enhancement module 3 includes a spatial adaptive transformation unit 31, a key point recognition and target tracking unit 32, a region adaptive segmentation unit 33, an edge feature extraction and splicing unit 34 and a multi-feature fusion optimization unit 35, and each unit is connected in sequence to form a processing pipeline.

[0098] See also Figure 3 The spatial adaptive transformation unit 31 includes an image preprocessing component 311, a local characteristic analysis component 312, a transformation function construction component 313, a transformation execution component 314 and a grayscale conversion component 315.

[0099] The image preprocessing component 311 performs noise reduction and equalization on the three-dimensional vascular image. Specifically, a bilateral filtering algorithm is used for noise reduction, effectively suppressing noise while preserving edge information. For ultrasound images, an adaptive histogram equalization method is used for contrast enhancement. The algorithm is described as follows:

[0100] First, the input image I is divided into several blocks of equal size, preferably 8×8 or 16×16 pixels. For each block, its histogram is calculated and histogram equalization is applied to obtain the conversion function:

[0101] ,

[0102] in: is the conversion function of the kth image block, i is the pixel grayscale value (range 0-255), is the number of pixels with grayscale value j, and N is the total number of pixels in the block.

[0103] The transfer functions of each block are then combined using bilinear interpolation to avoid discontinuities at block boundaries:

[0104] ,

[0105] in: is the grayscale value of the output image at position (x, y), is the grayscale value of the input image at position (x, y), is the weight coefficient of the kth adjacent block, satisfying .

[0106] The local characteristics analysis component 312 calculates the statistical characteristics of local regions in the 3D vascular image. Specifically, a sliding window method is used to calculate the local statistical characteristics of each image region, including the local mean, variance, gradient magnitude, and direction. For vascular images, the local variance can effectively reflect the contrast difference between the blood vessels and the background, so the local variance is the focus of calculation:

[0107] ,

[0108] in: is the local variance at position (x, y), which indicates the texture complexity of the image at that position, W is the window size (usually 7×7 or 9×9 pixels are selected when processing cardiac ultrasound images), is the grayscale value of the pixel at position (i, j), is the local mean at position (x,y).

[0109] The transformation function construction component 313 constructs a spatially adaptive transformation function based on the statistical characteristics of the local area. The present invention uses an innovative adaptive transformation function to automatically adjust the transformation parameters according to the local characteristics:

[0110] ,

[0111] in: is the transformation function, which maps the original image grayscale value to the enhanced grayscale value. For the original image at position The gray value at and is a position-dependent adaptive parameter.

[0112] These two parameters are dynamically adjusted based on the local variance:

[0113] ,

[0114] ,

[0115] in: and As basic parameters, the empirical values ​​in cardiac ultrasound image processing are 1.0 and 3.0 respectively. and is the adjustment coefficient, and its empirical values ​​are 0.5 and 0.8 respectively. is the maximum local variance in the image.

[0116] The transformation execution component 314 applies a spatially adaptive transformation function to process the three-dimensional image of the blood vessel. Specifically, the above transformation function is applied to each pixel in the image to achieve spatially adaptive enhancement processing:

[0117] ,

[0118] in: is the transformed image.

[0119] Grayscale conversion component 315 converts the transformed vascular 3D image into a grayscale image. For color images, it is converted into a grayscale image by weighted averaging:

[0120] ,

[0121] in: is the converted grayscale image, 、 and The original image is at position The red, green, and blue channel values ​​at .

[0122] The innovation of the spatial adaptive transformation unit 31 is that it realizes spatial adaptive enhancement of the vascular image through local characteristic analysis and adaptive parameter adjustment, effectively improving the visibility of the vascular structure, especially the enhancement effect on low-contrast areas.

[0123] See also Figure 4 The key point recognition and target tracking unit 32 includes a feature point detection component 321, a multi-scale analysis component 322, a connected structure construction component 323, a key point marking component 324 and a dynamic target tracking component 325.

[0124] The feature point detection component 321 detects vascular feature points on the grayscale image. The present invention uses an improved FAST (Features from Accelerated Segment Test) corner detection algorithm, which has high computational efficiency and strong noise resistance, making it particularly suitable for real-time vascular image processing. The algorithm steps are as follows:

[0125] For each point p in the image, examine the 16 pixels around it with a radius of r (in vascular image processing, r is preferably 3). If there are n consecutive points (n is preferably 9) whose grayscale values ​​are all higher than the grayscale value of point p by a threshold t or lower than the grayscale value t of point p, then point p is determined to be a corner point. The threshold t is set adaptively according to the statistical characteristics of the image:

[0126] ,

[0127] in: is the average grayscale value of the image, indicating the overall brightness level, is the standard deviation of the image, indicating the overall contrast level, and k is the adjustment coefficient, with an empirical value of 0.5.

[0128] The multi-scale analysis component 322 analyzes the vascular feature points at different scales. By constructing the scale space of the image and detecting feature points at different scales, it helps to capture vascular structures of different thicknesses:

[0129] ,

[0130] in: The scale is The scale space image of The standard deviation is Gaussian kernel function, * represents convolution operation.

[0131] In transcatheter occlusion, 3-5 scales are usually used, with scale parameters $\sigma$ of 1.0, 2.0, 4.0, 8.0, and 16.0 pixels, respectively, covering different structures from small blood vessels to major vascular trunks.

[0132] The connectivity structure building component 323 builds a vascular network connectivity graph based on the vascular feature points. Using an improved minimum spanning tree algorithm, the vascular connectivity structure is constructed with the feature points as nodes:

[0133] First, calculate the Euclidean distance between feature points as the weight of the edge:

[0134] ,

[0135] in: Feature points and The edge weight between them represents the spatial distance between two points. and are the coordinates of two feature points respectively.

[0136] Then, apply Kruskal's algorithm to construct the minimum spanning tree:

[0137] 1. Sort all edges by weight from smallest to largest; 2. Add edges one by one to avoid forming loops; 3. Repeat until all nodes are connected. Finally, based on knowledge of vascular anatomy, optimize the generated connectivity graph to remove unreasonable connections.

[0138] The key point marking component 324 classifies and marks the blood vessel feature points. According to the topological position of the feature points in the connectivity graph, they are classified into bifurcation points, intersection points and endpoints:

[0139] Bifurcation point: a node with a connectivity of 3 (three blood vessels intersecting);

[0140] Intersection: a node with a connectivity greater than 3 (where multiple blood vessels intersect);

[0141] Endpoint: a node with a connectivity of 1 (the end of a blood vessel);

[0142] In order to improve the classification accuracy, local grayscale distribution and gradient information are combined for verification to eliminate pseudo feature points.

[0143] The dynamic target tracking component 325 establishes a tracking model to achieve dynamic tracking of blood vessel feature points. The present invention uses an improved KLT (Kanade-Lucas-Tomasi) optical flow tracking algorithm to achieve tracking of feature points between consecutive frames:

[0144] Assuming that the grayscale value of point (x, y) remains unchanged between time t and t+Δt, we have:

[0145] ,

[0146] The displacement vector (Δx, Δy) is solved by least squares optimization within a local window:

[0147] ,

[0148] Wherein: W represents the local window centered on the feature point, and the window size is preferably 15×15 pixels.

[0149] In order to improve tracking stability, the pyramid LK optical flow algorithm is adopted. By constructing an image pyramid, it solves the problem layer by layer from low resolution to high resolution, effectively handling large displacement situations.

[0150] The innovation of the key point recognition and target tracking unit 32 lies in that it achieves accurate characterization of the vascular network through multi-scale analysis and topological structure construction; through dynamic target tracking, it ensures continuous tracking of key vascular structures during catheter operation, providing a reliable basis for regional segmentation.

[0151] See also Figure 5 The region adaptive segmentation unit 33 includes an information entropy calculation component 331, a sub-region division component 332, a region characteristic analysis component 333, an adaptive threshold generation component 334 and a region labeling component 335.

[0152] The information entropy calculation component 331 calculates the information entropy for each blood vessel unit in the blood vessel connectivity structure. Information entropy is an important indicator for measuring the complexity of an image region. The present invention adopts a grayscale information entropy calculation method:

[0153] ,

[0154] Where: H is the information entropy, L is the number of gray levels (usually 256), and p(i) is the probability of a pixel with gray value i appearing in the region.

[0155] The sub-region division component 332 divides each vessel unit into multiple sub-regions. Considering the continuity and variability of the vessel structure, the vessel unit is divided into multiple sub-regions along the vessel direction. The preferred number of sub-regions is 5-8 to balance the computational complexity and accuracy requirements.

[0156] For each sub-region, calculate its local information entropy:

[0157] ,

[0158] in: For the The information entropy of the sub-regions, The gray value is The probability of a pixel appearing in this sub-region.

[0159] The regional characteristic analysis component 333 analyzes the complexity of different regions based on information entropy. The present invention innovatively introduces the concept of weighted information entropy, which comprehensively considers the information entropy and location importance of the sub-region:

[0160] ,

[0161] in: Indicates the The information entropy of a vascular unit, Indicates the The first vascular unit The information entropy of the sub-regions, Indicates the The number of sub-regions of a vascular unit, It represents the target weight, which is determined according to the importance of the sub-region in the vascular structure, and the preferred range is 0.5-2.0.

[0162] Calculate the local entropy value to reflect the entropy difference between regions:

[0163] ,

[0164] in: Represents the local entropy value, which indicates the degree of difference between the region and the average level, represents the average value of information entropy of all vascular units,

[0165] The adaptive threshold generation component 334 automatically calculates the segmentation threshold based on the regional information entropy. The traditional fixed threshold segmentation method is difficult to adapt to the changes in complex vascular structures. The present invention dynamically generates the segmentation threshold based on information entropy:

[0166] ,

[0167] in: For the The segmentation threshold of the vascular unit, is the basic threshold (usually set to 0.85 times the average gray value of the image in cardiac ultrasound images), is the adjustment coefficient, and its empirical value is 0.3.

[0168] The region marking component 335 marks the segmented regions. Connected component analysis is used to mark and analyze the connectivity of the segmentation results to eliminate isolated small regions and noise interference:

[0169] For each connected region , if its area is smaller than the preset threshold If the image is larger than 50 pixels (the empirical value is 50 pixels), it will be merged into the adjacent large area or deleted. This can effectively remove the small areas caused by noise and maintain the integrity of the blood vessel structure.

[0170] The innovation of the region-adaptive segmentation unit 33 lies in its ability to accurately segment complex vascular structures through an information entropy-driven adaptive segmentation mechanism, demonstrating excellent adaptability, particularly for areas with large contrast variations. This lays the foundation for subsequent edge extraction and feature fusion.

[0171] See also Figure 6 The edge feature extraction and splicing unit 34 includes an edge detection component 341 , an edge enhancement component 342 , a region component extraction component 343 , a feature image generation component 344 and an edge guided splicing component 345 .

[0172] The edge detection component 341 detects blood vessel edges using gradient and direction information. The present invention uses an improved Canny edge detection algorithm optimized for the characteristics of blood vessel structure:

[0173] First, smooth the image using a Gaussian filter:

[0174] ,

[0175] in: is the smoothed image, is the Gaussian kernel function, is the smoothing parameter, and its empirical value is 1.2.

[0176] Then, calculate the image gradient:

[0177] ,

[0178] ,

[0179] in, and The gradients in the x and y directions, respectively, represent the rate of change of the image at that point

[0180] Gradient magnitude and direction:

[0181] ,

[0182] ,

[0183] Finally, the edge is obtained by non-maximum suppression and double threshold detection:

[0184] Perform non-maximum suppression and retain the local maximum points in the gradient direction.

[0185] Using dual threshold detection and edge connection, low threshold and high threshold They are set to 15% and 30% of the maximum gradient amplitude respectively.

[0186] The edge enhancement component 342 enhances the detected blood vessel edges. In order to improve the continuity and clarity of the edges, the present invention adopts an adaptive edge enhancement method:

[0187] ,

[0188] in: is the enhanced edge image, is the original edge image, is the edge continuity measure, is the enhancement coefficient, and its empirical value is 0.6.

[0189] The edge continuity measure is calculated based on the consistency of local edge directions:

[0190] ,

[0191] Where: W is the window size, preferably 5×5 pixels.

[0192] The regional component extraction component 343 segments the image into multiple independent regional components based on the blood vessel edge information. Using the edge as the segmentation boundary, the regional component is extracted using the region growing algorithm:

[0193] Starting from the seed point of each region, the region is gradually expanded until it reaches the edge or meets the termination condition. For vascular images of transcatheter occlusion, the region is usually divided into different components such as the vascular lumen, vascular wall, and surrounding tissue.

[0194] The feature image generation component 344 generates a feature image for each region component. For each region component, its feature image is extracted to retain the unique features of the region:

[0195] ,

[0196] in: is the feature image of the kth region component, is the kth region, such as defect area, atrial septum tissue, etc.

[0197] Furthermore, the feature image is enhanced to highlight the regional characteristics:

[0198] ,

[0199] in is the enhanced feature image, is the average gray value of the region, is the enhancement coefficient, preferably in the range of 0.3-0.8.

[0200] The edge-guided stitching component 345 uses edge information to guide the precise stitching of regional components. Traditional image stitching methods may cause distortion at the edges. The present invention innovatively proposes an edge-guided stitching method:

[0201] ,

[0202] in: is the stitched image, K is the total number of regional components, For location The weight of the k-th region component.

[0203] The weight function is defined based on the distance to the edge:

[0204] ,

[0205] in: For location To The distance to the region boundary, is the smoothing parameter, and its empirical value is 3.0.

[0206] The innovation of the edge feature extraction and stitching unit 34 is that, through edge-guided feature extraction and stitching, it achieves accurate separation and seamless integration of features of different regions in the vascular image, especially retains the fine structure at the edge, and provides high-quality feature images for subsequent feature fusion.

[0207] See also Figure 7 The multi-feature fusion optimization unit 35 includes a feature analysis component 351, a feature screening component 352, a weight calculation component 353, a weighted fusion component 354 and a quality assessment component 355.

[0208] The feature analysis component 351 analyzes the feature image of each region component. For each feature image, multiple feature descriptors are extracted, including statistical features, texture features, and shape features:

[0209] Statistical characteristics: mean, variance, skewness, kurtosis, etc.;

[0210] Texture features: energy, contrast, correlation, etc. based on gray-level co-occurrence matrix;

[0211] Shape features: edge density, directional consistency, etc.;

[0212] The feature selection component 352 selects representative features based on information value assessment. It uses a feature selection method based on information gain to remove redundant features and retain the most representative features:

[0213] For features , calculate its information gain:

[0214] ,

[0215] in: Features The information gain of is the information entropy of the region, Known features Conditional entropy of the region under conditions.

[0216] The top N features with the highest information gain are selected. N is determined based on computing resources and accuracy requirements. For real-time processing of transcatheter occlusion, N is preferably 5-8.

[0217] The weight calculation component 353 calculates the fusion weight according to the feature importance and reliability. The present invention adopts an adaptive weight calculation method, which comprehensively considers the information gain, regional stability and edge clarity of the feature:

[0218] ,

[0219] in: Features The fusion weight of Features The stability measure of the region, Features A measure of the edge sharpness of the region.

[0220] The stability measure is defined based on the variance of the pixel values ​​in a region:

[0221] ,

[0222] in: is the pixel variance of the feature area, is the benchmark variance, and its empirical value is 0.5 times of the global variance of the image.

[0223] The edge sharpness metric is defined based on the edge gradient magnitude:

[0224] ,

[0225] in: is the set of boundary pixels in the feature area, is the boundary length, For location The gradient magnitude at .

[0226] The weighted fusion component 354 uses a weighted fusion strategy to integrate multiple features to generate an enhanced image. Based on the calculated weights, the feature images are weighted fused:

[0227] ,

[0228] in: is the fused image, is the i-th feature image.

[0229] In order to maintain the overall visual effect of the image, the fusion result is normalized and grayscale adjusted:

[0230] ,

[0231] in: To finally enhance the image, and Adjust the grayscale parameters and ensure that the grayscale range of the output image is [0, 255] by linear stretching.

[0232] The quality assessment component 355 assesses the quality of the fusion result and feeds the assessment result back to the aforementioned processing link. The present invention adopts a multi-index comprehensive evaluation method to assess the quality of the enhanced image:

[0233] ,

[0234] in: Rate the quality, is the contrast measure, is a measure of clarity, is the edge fidelity metric, is the structural fidelity measure, 、 、 and are the weights of each indicator, and their empirical values ​​are 0.3, 0.2, 0.3 and 0.2 respectively.

[0235] If the quality rating Below the preset threshold (The empirical value is 0.65), then adjust the parameters of the aforementioned processing link and reprocess it to form a closed-loop optimization mechanism.

[0236] The innovation of the multi-feature fusion optimization unit 35 lies in that it achieves high-quality enhancement of vascular images through feature selection, adaptive weight calculation and quality feedback mechanism, especially showing excellent performance in maintaining edge clarity and structural integrity, providing a high-quality visual foundation for catheter navigation.

[0237] See also Figure 8 The vascular three-dimensional model construction module 2 includes a three-dimensional data acquisition unit 21 , a vascular structure recognition unit 22 , a vascular layer division unit 23 , a vascular connection processing unit 24 and a vascular tree construction unit 25 .

[0238] The 3D data acquisition unit 21 is used to read 3D image data and perform 3D spatial reconstruction. Specifically, this unit receives 3D ultrasound data from the real-time image acquisition module 1, or reads pre-acquired CT / MRI data, and constructs an initial 3D spatial model using voxel reconstruction technology. During transcatheter occlusion, high-resolution CT angiography (CTA) data is preferred for reconstruction, typically with a resolution of 0.5-0.7 mm.

[0239] The vascular structure recognition unit 22 is connected to the 3D data acquisition unit 21 and is used to obtain the vascular trunk and vascular branches as well as the starting and ending points of the branches. This unit uses a method based on region growing and centerline extraction to recognize vascular structures:

[0240] First, blood vessel enhancement filtering is performed to improve the contrast between blood vessels and background;

[0241] Then the blood vessel region is extracted by region growing and threshold segmentation;

[0242] Finally, a thinning algorithm is used to extract the vascular centerline and obtain the vascular skeleton representation.

[0243] Preferably, a Frangi filter is used for blood vessel enhancement. The filter is based on the eigenvalue analysis of the Hessian matrix and has a good enhancement effect on tubular structures.

[0244] The vascular layer segmentation unit 23 is connected to the vascular structure identification unit 22 and is used to divide the vascular trunks and branches into vascular segments and vascular layers. Based on the vascular topology, this unit organizes the vascular network into a hierarchical structure: the first layer: main vascular trunks (such as the aorta and pulmonary artery); the second layer: main branches (such as the left and right pulmonary arteries); the third layer: secondary branches;

[0245] The blood vessels within each layer are further divided into segments, which are sections of blood vessels with relatively uniform diameter and curvature, usually segmented at bifurcations or where there are obvious changes in blood vessel morphology.

[0246] The vessel connection processing unit 24 is connected to the vessel layer division unit 23 and is used to establish the vessel segments where the vessel branches connect to the vessel trunk based on the starting and ending points of the vessels. This unit processes the connection relationship between the vessel branches and the trunk to ensure the integrity and continuity of the model:

[0247] Identify the intersection of vascular branches and the main trunk;

[0248] Construct connecting blood vessel segments to ensure smooth transitions at intersections;

[0249] Handle complex situations where multiple blood vessels intersect and ensure correct topology.

[0250] Preferably, a spline interpolation method is used when processing the connection relationship to ensure the curvature continuity of the connection and avoid sharp transitions.

[0251] The vascular tree construction unit 25 is connected to the vascular connection processing unit 24 and is used to merge the connected vascular segments to form a complete vascular tree structure. This unit organizes all vascular segments into a hierarchical tree structure for easy navigation and visualization:

[0252] Establish a hierarchical index to record the parent-child relationship of each vascular segment;

[0253] Merge adjacent vascular segments and eliminate redundant nodes;

[0254] Optimize the tree structure to ensure that the topology is logically reasonable and consistent with anatomical characteristics.

[0255] The final generated vascular tree model not only retains the geometric characteristics of the blood vessels, but also contains topological relationship information, providing a complete spatial reference for the navigation module.

[0256] The innovation of Module 2 for constructing the 3D vascular model is that it achieves efficient expression of complex vascular networks through hierarchical division and connection processing. It is particularly suitable for navigation needs in transcatheter occlusion surgery and can accurately represent the spatial relationships and morphological characteristics of blood vessels.

[0257] See also Figure 9 The navigation module 4 includes a registration processing unit 41 , a position tracking unit 42 , a trajectory prediction unit 43 , a space mapping unit 44 and a navigation information generation unit 45 .

[0258] The registration processing unit 41 is used to calculate the spatial transformation relationship between the 3D blood vessel model and the enhanced 3D blood vessel image. This unit achieves accurate registration between the model and the real-time image, which is the basis for accurate navigation:

[0259] First, feature points (such as vascular bifurcation points, vascular curvature extreme points, etc.) are extracted from the two data;

[0260] Then the iterative closest point (ICP) algorithm is used for coarse registration;

[0261] Finally, fine registration is performed using a non-rigid registration algorithm to handle tissue deformation.

[0262] The registration transformation relationship is expressed as:

[0263] ,

[0264] in: for point The transformed coordinates, is the rotation matrix, is the translation vector, is a non-rigid deformation field used to handle local deformation.

[0265] The position tracking unit 42 is connected to the registration processing unit 41 and is used to track the real-time position of the catheter tip in the three-dimensional space of the blood vessel. This unit locates the position of the catheter tip in real time by processing X-ray or ultrasound images:

[0266] First, the catheter tip is identified using image processing technology (e.g., high contrast, special shape markings, etc.);

[0267] Then, the trajectory is smoothed by time series filtering (such as Kalman filtering) to reduce the influence of noise;

[0268] Finally, the two-dimensional projection position is restored to the three-dimensional space position.

[0269] During transcatheter occlusion, the catheter tip position update frequency is usually 10-25 Hz, ensuring real-time requirements.

[0270] The trajectory prediction unit 43 is connected to the position tracking unit 42 and is used to predict the future movement trajectory of the catheter tip based on the historical position data of the catheter tip. This unit uses a time series prediction model to predict the future position of the catheter and provide forward-looking guidance to the doctor:

[0271] Build a dynamic model of the catheter motion, taking into account velocity, acceleration, and constraints;

[0272] Train the prediction model through historical trajectory data;

[0273] Generate predicted trajectories within a future time window (e.g., 0.5-2 seconds).

[0274] Preferably, an autoregressive model or a recursive neural network model is used for trajectory prediction, and the model training data comes from historical surgical records and real-time operation data.

[0275] The spatial mapping unit 44 is connected to the trajectory prediction unit 43 and the registration processing unit 41, and is used to map the position of the catheter tip to the corresponding position of the vascular 3D model. This unit realizes the accurate mapping between the real-time position and the model space:

[0276] The real-time position of the catheter tip is converted to the model space through the registration transformation relationship;

[0277] Calculate the shortest distance from the catheter tip to the centerline of the blood vessel;

[0278] Determine the exact location of the catheter in the vascular tree (segment and relative position).

[0279] Mapping accuracy directly affects the navigation effect. The optimal mapping error is controlled within 1mm to meet clinical needs.

[0280] The navigation information generation unit 45 is connected to the spatial mapping unit 44 and is used to generate catheter operation suggestions and path planning information. This unit generates navigation guidance information based on the current position and the target position:

[0281] Calculate the best path from the current position to the target position;

[0282] Generate direction instructions, distance information and warning prompts;

[0283] Provide operational suggestions such as steering angle, propulsion distance, etc.

[0284] Before the occluder is released, the system will also provide a release position assessment, analyze the matching degree between the occluder and the defect site, and assist doctors in making decisions.

[0285] The innovation of Navigation Module 4 lies in that it achieves high-precision, forward-looking catheter navigation through the combination of precise registration, real-time tracking and trajectory prediction, greatly improving the operational accuracy and safety of transcatheter occlusion.

[0286] See also Figure 10 The fusion module 5 includes a data synchronization unit 51, a feature matching unit 52, a transparency adjustment unit 53, a viewing angle optimization unit 54 and a fusion rendering unit 55.

[0287] The data synchronization unit 51 is used to ensure the synchronization of the enhanced vascular 3D spatial image and the vascular 3D model in time and space. This unit handles the temporal alignment and spatial coordination of data from different sources:

[0288] Time synchronization: aligning different data streams through timestamps to compensate for processing delays;

[0289] Spatial alignment: Ensure that different data use the same coordinate system and scale.

[0290] In the actual system, a timestamp-based buffering mechanism is used to ensure data synchronization, and the synchronization accuracy is preferably controlled within 50ms.

[0291] The feature matching unit 52 is connected to the data synchronization unit 51 and is used to identify the corresponding feature points in the two data. This unit establishes the corresponding relationship between different data through the feature matching algorithm:

[0292] Extract feature points from images and models (such as vascular bifurcation points, anatomical landmarks, etc.);

[0293] Use feature descriptors (such as SIFT, SURF, etc.) to represent feature points;

[0294] Correspondence is established through feature matching algorithms to eliminate false matches.

[0295] Preferably, feature matching is performed in combination with domain knowledge (such as anatomical features) to improve matching accuracy.

[0296] The transparency adjustment unit 53 is connected to the feature matching unit 52 and is used to dynamically adjust the model transparency according to clinical needs and image quality. This unit balances the visibility of the real-time image and the 3D model through intelligent transparency control:

[0297] Automatically adjust model transparency based on image quality score;

[0298] Reduce model transparency in key locations (such as near the defect site) to enhance visibility;

[0299] Provides a manual adjustment interface to meet the preferences of different doctors.

[0300] The transparency value range is 0-1, where 0 means completely transparent and 1 means completely opaque. The default value is usually set to 0.3-0.7 and is dynamically adjusted according to the specific scene.

[0301] The viewing angle optimization unit 54 is connected to the transparency adjustment unit 53 and is used to optimize the model display viewing angle according to the operator's line of sight and surgical needs. This unit provides the best viewing angle by calculating and adjusting the viewing angle:

[0302] Track the doctor's head position and gaze direction;

[0303] Calculate the current area of ​​interest (e.g., near the catheter tip);

[0304] Automatically adjusts viewing angles to ensure optimal visibility of key areas.

[0305] During transcatheter occlusion, the preferred viewing angle is set to an angle that allows simultaneous observation of the catheter tip, the current vascular segment, and the preceding vascular path, providing a sense of space and direction during the operation.

[0306] The fusion rendering unit 55 is connected to the viewing angle optimization unit 54 and is used to fuse and render the adjusted data into the final display image. This unit uses advanced graphics rendering technology to generate intuitive and information-rich fusion displays:

[0307] A method combining voxel rendering and surface rendering is used;

[0308] Applying lighting models to enhance depth perception;

[0309] Add auxiliary information (such as distance markers, warning prompts, etc.);

[0310] Optimize rendering efficiency to ensure real-time performance (the frame rate should preferably be kept above 30fps).

[0311] The final rendering results are displayed to doctors through AR devices, providing an immersive navigation experience.

[0312] The innovation of Fusion Module 5 lies in that, through the intelligent fusion and dynamic adjustment of multi-source data, it achieves information-rich, intuitive and clear visualization effects, effectively supporting doctors' operational decisions during transcatheter occlusion surgery.

[0313] The AR device 6 includes a display component, a camera component, a processing component, an interaction component, a communication component, and a positioning component.

[0314] The display component 61 is used to display the fused vascular 3D model in a 3D form. Preferably, using optical waveguide perspective display technology, the doctor can observe the patient and the superimposed virtual content at the same time:

[0315] Resolution: preferably 1920×1080 or higher;

[0316] Field of view: preferably above 40°;

[0317] Refresh rate: preferably above 60Hz to meet medical display requirements.

[0318] The camera assembly 62 is used to collect real-time images of the surgical scene. It is equipped with a stereo camera to collect three-dimensional information of the surgical scene:

[0319] Resolution: preferably 1080p or higher;

[0320] Frame rate: preferably 30fps or higher;

[0321] Equipped with an infrared sensor to enhance imaging effects in low-light environments.

[0322] The processing component 63 is connected to the display component 61 and the camera component 62 to integrate the surgical scene image with the 3D blood vessel model. A high-performance processor is built in for image processing and model rendering:

[0323] Processing capabilities: support real-time image processing and 3D rendering;

[0324] Memory: preferably 8GB or higher;

[0325] Heat dissipation system: ensures stable operation during long-term surgery.

[0326] The interactive component 64 is connected to the processing component 63 and is used to receive the doctor's gestures or voice commands and adjust the display content. It provides multiple interactive modes to meet the special needs of the surgical environment:

[0327] Gesture recognition: contact-free operation, keeping the surgical area sterile;

[0328] Voice control: supports key command recognition, with a response rate of more than 95%;

[0329] Gaze tracking: Automatically adjusts the focus area based on the doctor's gaze.

[0330] The communication component 65 is connected to the processing component 63 and is used to exchange data with the navigation module and the fusion module. It supports multiple communication methods to ensure the stability and security of data transmission:

[0331] Wireless communication: supports high-speed wireless transmission such as Wi-Fi 6 and Bluetooth 5.0;

[0332] Wired interface: provides a backup connection method to ensure stability in critical scenarios;

[0333] Data encryption: protect the security of sensitive patient information.

[0334] Positioning component 66 is connected to processing component 63 and is used to obtain the AR device's position and posture in space in real time. Multi-sensor fusion technology is used to achieve high-precision positioning: an inertial measurement unit (IMU) tracks the device's posture and movement; optical tracking uses a camera to identify environmental features to assist in positioning; positioning accuracy is preferably controlled within 1mm and 0.5°, meeting medical navigation requirements.

[0335] The innovation of AR Device 6 lies in the realization of an immersive surgical navigation experience through multi-sensor fusion and high-performance processing. In particular, it has been specially optimized in terms of device lightweight, interactive convenience and display effect to adapt to the special needs of the surgical environment.

[0336] The workflow of the augmented reality vascular fusion navigation system for transcatheter occlusion surgery of the present invention in practical application is as follows:

[0337] 1. Preoperative preparation:

[0338] Acquire the patient's CTA or MRI imaging data; use the vascular 3D model building module 2 to build the patient's vascular 3D model; mark and measure the defect location and select the appropriate occluder specifications;

[0339] 2. System Settings:

[0340] The doctor wears the AR device 6; the system automatically performs spatial calibration and registration; and the 3D model of the blood vessels is loaded into the system;

[0341] 3. Catheter insertion and navigation:

[0342] The real-time image acquisition module 1 acquires ultrasound and X-ray images; the image enhancement module 3 enhances the images; the navigation module 4 tracks the position of the catheter tip in real time; the fusion module 5 fuses the enhanced images with the three-dimensional model; and the AR device 6 displays the fused vascular model and catheter position at the corresponding position on the patient's body surface.

[0343] 4. Occluder positioning and release:

[0344] When the catheter approaches the defect, the system automatically adjusts the display scale and transparency; provides precise position feedback and operational recommendations; the doctor adjusts the occluder position based on the AR display information; the system assesses the fit between the occluder and the defect and provides release recommendations;

[0345] 5. Postoperative evaluation:

[0346] The system displays the occlusion effect in real time; evaluates the presence of residual shunts through enhanced images; and records surgical data for subsequent analysis and improvement.

[0347] In a specific application case, this system was used in an atrial septal defect occlusion surgery. The system successfully guided the doctor to precisely navigate the catheter to the defect location, and the occluder was accurately released at one time, with good occlusion effect. The entire operation time was reduced by 25% compared with traditional methods, the amount of contrast agent used was reduced by 40%, and the radiation dose received by patients and doctors was also significantly reduced.

[0348] The augmented reality vascular fusion navigation system for transcatheter occlusion surgery of the present invention significantly improves the accuracy and safety of transcatheter occlusion surgery through innovative image enhancement technology, precise vascular modeling and intuitive AR navigation, reduces the risk of complications, shortens operation time, and has broad clinical application prospects.

[0349] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. An augmented reality vascular fusion navigation system for transcatheter occlusion, characterized in that: include: A real-time image acquisition module, connected to the ultrasound catheter, for acquiring three-dimensional images of blood vessels; a blood vessel three-dimensional model construction module, connected to the real-time image acquisition module, and configured to construct a blood vessel three-dimensional model based on the blood vessel three-dimensional spatial image; An image enhancement module is connected to the real-time image acquisition module and is used to enhance the three-dimensional image of the blood vessel. The image enhancement module includes: a spatial adaptive transformation unit, configured to perform spatial adaptive transformation processing on the three-dimensional spatial image of the blood vessel to generate a grayscale image; a key point recognition and target tracking unit, connected to the spatial adaptive transformation unit, for recognizing blood vessel key points on the grayscale image and constructing a blood vessel connectivity structure; a region adaptive segmentation unit, connected to the key point recognition and target tracking unit, for calculating the vascular region information entropy based on the vascular connectivity structure and performing adaptive segmentation; An edge feature extraction and splicing unit, connected to the region adaptive segmentation unit, for extracting the edge of the segmented blood vessel region and generating a region feature image; A multi-feature fusion optimization unit, connected to the edge feature extraction and splicing unit, for performing feature selection, weighted fusion and optimization on the regional feature image; a navigation module, connected to the vascular three-dimensional model construction module and the image enhancement module, respectively, for performing fusion comparison based on the vascular three-dimensional model and the enhanced vascular three-dimensional spatial image to determine the position of the catheter tip; a fusion module connected to the image enhancement module and the vascular three-dimensional model construction module, and configured to fuse the enhanced vascular three-dimensional spatial image with the vascular three-dimensional model to generate fused display data; An AR device is connected to the navigation module and the fusion module, and is used to overlay and display the three-dimensional model of the blood vessel at the position of the catheter tip based on the catheter tip position and the fusion display data.

2. The augmented reality vascular fusion navigation system according to claim 1, characterized in that: The spatial adaptive transformation unit includes: An image preprocessing component, configured to perform noise reduction and equalization processing on the three-dimensional image of the blood vessel; A local characteristic analysis component, connected to the image preprocessing component, for calculating the local regional statistical characteristics of the three-dimensional spatial image of the blood vessel; A transformation function construction component, connected to the local characteristic analysis component, for constructing a spatially adaptive transformation function based on the local area statistical characteristics; a transformation execution component connected to the transformation function construction component, for applying the spatially adaptive transformation function to process the three-dimensional spatial image of the blood vessel; The grayscale conversion component is connected to the transformation execution component and is used to convert the transformed three-dimensional image of the blood vessel into a grayscale image.

3. The augmented reality vascular fusion navigation system according to claim 1, characterized in that: The key point recognition and target tracking unit includes: A feature point detection component, configured to detect blood vessel feature points on the grayscale image; A multi-scale analysis component, connected to the feature point detection component, for analyzing the blood vessel feature points in different scale spaces; a connectivity structure building component, connected to the multi-scale analysis component, for establishing a vascular network connectivity graph based on the vascular feature points; A key point marking component, connected to the connectivity structure building component, for classifying and marking the blood vessel feature points; The dynamic target tracking component is connected to the key point marking component and is used to establish a tracking model to realize the dynamic tracking of the blood vessel feature points.

4. The augmented reality vascular fusion navigation system according to claim 1, characterized in that: The region adaptive segmentation unit includes: An information entropy calculation component, used to calculate the information entropy of each blood vessel unit in the blood vessel connection structure; a sub-region division component, connected to the information entropy calculation component, for dividing each blood vessel unit into a plurality of sub-regions; A regional characteristic analysis component, connected to the sub-region division component, for analyzing the complexity of different regions based on information entropy; An adaptive threshold generation component, connected to the regional characteristic analysis component, for automatically calculating the segmentation threshold according to regional information entropy; The region marking component is connected to the adaptive threshold generation component and is used to mark the segmented region.

5. The augmented reality vascular fusion navigation system according to claim 1, characterized in that: The edge feature extraction and splicing unit includes: An edge detection component for detecting blood vessel edges using gradient and direction information; an edge enhancement component, connected to the edge detection component, for enhancing the detected blood vessel edge; A regional component extraction component, connected to the edge enhancement component, for segmenting the image into multiple independent regional components based on blood vessel edge information; A feature image generating component, connected to the region component extracting component, for generating a feature image for each region component; The edge-guided stitching component is connected to the feature image generation component and is used to guide the precise stitching of regional components using edge information.

6. The augmented reality vascular fusion navigation system according to claim 1, characterized in that: The multi-feature fusion optimization unit includes: A feature analysis component for analyzing the feature image of each regional component; A feature screening component, connected to the feature analysis component, for screening representative features based on information value assessment; A weight calculation component, connected to the feature screening component, for calculating a fusion weight based on feature importance and reliability; A weighted fusion component, connected to the weight calculation component, for integrating multiple features using a weighted fusion strategy to generate an enhanced image; The quality assessment component is connected to the weighted fusion component and is used to evaluate the quality of the fusion result and feed the evaluation result back to the aforementioned processing link.

7. The augmented reality vascular fusion navigation system according to claim 1, characterized in that: The blood vessel three-dimensional model building module includes: A three-dimensional data acquisition unit, used for reading three-dimensional image data and performing three-dimensional spatial reconstruction; a blood vessel structure recognition unit, connected to the three-dimensional data acquisition unit, for acquiring the main blood vessel trunk and blood vessel branches as well as the starting and ending points of the branches; a blood vessel layer division unit, connected to the blood vessel structure identification unit, for dividing the blood vessel trunk and blood vessel branches into blood vessel segments and blood vessel layers; a blood vessel connection processing unit, connected to the blood vessel layer division unit, for establishing a blood vessel segment connecting a blood vessel branch with a blood vessel trunk according to the starting point and end point of the blood vessel; The vascular tree construction unit is connected to the vascular connection processing unit and is used to merge the connected vascular segments to form a complete vascular tree structure.

8. The augmented reality vascular fusion navigation system according to claim 1, characterized in that: The navigation module includes: A registration processing unit, used for calculating the spatial transformation relationship between the three-dimensional blood vessel model and the enhanced three-dimensional blood vessel image; a position tracking unit, connected to the registration processing unit, for tracking the real-time position of the catheter tip in the three-dimensional space of the blood vessel; a trajectory prediction unit, connected to the position tracking unit, for predicting the future movement trajectory of the catheter tip based on the historical position data of the catheter tip; a spatial mapping unit, connected to the trajectory prediction unit and the registration processing unit, for mapping the position of the catheter tip to a corresponding position of the 3D model of the blood vessel; A navigation information generating unit is connected to the space mapping unit and is used to generate catheter operation suggestions and path planning information.

9. The augmented reality vascular fusion navigation system according to claim 1, characterized in that: The fusion module includes: A data synchronization unit, used to ensure the synchronization of the enhanced vascular three-dimensional spatial image and the vascular three-dimensional model in time and space; a feature matching unit, connected to the data synchronization unit, for identifying corresponding feature points in two types of data; a transparency adjustment unit, connected to the feature matching unit, for dynamically adjusting the model transparency according to clinical needs and image quality; A viewing angle optimization unit, connected to the transparency adjustment unit, for optimizing the model display viewing angle according to the operator's line of sight and surgical needs; The fusion rendering unit is connected to the viewing angle optimization unit and is used to fuse and render the adjusted data into a final display image.

10. The augmented reality vascular fusion navigation system according to claim 1, characterized in that: The AR device includes: A display component, used for displaying the fused three-dimensional blood vessel model in a three-dimensional form; Camera component, used to collect real-time images of surgical scenes; a processing component connected to the display component and the camera component, and configured to integrate the surgical scene image with the 3D blood vessel model; An interactive component, connected to the processing component, for receiving gestures or voice commands from the doctor and adjusting display content; a communication component connected to the processing component and configured to exchange data with the navigation module and the fusion module; The positioning component is connected to the processing component and is used to obtain the position and posture of the AR device in space in real time.

Images