Surgical robot navigation system based on neuroendoscope

Abstract

The invention belongs to the technical field of surgical navigation, and particularly relates to a neuroendoscope-based surgical robot navigation system, which comprises a surgical robot, a neuroendoscope, a high-resolution feature extraction module, a fluoroscopic registration module and augmented reality display equipment. According to the scheme, a fluoroscopic registration method is provided, a traditional fluoroscopic method is optimized, the structural features of the blood vessels of the patient are compared, accurate registration of a preoperative angiography image sequence and an intraoperative fluoroscopic angiography image is achieved, and the accuracy and stability of an image guidance navigation system are improved; a high-resolution feature extraction method is provided, detail information in an image is effectively captured through a visual vocabulary, features of a left view and a right view are fused, the view precision of a target operation area collected by the neuroendoscope is improved, a high-resolution image is reconstructed, and a doctor is helped to accurately navigate to the target operation area.

Description

Technical Field

[0001] The present invention relates to the technical field of surgical navigation, and specifically refers to a surgical robot navigation system based on a neuroendoscope. Background Art

[0002] A surgical robot navigation system based on a neuroendoscope is a system that uses deep learning technology to optimize surgical robot navigation.

[0003] In existing similar solutions, for example, CN114191078B, a hybrid reality-based endoscopic surgery navigation robot system, this solution addresses the technical problem that the surgical field of view is small and it is difficult for doctors to master the situation of the surgical area. It performs non-rigid registration of intraoperative ultrasound images and preoperative MR / CT images to obtain the original image-guided space. Using the epipolar geometry method, by comparing the angular relationship between the poles and the up direction of the endoscopic image and the virtual endoscopic image, it compensates for the endoscopic rotation error, obtains a virtual-real combined navigation image, realizes the integration of surgical navigation information and the endoscopic operation screen, provides a more intuitive and comprehensive surgical field of view, and through image registration and endoscopic position correction technology, ensures the accuracy of navigation information. However, the existing technology uses an optical locator and an optical tracker to calibrate the endoscope through infrared light. There are likely to be other infrared light sources such as surgical lights and monitoring equipment in the surgical environment, which will interfere with the signal of the optical tracker, resulting in position information errors and causing the image-guided navigation system to easily lose its direction when executing the program.

[0004] For example, CN111772792B, an endoscopic surgery navigation method, system, and readable storage medium based on augmented reality and deep learning, this solution addresses the technical problem that existing VR technology still cannot synchronize preoperative three-dimensional scan information with the visual field information during actual intraoperative operation. It scans the patient's target area with CT or MRI, and collects surgical site images through a high-definition camera, obtains the endoscopic position and intraoperative point cloud data in real time, performs global optimization and fine registration, constructs a three-dimensional model of the surgical target area, and outputs an augmented reality surgical navigation image through an augmented reality display device in real time, realizing a more accurate and intuitive surgical navigation effect. However, in the existing technology, due to factors such as the deformation of human tissues and the occlusion of surgical instruments, it is difficult to ensure the registration accuracy, resulting in the technical problem of surgical navigation deviation. Summary of the Invention

[0005] In view of the above situation, to overcome the defects of the prior art, the present invention provides a surgical robot navigation system based on a neuroendoscope. In the prior art, an optical locator and an optical tracker are used to calibrate the endoscope through infrared light. In a surgical environment, there are likely to be other infrared light sources such as surgical lights and monitoring equipment, which will interfere with the signals of the optical tracker, resulting in errors in position information and causing the image-guided navigation system to easily lose its direction when executing programs. This solution proposes a fluoroscopy registration method to optimize the traditional fluoroscopy method, compare the vascular structure characteristics of the patient, screen and fine-tune the pre-trained model, realize the accurate registration of the preoperative angiographic image sequence and the intraoperative fluoroscopic angiographic image, reduce the positioning error caused by the interference of infrared light sources, and improve the accuracy and stability of the image-guided navigation system; in view of the technical problem in the prior art that due to factors such as the deformation of human tissues and the occlusion of surgical instruments, it is difficult to ensure the registration accuracy and cause navigation deviation, this solution proposes a high-resolution feature extraction method to extract texture features from low-resolution original images, cluster the features into clusters using a clustering algorithm, effectively capture the detailed information in the images through a visual vocabulary, fuse the features of the left and right views, improve the view accuracy of the neuroendoscope for collecting the target surgical area, and reconstruct high-resolution images to help doctors accurately navigate to the target surgical area.

[0006] The technical solution adopted by the present invention is as follows: The present invention provides a surgical robot navigation system based on a neuroendoscope. The surgical robot navigation system based on a neuroendoscope includes a surgical robot, a neuroendoscope, a high-resolution feature extraction module, a fluoroscopy registration module, and an augmented reality display device;

[0007] The surgical robot controls the posture and position of the neuroendoscope in the patient's body;

[0008] The neuroendoscope includes two miniature cameras placed at the tip of the neuroendoscope. The surgical robot sends the neuroendoscope into the patient's body. The two miniature cameras are responsible for capturing different perspectives of the same scene to form low-resolution original images, and the low-resolution original images include a left view and a right view;

[0009] The high-resolution feature extraction module uses a high-resolution feature extraction method to improve the view accuracy of the neuroendoscope for collecting the target surgical area;

[0010] The fluoroscopy registration module uses a fluoroscopy registration method to optimize the traditional fluoroscopy method, obtains the preoperative angiographic image sequence and the intraoperative fluoroscopic angiographic image, realizes the registration of the preoperative angiographic image sequence and the intraoperative fluoroscopic angiographic image, and helps the doctor accurately navigate to the target surgical area through the augmented reality display device.

[0011] The high-resolution feature extraction module adopts a high-resolution feature extraction method, and the high-resolution feature extraction method specifically includes the following steps:

[0012] Step A1: Extract features from the low-resolution original image. The specific operations are as follows:

[0013] Step A11: Preprocessing, perform preprocessing operations on the low-resolution original image;

[0014] Step A12: Extract texture features, extract texture features from the low-resolution original image, and the texture features are represented in the form of feature vectors;

[0015] Step A13: Extract feature frequencies, which are used to convert the texture features in the low-resolution original image into text-based descriptions. Specifically, use the K-means clustering algorithm to cluster the texture features into clusters, and use the feature vector at the centroid of each cluster as the visual word representing the cluster. Construct a visual word table with visual words as columns and the occurrence frequencies of visual words as rows;

[0016] Step A2: Reconstruct a high-resolution feature image from the low-resolution original image. The specific operations are as follows:

[0017] Step A21: Extract original features, which are used to extract original features using a neural network. Specifically, through instance normalization operations, standardize the feature information of each channel in the low-resolution original image, use dilated convolutions to expand the receptive field of the neural network convolutional layer. First, perform convolutions on each channel of the low-resolution original image in turn, then perform convolutions on all channels simultaneously, and finally generate a gating signal through the sigmoid function. The gating signal is used to suppress unimportant feature information in the low-resolution original image to obtain the original features;

[0018] Step A22: Occlusion processing, which is used to handle occlusion relationships. Specifically, calculate an effective mask through a scaled hyperbolic tangent function. The effective mask is used to distinguish between occluded regions and non-occluded regions, and ignore the original features of the occluded regions based on the effective mask. The formula for calculating the effective mask using the scaled hyperbolic tangent function is as follows: ;

[0019] In the formula, represents the effective mask on the left view or right view, represents the hyperbolic tangent function, represents the scaling factor, represents the original features related to the view, represents the left view and right view, ;

[0020] Step A23: Cross-view feature interaction, which is used to fuse the left view and the right view. Specifically, first calculate the disparity attention vector between the left view and the right view. The disparity attention vector is used to mark the positions in the views where there may be occlusion relationships. Transform the left view and the right view according to the disparity attention vector. The calculation formula of the disparity attention vector is as follows:

[0021] Step A231: Among them, the calculation formula of the disparity attention vector from the right view to the left view is as follows: ;

[0022] In the formula, is the disparity attention vector from the right view to the left view, represents the original feature after transforming the right view to the left view perspective, is the original feature of the right view;

[0023] Step A232: Among them, the calculation formula of the disparity attention vector for the transformation from the left view to the right view is as follows: ;

[0024] In the formula, is the disparity attention vector from the left view to the right view, represents the feature after transforming the left view to the right view perspective, is the original feature of the left view;

[0025] Step A24: Use the attention mechanism to fuse the original features of the left and right views and output the fused feature;

[0026] Step A25: Reconstruction. Specifically, perform a linking operation on the visual vocabulary and the fused feature to output a feature map. Use strip pooling operation to divide the feature map into multiple strips and perform pooling on each strip. Use a sub-pixel convolutional layer to rearrange the pixels of the feature map to increase the feature resolution and output a high-resolution feature image;

[0027] Step A26: Calculate the high-resolution loss. Train the neural network by minimizing the high-resolution loss. The calculation formula of the used high-resolution loss is as follows: ;

[0028] In the formula, represents the high-resolution loss, represents the weight coefficient, represents the L1 norm, and represent the high-resolution feature image generated by the neural network, and High-resolution feature images representing the true left and right views Represents the sum of the photometric consistency loss, cycle consistency loss, and smoothness loss.

[0029] The fluoroscopy registration module uses a fluoroscopy registration method to optimize the traditional fluoroscopy method. The fluoroscopy registration method specifically includes the following steps:

[0030] Step B1: Sequence capture. Using the traditional fluoroscopy method, obtain a preoperative angiographic image sequence of the complete cardiac cycle of the patient's target surgical area. The preoperative angiographic image sequence is used to display the anatomical structure after the patient ingests the contrast agent.

[0031] Step B2: Marker line creation. Mark the center line extending along the blood vessel on the preoperative angiographic image sequence as the marker line, and smooth the marker line by spline interpolation.

[0032] Step B3: Marker line segmentation. Specifically, divide each marker line into multiple marker sub-lines, and the marker sub-lines are used to gradually simulate the advancement process of the blood vessel in fluoroscopy after ingesting the contrast agent.

[0033] Step B4: Pre-training. Specifically, collect an angiographic image knowledge base, use a deep learning model, train the deep learning model based on the angiographic image knowledge base to obtain a pre-trained model, and group the pre-trained models according to the C-arm angle. The C-arm angle refers to the angle between the X-ray tube of the angiographic device and the patient's body.

[0034] Step B5: Pre-trained model selection. Specifically, compare the preoperative angiographic image sequence with the pre-trained models within the same C-arm angle group trained by the angiographic image knowledge base, calculate the similarity in turn, and select the pre-trained model with the highest similarity.

[0035] Step B6: Training pairing. To improve the adaptability of the pre-trained model to the patient's blood vessel structure, use the preoperative angiographic image sequence to fine-tune the selected pre-trained model. Specifically, pair the marker sub-lines with the corresponding preoperative angiographic image sequence to obtain training pairs, and use the training pairs to adjust the pre-trained model.

[0036] Step B7: Acquire intraoperative fluoroscopic angiographic images. Use the trained pre-trained model to segment the marker lines of the intraoperative fluoroscopic angiographic images, and align and register the marker lines of the intraoperative fluoroscopic angiographic images and the marker lines of the preoperative angiographic image sequence.

[0037] The beneficial effects achieved by the present invention using the above solution are as follows:

[0038] (1) In view of the technical problem that in the prior art, an endoscope is calibrated using an optical locator and an optical tracker through infrared light, and there are likely to be other infrared light sources such as surgical lights and monitoring equipment in the surgical environment, which can interfere with the signals of the optical tracker, resulting in position information errors and causing the image-guided navigation system to easily lose its direction during the execution of the program. This solution proposes a fluoroscopy registration method to optimize the traditional fluoroscopy method, compare the vascular structure characteristics of the patient, screen and fine-tune the pre-trained model, achieve accurate registration of preoperative angiographic image sequences and intraoperative fluoroscopic angiographic images, reduce the positioning errors caused by infrared light source interference, and improve the accuracy and stability of the image-guided navigation system;

[0039] (2) In view of the technical problem that in the prior art, due to factors such as the deformation of human tissues and the occlusion of surgical instruments, it is difficult to ensure the registration accuracy, resulting in navigation deviation. This solution proposes a high-resolution feature extraction method to extract texture features from low-resolution original images, use a clustering algorithm to cluster the features into clusters, effectively capture the detailed information in the images through a visual vocabulary, fuse the features of the left and right views, improve the view accuracy of the neuroendoscope for collecting the target surgical area, and reconstruct high-resolution images to help doctors accurately navigate to the target surgical area. Brief Description of the Drawings

[0040] Figure 1 It is a module connection diagram of a surgical robot navigation system based on a neuroendoscope provided by the present invention;

[0041] Figure 2 It is a flow schematic diagram of the high-resolution feature extraction method;

[0042] Figure 3 It is a flow schematic diagram of the fluoroscopy registration method.

[0043] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, and do not constitute a limitation to the present invention. Detailed Embodiments

[0044] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0045] Embodiment 1: Refer to Figures 1 to 3, this embodiment provides a surgical robot navigation system based on a neuroendoscope. The surgical robot navigation system based on a neuroendoscope includes a surgical robot, a neuroendoscope, a high-resolution feature extraction module, a fluoroscopy registration module, and an augmented reality display device;

[0046] The surgical robot controls the posture and position of the neuroendoscope inside the patient's body;

[0047] The neuroendoscope includes two miniature cameras placed at the tip of the neuroendoscope, one on the left and one on the right. The surgical robot inserts the neuroendoscope into the patient's body, and the two miniature cameras are responsible for capturing different perspectives of the same scene, forming low-resolution original images, where the low-resolution original images include a left view and a right view;

[0048] The high-resolution feature extraction module uses a high-resolution feature extraction method to improve the view accuracy of the neuroendoscope for collecting the target surgical area;

[0049] The fluoroscopy registration module uses a fluoroscopy registration method to optimize the traditional fluoroscopy method, obtains preoperative contrast image sequences and intraoperative fluoroscopic contrast images, realizes the registration of the preoperative contrast image sequences and the intraoperative fluoroscopic contrast images, and helps the doctor accurately navigate to the target surgical area through the augmented reality display device.

[0050] Embodiment 2: Refer to Figures 1 to 2 , this embodiment is based on the above embodiment. The high-resolution feature extraction module uses a high-resolution feature extraction method, and the high-resolution feature extraction method specifically includes the following steps:

[0051] Step A1: Extract features from the low-resolution original image. The specific operations are as follows:

[0052] Step A11: Preprocess the low-resolution original image by performing preprocessing operations on it;

[0053] Step A12: Extract texture features by extracting texture features from the low-resolution original image, where the texture features are represented in the form of feature vectors;

[0054] Step A13: Extract feature frequencies, which are used to convert the texture features in the low-resolution original image into a text-form description. Specifically, use the K-means clustering algorithm to cluster the texture features into clusters, and use the feature vector at the centroid of each cluster as the visual word representing the cluster. Construct a visual word table with the visual words as columns and the occurrence frequencies of the visual words as rows;

[0055] Step A2: Reconstruct a high-resolution feature image from the low-resolution original image. The specific operations are as follows:

[0056] Step A21: Extract the original features. The original features are extracted using a neural network. Specifically, through instance normalization operation, the feature information of each channel in the low-resolution original image is standardized. Dilated convolution is used to expand the receptive field of the convolutional layer of the neural network. First, convolution is performed on each channel of the low-resolution original image in sequence. Secondly, convolution is performed on all channels simultaneously. Finally, a gating signal is generated through the sigmoid function. The gating signal is used to suppress unimportant feature information in the low-resolution original image to obtain the original features;

[0057] Step A22: Occlusion processing, which is used to handle occlusion relationships. Specifically, an effective mask is calculated through a scaled hyperbolic tangent function. The effective mask is used to distinguish the occluded area and the non-occluded area. Based on the effective mask, the original features in the occluded area are ignored. The formula for calculating the effective mask by the scaled hyperbolic tangent function is as follows: ;

[0058] In the formula, represents the effective mask on the left view or the right view, represents the hyperbolic tangent function, represents the scaling factor, represents the original features related to the view, represents the left view and the right view, ;

[0059] Step A23: Cross-view feature interaction, which is used to fuse the left view and the right view. Specifically, first, the disparity attention vector between the left view and the right view is calculated. The disparity attention vector is used to mark the positions in the view where there may be occlusion relationships. The left view and the right view are transformed according to the disparity attention vector. The formula for calculating the disparity attention vector is as follows:

[0060] Step A231: Among them, the formula for calculating the disparity attention vector from the right view to the left view is as follows: ;

[0061] In the formula, is the disparity attention vector from the right view to the left view, represents the original features after transforming the right view to the left view perspective, is the original feature of the right view;

[0062] Step A232: Among them, the formula for calculating the disparity attention vector for the transformation from the left view to the right view is as follows: ;

[0063] In the formula, is the disparity attention vector from the left view to the right view, Features after converting the left view to the right view perspective, are the original features of the left view;

[0064] Step A24: Use the attention mechanism to fuse the original features of the left and right views and output the fused features;

[0065] Step A25: Reconstruction, specifically, perform a linking operation on the visual vocabulary and the fused features to output a feature map, use strip pooling operation to divide the feature map into 15 strips, and perform pooling on each strip, and use a sub-pixel convolutional layer to rearrange the pixels of the feature map to increase the feature resolution and output a high-resolution feature image;

[0066] Step A26: Calculate the high-resolution loss, and train the neural network by minimizing the high-resolution loss. The formula for the high-resolution loss used is as follows: ;

[0067] In the formula, represents the high-resolution loss, represents the weight coefficient, represents the L1 norm, and represent the high-resolution feature image generated by the neural network, and represent the high-resolution feature images of the real left view and right view, represents the sum of the photometric consistency loss, cycle consistency loss, and smoothness loss.

[0068] Example 3: Refer to Figures 1 to 3 , this example is based on the above example. The fluoroscopy registration module uses the fluoroscopy registration method to optimize the traditional fluoroscopy method. The fluoroscopy registration method specifically includes the following steps:

[0069] Step B1: Sequence capture, use the traditional fluoroscopy method to obtain the preoperative angiography image sequence of the complete cardiac cycle of the patient's target surgical area. The preoperative angiography image sequence is used to show the anatomical structure of the patient after ingesting the contrast agent;

[0070] Step B2: Marker line creation, mark the center line extending along the blood vessel on the preoperative angiography image sequence as the marker line, and smooth the marker line by spline interpolation method;

[0071] Step B3: Marker line segmentation, specifically, divide each marker line into 60 marker sub-lines, and the marker sub-lines are used to gradually simulate the advancing process of the blood vessel in fluoroscopy after ingesting the contrast agent;

[0072] Step B4: Pre-training. Specifically, collect a contrast image knowledge base, and use a deep learning model to train the deep learning model based on the contrast image knowledge base to obtain a pre-trained model. Group the pre-trained models according to the C-arm angle, where the C-arm angle refers to the angle between the X-ray tube of the contrast device and the patient's body.

[0073] Step B5: Selection of pre-trained model. Specifically, compare the pre-operative contrast image sequence with the pre-trained models within the same C-arm angle group trained from the contrast image knowledge base, calculate the similarity in sequence, and select the pre-trained model with the highest similarity.

[0074] Step B6: Training pairing, which is used to improve the adaptability of the pre-trained model to the patient's vascular structure. Use the pre-operative contrast image sequence to fine-tune the selected pre-trained model. Specifically, pair the marked sub-lines with the corresponding pre-operative contrast image sequence to obtain training pairs, and use the training pairs to adjust the pre-trained model.

[0075] Step B7: Collect intraoperative fluoroscopic contrast images, use the trained pre-trained model to segment the marked lines in the intraoperative fluoroscopic contrast images, and align and register the marked lines in the intraoperative fluoroscopic contrast images and the marked lines in the pre-operative contrast image sequence.

[0076] Example 4: Refer to Figures 1 to 3 , this example is based on the above example. In step A11, the preprocessing operation on the low-resolution original image is specifically

[0077] Step A111: Contrast adjustment, adjust the contrast of the image.

[0078] Step A112: Noise reduction, reduce the noise in the image and improve the image quality.

[0079] Step A113: Illumination correction, adjust the illumination intensity of the image to make the image uniform.

[0080] Example 5: Refer to Figures 1 to 3 , this example is based on the above example. In step A12, the texture feature is specifically to calculate the gray-level co-occurrence matrix (GLCM) of the image and extract Haralick features from it.

[0081] Example 6: Refer to Figures 1 to 3 , this example is based on the above example. This example is only different from the previous example in terms of texture features. The texture feature is specifically to perform a convolution operation on the image using a Gabor filter bank and extract Gabor features in different directions and frequencies.

[0082] Example 7: Refer to Figures 1 to 3, based on the above embodiment, in step B5, the similarity is specifically the normalized cross - correlation index.

[0083] Embodiment Eight: Refer to Figures 1 to 3 , based on the above embodiment, this embodiment is only different from the previous embodiment in terms of similarity, and the similarity is specifically the structural similarity index.

[0084] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non - exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed, or elements inherent to such process, method, article or device.

[0085] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

[0086] The above describes the present invention and its implementation manners. Such description is not restrictive. What is shown in the drawings is only one of the implementation manners of the present invention, and the actual structure is not limited thereto. Generally speaking, if those of ordinary skill in the art are inspired by it and design similar structural manners and embodiments without creative efforts without departing from the purpose of the present invention, they shall fall within the protection scope of the present invention.

Claims

1. A surgical robot navigation system based on neuroendoscopy, characterized in that: It includes surgical robots, neuroendoscopes, high-resolution feature extraction modules, fluoroscopic registration modules, and augmented reality display devices; The surgical robot controls the posture and position of the neuroendoscope in the patient's body; The neuroendoscope includes two left and right micro cameras placed at the tip of the neuroendoscope. The surgical robot delivers the neuroendoscope into the patient's body. The two micro cameras are responsible for capturing different perspectives of the same scene to form a low-resolution original image, which includes a left view and a right view. The high-resolution feature extraction module adopts a high-resolution feature extraction method to improve the accuracy of the view of the target surgical area acquired by the neuroendoscopy; The fluoroscopic registration module uses a fluoroscopic registration method to optimize the traditional fluoroscopic method, obtains a preoperative angiography image sequence and an intraoperative fluorescent angiography image, realizes the registration of the preoperative angiography image sequence and the intraoperative fluorescent angiography image, and helps the doctor to accurately navigate to the target surgical area through an augmented reality display device.

2. A surgical robot navigation system based on neuroendoscopy according to claim 1, characterized in that: The high-resolution feature extraction method specifically comprises the following steps: Step A1: Extract features from the low-resolution original image. The specific operations are as follows: Step A11: preprocessing, performing preprocessing operations on the low-resolution original image; Step A12: extracting texture features, extracting texture features from the low-resolution original image, wherein the texture features are represented by feature vectors; Step A13: extracting feature frequencies to convert texture features in the low-resolution original image into a description in text form, specifically, clustering the texture features into clusters using a K-means clustering algorithm, taking the feature vector at the centroid of each cluster as the visual vocabulary representing the cluster, and constructing a visual vocabulary table with visual vocabulary as columns and the frequency of occurrence of visual vocabulary as rows; Step A2: Reconstruct a high-resolution feature image from the low-resolution original image. The specific operations are as follows: Step A21: extracting original features, which is used to extract original features using a neural network. Specifically, the feature information of each channel in the low-resolution original image is standardized through an instance normalization operation, and the receptive field of the convolution layer of the neural network is expanded using an expanded convolution. First, each channel of the low-resolution original image is convolved in turn, and then all channels are convolved at the same time. Finally, a gating signal is generated through a sigmoid function. The gating signal is used to suppress unimportant feature information in the low-resolution original image to obtain the original features; Step A22: occlusion processing, used to process the occlusion relationship, specifically, calculating the effective mask by scaling the hyperbolic tangent function, and ignoring the original features of the occluded area based on the effective mask; Step A23: Cross-view feature interaction, used to fuse the left view and the right view. Specifically, firstly, a disparity attention vector between the left view and the right view is calculated; Step A24: Use the attention mechanism to fuse the original features of the left and right views and output the fused features; Step A25: Reconstruction, specifically, linking the visual vocabulary and the fused features, outputting a feature map, dividing the feature map into multiple strips using a strip pooling operation, and pooling each strip, rearranging pixels of the feature map using a sub-pixel convolution layer, increasing feature resolution, and outputting a high-resolution feature image; Step A26: Calculate the high-resolution loss and train the neural network by minimizing the high-resolution loss.

3. A surgical robot navigation system based on neuroendoscopy according to claim 2, characterized in that: The fluoroscopic registration method specifically comprises the following steps: Step B1: Sequence capture, using traditional fluoroscopic methods to obtain a preoperative angiographic image sequence of the patient's target surgical area throughout the entire cardiac cycle; Step B2: creating a marker line, marking the center line extending along the blood vessel on the preoperative angiography image sequence as the marker line, and smoothing the marker line by spline interpolation; Step B3: segmenting the marking line, specifically, segmenting each marking line into a plurality of marking sub-lines; Step B4: pre-training, specifically, collecting a knowledge base of contrast images, using a deep learning model, training the deep learning model on the basis of the contrast image knowledge base to obtain a pre-trained model, and grouping the pre-trained models according to the C-arm angle, wherein the C-arm angle refers to the angle between the X-ray tube of the contrast equipment and the patient's body; Step B5: pre-trained model selection, specifically, comparing the pre-operative angiography image sequence with the pre-trained models in the same C-arm angle group obtained by training the angiography image knowledge base, calculating the similarities in turn, and selecting the pre-trained model with the highest similarity; Step B6: training pairing, used to improve the adaptability of the pre-trained model to the patient's vascular structure, using the pre-operative angiography image sequence to fine-tune the selected pre-trained model, specifically, pairing the marked sub-line with the corresponding pre-operative angiography image sequence to obtain a training pair, and using the training pair to adjust the pre-trained model; Step B7: Acquire intraoperative fluorescence angiography images, use the trained pre-trained model to segment the marking lines of the intraoperative fluorescence angiography images, and align and register the marking lines of the intraoperative fluorescence angiography images with the marking lines of the preoperative angiography image sequence.

Images