A surgical robot navigation system based on neuroendoscope

Abstract

The present invention belongs to the field of surgical navigation technology, and specifically refers to a surgical robot navigation system based on a neuroendoscope, wherein the system includes a surgical robot, a neuroendoscope, a high-resolution feature extraction module, a fluoroscopic registration module, and an augmented reality display device. This solution proposes a fluoroscopic registration method to optimize the traditional fluoroscopic method, compare the patient's vascular structure characteristics, and achieve accurate registration of preoperative angiographic image sequences and intraoperative fluoroscopic angiographic images, thereby improving the accuracy and stability of the image-guided navigation system; it also proposes a high-resolution feature extraction method to effectively capture detailed information in the image through a visual vocabulary, fuse the features of the left and right views, improve the accuracy of the view of the target surgical area captured by the neuroendoscope, reconstruct high-resolution images, and help doctors accurately navigate to the target surgical area.

Description

Technical Field

[0001] The present invention relates to the technical field of surgical navigation, and in particular to a surgical robot navigation system based on a neuroendoscope. Background Art

[0002] The neuroendoscope-based surgical robot navigation system is a system that uses deep learning technology to optimize surgical robot navigation.

[0003] Among the existing approximate solutions, for example, CN114191078B is an endoscopic surgical navigation robot system based on mixed reality. This solution addresses the technical problem that the surgical field of view is small and it is difficult for doctors to grasp the situation in the surgical area. The intraoperative ultrasound image is non-rigidly aligned with the preoperative MR / CT image to obtain the original image guidance space. The epipolar geometry method is used to compensate for the endoscope rotation error by comparing the angular relationship between the pole and the upward direction of the endoscopic image and the virtual endoscopic image, and a navigation image combining virtual and real is obtained. The surgical navigation information is integrated with the endoscopic operation screen, providing a more intuitive and comprehensive surgical field of view. The technical effect of ensuring the accuracy of the navigation information is achieved through image registration and endoscope position correction technology. However, the existing technology uses optical positioners and optical trackers to calibrate the endoscope using infrared light. There is a high possibility that other infrared light sources such as surgical lights and monitoring equipment may exist in the surgical environment, which will interfere with the signal of the optical tracker, resulting in position information errors and causing the image-guided navigation system to easily get lost when executing the procedure.

[0004] For example, CN111772792B is an endoscopic surgical navigation method, system and readable storage medium based on augmented reality and deep learning. This solution addresses the technical problem that existing VR technology still cannot allow medical personnel to synchronize preoperative three-dimensional scanning information with the field of view information during actual intraoperative operations. CT or MRI scans are performed on the patient's target part, and surgical site images are collected through high-definition cameras. The endoscope position and intraoperative point cloud data are obtained in real time, and global optimization and precise alignment are performed to construct a three-dimensional model of the surgical target part. The augmented reality surgical navigation image is output in real time through an augmented reality display device, achieving the technical effect of more accurate and intuitive surgical navigation. However, in the existing technology, due to factors such as deformation of human tissue and occlusion of surgical instruments, the alignment accuracy is difficult to guarantee, resulting in the technical problem of surgical navigation deviation. Summary of the Invention

[0005] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a surgical robot navigation system based on neuroendoscope. In view of the prior art using optical positioners and optical trackers to calibrate the endoscope through infrared light, there are likely to be other infrared light sources such as surgical lights and monitoring equipment in the surgical environment, which will interfere with the signal of the optical tracker, resulting in position information errors, causing the image-guided navigation system to easily get lost when executing the procedure. This solution proposes a fluorescent perspective alignment method, optimizes the traditional fluorescent perspective method, compares the patient's vascular structure characteristics, screens and fine-tunes the pre-trained model, and realizes the preoperative angiography image sequence and intraoperative fluorescent angiography image. The precise registration of images reduces positioning errors caused by interference from infrared light sources, thereby improving the accuracy and stability of the image-guided navigation system. In response to the technical problem in existing technologies that navigation deviations are caused by factors such as deformation of human tissue and occlusion of surgical instruments, it is difficult to ensure registration accuracy. This solution proposes a high-resolution feature extraction method to extract texture features from low-resolution original images, cluster the features into clusters using a clustering algorithm, effectively capture detailed information in the image through a visual vocabulary, fuse the features of the left and right views, improve the view accuracy of the target surgical area acquired by the neuroendoscopy, reconstruct high-resolution images, and help doctors accurately navigate to the target surgical area.

[0006] The technical solution adopted by the present invention is as follows: The present invention provides a neuroendoscopic-based surgical robot navigation system, which includes a surgical robot, a neuroendoscopic, a high-resolution feature extraction module, a fluoroscopic registration module, and an augmented reality display device;

[0007] The surgical robot controls the posture and position of the neuroendoscope within the patient's body;

[0008] The neuroendoscope includes two left and right micro cameras placed at the tip of the neuroendoscope. The surgical robot inserts the neuroendoscope into the patient's body. The two micro cameras are responsible for capturing different perspectives of the same scene to form low-resolution original images. The low-resolution original images include a left view and a right view.

[0009] The high-resolution feature extraction module uses a high-resolution feature extraction method to improve the accuracy of the view of the target surgical area acquired by the neuroendoscopy;

[0010] The fluoroscopic registration module uses a fluoroscopic registration method to optimize the traditional fluoroscopic method, obtains a preoperative angiography image sequence and an intraoperative fluoroscopic angiography image, realizes the registration of the preoperative angiography image sequence and the intraoperative fluoroscopic angiography image, and helps the doctor accurately navigate to the target surgical area through an augmented reality display device.

[0011] The high-resolution feature extraction module adopts a high-resolution feature extraction method, which specifically includes the following steps:

[0012] Step A1: Extract features from the low-resolution original image. The specific operations are as follows:

[0013] Step A11: preprocessing, performing preprocessing operations on the low-resolution original image;

[0014] Step A12: extracting texture features, extracting texture features from the low-resolution original image, wherein the texture features are represented by feature vectors;

[0015] Step A13: Extracting feature frequencies to convert texture features in the low-resolution original image into a textual description. Specifically, the texture features are clustered using a K-means clustering algorithm. The feature vector at the centroid of each cluster is used as the visual vocabulary representing the cluster. A visual vocabulary table is constructed with visual vocabulary as columns and the frequency of occurrence of visual vocabulary as rows.

[0016] Step A2: Reconstruct a high-resolution feature image from the low-resolution original image. The specific operations are as follows:

[0017] Step A21: Extracting original features, which is used to extract original features using a neural network. Specifically, the feature information of each channel in the low-resolution original image is standardized through instance normalization, and the receptive field of the convolution layer of the neural network is expanded using dilated convolution. First, each channel of the low-resolution original image is convolved in sequence, and then all channels are convolved simultaneously. Finally, a gating signal is generated through a sigmoid function. The gating signal is used to suppress unimportant feature information in the low-resolution original image to obtain the original features.

[0018] Step A22: Occlusion processing, which is used to process occlusion relationships. Specifically, an effective mask is calculated by scaling the hyperbolic tangent function. The effective mask is used to distinguish between occluded areas and non-occluded areas. The original features of the occluded areas are ignored based on the effective mask. The formula for calculating the effective mask using the scaled hyperbolic tangent function is as follows:

[0019] ;

[0020] Where, Indicates the valid mask on the left or right view, represents the hyperbolic tangent function, represents the scaling factor, represents the primitive features associated with the view, Represents the left and right views, ;

[0021] Step A23: Cross-view feature interaction is used to fuse the left view and the right view. Specifically, the disparity attention vector between the left view and the right view is first calculated. The disparity attention vector is used to mark the positions where there may be occlusion relationships in the view. The left view and the right view are transformed according to the disparity attention vector. The calculation formula of the disparity attention vector is as follows:

[0022] Step A231: The disparity attention vector from the right view to the left view is calculated as follows:

[0023] ;

[0024] Where, is the disparity attention vector from the right view to the left view, Represents the original features after converting the right view to the left view. is the original feature of the right view;

[0025] Step A232: The calculation formula for the disparity attention vector converted from the left view to the right view is as follows:

[0026] ;

[0027] Where, is the disparity attention vector from the left view to the right view, Represents the features after converting the left view to the right view. is the original feature of the left view;

[0028] Step A24: Use the attention mechanism to fuse the original features of the left and right views and output the fused features;

[0029] Step A25: Reconstruction, specifically, linking the visual vocabulary and the fused features to output a feature map, dividing the feature map into multiple strips using a strip pooling operation, pooling each strip, and rearranging pixels of the feature map using a sub-pixel convolution layer to increase feature resolution and output a high-resolution feature image;

[0030] Step A26: Calculate the high-resolution loss and train the neural network by minimizing the high-resolution loss. The high-resolution loss calculation formula used is as follows:

[0031] ;

[0032] Where, represents the high-resolution loss, represents the weight coefficient, represents the L1 norm, and represents the high-resolution feature image generated by the neural network, and High-resolution feature images representing the true left and right views, represents the sum of photometric consistency loss, cycle consistency loss and smoothness loss.

[0033] The fluoroscopic registration module adopts a fluoroscopic registration method to optimize the traditional fluoroscopic method. The fluoroscopic registration method specifically includes the following steps:

[0034] Step B1: Sequence capture, using conventional fluoroscopic methods to obtain a preoperative angiographic image sequence of the patient's target surgical area over a complete cardiac cycle, wherein the preoperative angiographic image sequence is used to display the patient's anatomical structure after the contrast agent is ingested;

[0035] Step B2: marker line creation: mark the center line extending along the blood vessel on the preoperative angiographic image sequence as the marker line, and smooth the marker line using spline interpolation;

[0036] Step B3: segmenting the marker line, specifically, segmenting each marker line into multiple marker sub-lines, where the marker sub-lines are used to gradually simulate the advancement process of the blood vessels during fluoroscopy after the contrast agent is taken up;

[0037] Step B4: Pre-training, specifically, collecting a knowledge base of contrast imaging images, using a deep learning model, training the deep learning model based on the contrast imaging image knowledge base to obtain pre-trained models, and grouping the pre-trained models according to C-arm angle, where the C-arm angle refers to the angle between the X-ray tube of the imaging device and the patient's body;

[0038] Step B5: Pre-trained model selection, specifically, comparing the preoperative angiographic image sequence with the pre-trained models in the same C-arm angle group obtained by training the angiographic image knowledge base, calculating the similarity in turn, and selecting the pre-trained model with the highest similarity;

[0039] Step B6: Training pairing, used to improve the adaptability of the pre-trained model to the patient's vascular structure, using the pre-operative angiographic image sequence to fine-tune the selected pre-trained model. Specifically, the marked sub-lines are paired with the corresponding pre-operative angiographic image sequence to obtain training pairs, and the training pairs are used to adjust the pre-trained model.

[0040] Step B7: Acquire intraoperative fluorescence angiography images, use the trained pre-trained model to segment the marker lines of the intraoperative fluorescence angiography images, and align and register the marker lines of the intraoperative fluorescence angiography images with the marker lines of the preoperative angiography image sequence.

[0041] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0042] (1) In view of the technical problem that the existing technology uses optical locators and optical trackers to calibrate endoscopes through infrared light, there are likely to be other infrared light sources such as surgical lights and monitoring equipment in the surgical environment, which will interfere with the signal of the optical tracker, resulting in position information errors, causing the image-guided navigation system to easily lose its way when executing the procedure. This solution proposes a fluoroscopic registration method, optimizes the traditional fluoroscopic method, compares the patient's vascular structure characteristics, screens and fine-tunes the pre-trained model, and achieves accurate registration of preoperative angiography image sequences and intraoperative fluoroscopic angiography images, reducing the positioning error caused by interference from infrared light sources and improving the accuracy and stability of the image-guided navigation system;

[0043] (2) In view of the technical problem that the existing technology is difficult to ensure the registration accuracy due to factors such as deformation of human tissue and occlusion of surgical instruments, which leads to navigation deviation, this solution proposes a high-resolution feature extraction method to extract texture features from low-resolution original images, cluster the features into clusters using a clustering algorithm, effectively capture the detailed information in the image through a visual vocabulary, fuse the features of the left and right views, improve the view accuracy of the target surgical area captured by the neuroendoscopy, reconstruct a high-resolution image, and help doctors accurately navigate to the target surgical area. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A module connection diagram of a neuroendoscopic-based surgical robot navigation system provided by the present invention;

[0045] Figure 2 Schematic diagram of the process of high-resolution feature extraction method;

[0046] Figure 3 Schematic diagram of the flow chart of the fluoroscopic registration method.

[0047] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0049] Example 1: See Figures 1 to 3, this embodiment provides a neuroendoscopic-based surgical robot navigation system, the neuroendoscopic-based surgical robot navigation system comprising a surgical robot, a neuroendoscopic, a high-resolution feature extraction module, a fluoroscopic registration module, and an augmented reality display device;

[0050] The surgical robot controls the posture and position of the neuroendoscope within the patient's body;

[0051] The neuroendoscope includes two left and right micro cameras placed at the tip of the neuroendoscope. The surgical robot inserts the neuroendoscope into the patient's body. The two micro cameras are responsible for capturing different perspectives of the same scene to form low-resolution original images. The low-resolution original images include a left view and a right view.

[0052] The high-resolution feature extraction module uses a high-resolution feature extraction method to improve the accuracy of the view of the target surgical area acquired by the neuroendoscopy;

[0053] The fluoroscopic registration module uses a fluoroscopic registration method to optimize the traditional fluoroscopic method, obtains a preoperative angiography image sequence and an intraoperative fluoroscopic angiography image, realizes the registration of the preoperative angiography image sequence and the intraoperative fluoroscopic angiography image, and helps the doctor accurately navigate to the target surgical area through an augmented reality display device.

[0054] Example 2: See Figures 1 to 2 This embodiment is based on the above embodiment. The high-resolution feature extraction module adopts a high-resolution feature extraction method. The high-resolution feature extraction method specifically includes the following steps:

[0055] Step A1: Extract features from the low-resolution original image. The specific operations are as follows:

[0056] Step A11: preprocessing, performing preprocessing operations on the low-resolution original image;

[0057] Step A12: extracting texture features, extracting texture features from the low-resolution original image, wherein the texture features are represented by feature vectors;

[0058] Step A13: Extracting feature frequencies to convert texture features in the low-resolution original image into a textual description. Specifically, the texture features are clustered using a K-means clustering algorithm. The feature vector at the centroid of each cluster is used as the visual vocabulary representing the cluster. A visual vocabulary table is constructed with visual vocabulary as columns and the frequency of occurrence of visual vocabulary as rows.

[0059] Step A2: Reconstruct a high-resolution feature image from the low-resolution original image. The specific operations are as follows:

[0060] Step A21: Extracting original features, which is used to extract original features using a neural network. Specifically, the feature information of each channel in the low-resolution original image is standardized through instance normalization, and the receptive field of the convolution layer of the neural network is expanded using dilated convolution. First, each channel of the low-resolution original image is convolved in sequence, and then all channels are convolved simultaneously. Finally, a gating signal is generated through a sigmoid function. The gating signal is used to suppress unimportant feature information in the low-resolution original image to obtain the original features.

[0061] Step A22: Occlusion processing, which is used to process occlusion relationships. Specifically, an effective mask is calculated by scaling the hyperbolic tangent function. The effective mask is used to distinguish between occluded areas and non-occluded areas. The original features of the occluded areas are ignored based on the effective mask. The formula for calculating the effective mask using the scaled hyperbolic tangent function is as follows:

[0062] ;

[0063] Where, Indicates the valid mask on the left or right view, represents the hyperbolic tangent function, represents the scaling factor, represents the primitive features associated with the view, Represents the left and right views, ;

[0064] Step A23: Cross-view feature interaction is used to fuse the left view and the right view. Specifically, the disparity attention vector between the left view and the right view is first calculated. The disparity attention vector is used to mark the positions where there may be occlusion relationships in the view. The left view and the right view are transformed according to the disparity attention vector. The calculation formula of the disparity attention vector is as follows:

[0065] Step A231: The disparity attention vector from the right view to the left view is calculated as follows:

[0066] ;

[0067] Where, is the disparity attention vector from the right view to the left view, Represents the original features after converting the right view to the left view. is the original feature of the right view;

[0068] Step A232: The calculation formula for the disparity attention vector converted from the left view to the right view is as follows:

[0069] ;

[0070] Where, is the disparity attention vector from the left view to the right view, Represents the features after converting the left view to the right view. is the original feature of the left view;

[0071] Step A24: Use the attention mechanism to fuse the original features of the left and right views and output the fused features;

[0072] Step A25: Reconstruction, specifically, linking the visual vocabulary and the fused features to output a feature map, dividing the feature map into 15 strips using a strip pooling operation, pooling each strip, and rearranging the pixels of the feature map using a sub-pixel convolution layer to increase feature resolution and output a high-resolution feature image;

[0073] Step A26: Calculate the high-resolution loss and train the neural network by minimizing the high-resolution loss. The high-resolution loss calculation formula used is as follows:

[0074] ;

[0075] Where, represents the high-resolution loss, represents the weight coefficient, represents the L1 norm, and represents the high-resolution feature image generated by the neural network, and High-resolution feature images representing the true left and right views, represents the sum of photometric consistency loss, cycle consistency loss and smoothness loss.

[0076] Example 3: See Figures 1 to 3 This embodiment is based on the above embodiment. The fluoroscopic registration module adopts a fluoroscopic registration method to optimize the traditional fluoroscopic method. The fluoroscopic registration method specifically includes the following steps:

[0077] Step B1: Sequence capture, using conventional fluoroscopic methods to obtain a preoperative angiographic image sequence of the patient's target surgical area over a complete cardiac cycle, wherein the preoperative angiographic image sequence is used to display the patient's anatomical structure after the contrast agent is ingested;

[0078] Step B2: marker line creation: mark the center line extending along the blood vessel on the preoperative angiographic image sequence as the marker line, and smooth the marker line using spline interpolation;

[0079] Step B3: segmenting the marker line, specifically, segmenting each marker line into 60 marker sub-lines, which are used to gradually simulate the advancement process of the blood vessels during fluoroscopy after the contrast agent is taken up;

[0080] Step B4: Pre-training, specifically, collecting a knowledge base of contrast imaging images, using a deep learning model, training the deep learning model based on the contrast imaging image knowledge base to obtain pre-trained models, and grouping the pre-trained models according to C-arm angle, where the C-arm angle refers to the angle between the X-ray tube of the imaging device and the patient's body;

[0081] Step B5: Pre-trained model selection, specifically, comparing the preoperative angiographic image sequence with the pre-trained models in the same C-arm angle group obtained by training the angiographic image knowledge base, calculating the similarity in turn, and selecting the pre-trained model with the highest similarity;

[0082] Step B6: Training pairing, used to improve the adaptability of the pre-trained model to the patient's vascular structure, using the pre-operative angiographic image sequence to fine-tune the selected pre-trained model. Specifically, the marked sub-lines are paired with the corresponding pre-operative angiographic image sequence to obtain training pairs, and the training pairs are used to adjust the pre-trained model.

[0083] Step B7: Acquire intraoperative fluorescence angiography images, use the trained pre-trained model to segment the marker lines of the intraoperative fluorescence angiography images, and align and register the marker lines of the intraoperative fluorescence angiography images with the marker lines of the preoperative angiography image sequence.

[0084] Example 4: See Figures 1 to 3 This embodiment is based on the above embodiment. In step A11, the low-resolution original image is pre-processed, specifically,

[0085] Step A111: contrast adjustment, adjusting the contrast of the image;

[0086] Step A112: noise reduction, reducing noise in the image and improving image quality;

[0087] Step A113: Light correction, adjusting the light intensity of the image to make the image uniform.

[0088] Example 5: See Figures 1 to 3 This embodiment is based on the above embodiment. In step A12, the texture feature is specifically calculated by calculating the gray level co-occurrence matrix (GLCM) of the image and extracting the Haralick feature therefrom.

[0089] Example 6: See Figures 1 to 3 This embodiment is based on the above embodiment. This embodiment differs from the above embodiment only in texture features. Specifically, the texture features are: using a Gabor filter group to perform a convolution operation on the image to extract Gabor features of different directions and frequencies.

[0090] Example 7: See Figures 1 to 3 This embodiment is based on the above embodiment. In step B5, the similarity is specifically a normalized mutual correlation index.

[0091] Example 8: See Figures 1 to 3 This embodiment is based on the above embodiment. This embodiment differs from the above embodiment only in the similarity, which is specifically a structural similarity index.

[0092] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0093] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

[0094] The present invention and its embodiments are described above. This description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs structures and embodiments similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. A surgical robot navigation system based on neuroendoscopy, characterized in that: Includes surgical robots, neuroendoscopes, high-resolution feature extraction modules, fluoroscopy registration modules, and augmented reality display devices; The surgical robot controls the posture and position of the neuroendoscope within the patient's body; The neuroendoscope includes two left and right micro cameras placed at the tip of the neuroendoscope. The surgical robot inserts the neuroendoscope into the patient's body. The two micro cameras are responsible for capturing different perspectives of the same scene to form low-resolution original images. The low-resolution original images include a left view and a right view. The high-resolution feature extraction module uses a high-resolution feature extraction method to improve the accuracy of the view of the target surgical area acquired by the neuroendoscopy; The fluoroscopic registration module uses a fluoroscopic registration method to optimize the traditional fluoroscopic method, obtains a preoperative angiographic image sequence and an intraoperative fluoroscopic angiographic image, realizes the registration of the preoperative angiographic image sequence and the intraoperative fluoroscopic angiographic image, and helps the doctor accurately navigate to the target surgical area through an augmented reality display device; The high-resolution feature extraction method specifically comprises the following steps: Step A1: Extract features from the low-resolution original image. The specific operations are as follows: Step A11: preprocessing, performing preprocessing operations on the low-resolution original image; Step A12: extracting texture features, extracting texture features from the low-resolution original image, wherein the texture features are represented by feature vectors; Step A13: Extracting feature frequencies to convert texture features in the low-resolution original image into a textual description. Specifically, the texture features are clustered using a K-means clustering algorithm. The feature vector at the centroid of each cluster is used as the visual vocabulary representing the cluster. A visual vocabulary table is constructed with visual vocabulary as columns and the frequency of occurrence of visual vocabulary as rows. Step A2: Reconstruct a high-resolution feature image from the low-resolution original image. The specific operations are as follows: Step A21: Extracting original features, which is used to extract original features using a neural network. Specifically, the feature information of each channel in the low-resolution original image is standardized through instance normalization, and the receptive field of the convolution layer of the neural network is expanded using dilated convolution. First, each channel of the low-resolution original image is convolved in sequence, and then all channels are convolved simultaneously. Finally, a gating signal is generated through a sigmoid function. The gating signal is used to suppress unimportant feature information in the low-resolution original image to obtain the original features. Step A22: Occlusion processing, used to process occlusion relationships, specifically, calculating an effective mask by scaling a hyperbolic tangent function, and ignoring original features of the occluded area based on the effective mask; Step A23: Cross-view feature interaction, used to fuse the left view and the right view. Specifically, first calculate the disparity attention vector between the left view and the right view; Step A24: Use the attention mechanism to fuse the original features of the left and right views and output the fused features; Step A25: Reconstruction, specifically, linking the visual vocabulary and the fused features to output a feature map, dividing the feature map into multiple strips using a strip pooling operation, pooling each strip, and rearranging pixels of the feature map using a sub-pixel convolution layer to increase feature resolution and output a high-resolution feature image; Step A26: Calculate the high-resolution loss and train the neural network by minimizing the high-resolution loss.

2. A neuroendoscopic surgical robot navigation system according to claim 1, characterized in that: The fluoroscopic registration method specifically comprises the following steps: Step B1: Sequence capture, using traditional fluoroscopic methods to obtain a preoperative angiographic image sequence of the patient's target surgical area throughout the entire cardiac cycle; Step B2: marker line creation: mark the center line extending along the blood vessel on the preoperative angiographic image sequence as the marker line, and smooth the marker line using spline interpolation; Step B3: Marking line segmentation, specifically, segmenting each marking line into multiple marking sub-lines; Step B4: Pre-training, specifically, collecting a knowledge base of contrast imaging images, using a deep learning model, training the deep learning model based on the contrast imaging image knowledge base to obtain pre-trained models, and grouping the pre-trained models according to C-arm angle, where the C-arm angle refers to the angle between the X-ray tube of the imaging device and the patient's body; Step B5: Pre-trained model selection, specifically, comparing the preoperative angiographic image sequence with the pre-trained models in the same C-arm angle group obtained by training the angiographic image knowledge base, calculating the similarity in turn, and selecting the pre-trained model with the highest similarity; Step B6: Training pairing, used to improve the adaptability of the pre-trained model to the patient's vascular structure, using the pre-operative angiographic image sequence to fine-tune the selected pre-trained model. Specifically, the marked sub-lines are paired with the corresponding pre-operative angiographic image sequence to obtain training pairs, and the training pairs are used to adjust the pre-trained model. Step B7: Acquire intraoperative fluorescence angiography images, use the trained pre-trained model to segment the marker lines of the intraoperative fluorescence angiography images, and align and register the marker lines of the intraoperative fluorescence angiography images with the marker lines of the preoperative angiography image sequence.

Images