Surgical navigation method and system of direct sparse odometer, electronic equipment and medium

Abstract

The invention discloses a surgical navigation method and system for a direct sparse odometer, electronic equipment and a medium, and solves the problem that an automatic registration technology capable of effectively collecting data is lacked for a narrow closed part operation similar to an esophageal endoscope. The method comprises the following steps of: generating a central curve containing key morphological characteristic positions of the esophagus by utilizing medical images; acquiring a complete endoscopy-entering digestive endoscopy image at an intraoperative registration stage; reconstructing a movement track and a posture of a digestive endoscopy camera by utilizing a DSO algorithm; calculating the key morphological characteristic positions of the esophagus in the operation; performing automatic registration by using two groups of curve point sets; the direct method not only avoids the calculation time of the features, but also avoids the condition of feature missing, as long as the light and shade change exists in the scene, the image features can be gradually changed and do not form local image features, the collection of the form of the internal organ such as the esophagus is realized, and a basis is provided for the navigation registration of the esophageal surgery.

Description

Technical Field

[0001] The present invention relates to the technical field of surgical navigation, and in particular to a surgical navigation method, system, electronic equipment and medium based on direct sparse odometry. Background Art

[0002] The purpose of surgical navigation is to accurately match the patient's preoperative and intraoperative imaging data with the intraoperative anatomical structure, track the surgical instruments during surgery, and update their position in real time in the form of probes on the three-dimensional reconstructed data, so that the doctor can clearly see the position of the surgical instruments relative to the patient's anatomical structure, making the surgical operation faster, more accurate and safer. Registration is a key technology of the surgical navigation system. The accuracy and efficiency of registration largely determine the quality of the navigation system and the success or failure of the operation.

[0003] Most of the related technologies focus on navigation and registration for surgeries on areas with obvious and easily identifiable external features, such as the nose and eyes. However, for surgeries performed in narrow and closed areas, such as esophageal endoscopy, it is not convenient to extract the external features of organ tissues through endoscopic images, or to set feature markers before or during surgery. It is also difficult to use large dedicated equipment to collect data with rich geometric features. Therefore, there is still a lack of an effective automatic registration technology. Summary of the Invention

[0004] The purpose of the present invention is to solve the above problems and provide a surgical navigation method, system, electronic device and medium based on direct sparse odometry to solve the problems in related technologies.

[0005] In order to solve the above problems, the present invention provides the following technical solutions: On the one hand, a direct sparse odometry surgical navigation method comprises Determine the image center curve of the esophagus based on the patient's esophageal imaging data and the curvature of the center point of the esophageal cross section; Determine the position of the endoscope trajectory based on the digestive endoscope image and signal decomposition under sparse regularization constraints; Determine the center curve of the endoscope trajectory according to the posture of the endoscope trajectory and the image center curve; The image center curve and trajectory center curve are automatically aligned to generate a spatial transformation matrix for surgical navigation.

[0006] In one embodiment, determining the pose of the endoscope trajectory based on the digestive endoscope image and signal decomposition under sparse regularization constraints includes: The frames of digestive endoscopy images are acquired and stored in a preset data structure. The first frame is designated as the key frame to complete initialization. The positions of feature points are determined based on the second frame and triangulation to create a motion model. Starting from the third frame, the predicted pose of each frame image is predicted based on the pose relationship between each frame image and its key frame; Each frame image is matched with the features of the key frame, and the photometric error is calculated to reflect the deviation between the current frame and the predicted pose to obtain the actual pose.

[0007] In one embodiment, obtaining the actual pose further includes optimizing the actual pose: Perform initial optimization by optimizing the photometric error through a sliding window; The newly arrived frame is used as a keyframe, and its pose estimate will be passed to the backend. The backend applies a global Gauss-Newton method to optimize the assumed pose, reduce the photometric error, and iterate continuously until the preset convergence conditions are met.

[0008] In one embodiment, determining the image center curve of the esophagus based on the patient target image data and the curvature of the cross-sectional center point includes: Obtain the patient's esophageal imaging data and construct a continuous spatial point set based on the center point of the esophageal cross section; Extract the starting point, end point and image feature points of the esophagus from the spatial point set and connect them to form an image center curve containing the key morphological feature positions of the esophagus; Image feature point extraction includes determining curvature values ​​based on a set of spatial points, and arranging the curvature values ​​in descending order to obtain a curvature list; A threshold is set, and multiple spatial point subset intervals are obtained based on the curvature list and adjacent differences. The points that best match the preset esophageal empirical model are selected from the spatial point subset intervals as image feature points.

[0009] In one embodiment, determining the trajectory center curve of the endoscope according to the posture of the endoscope trajectory and the image center curve includes: According to the pose calculation vector of the endoscope trajectory, the three-dimensional trajectory data is rotated to the coordinate system of the image center line, and the feature points of the trajectory data and the feature point set of the center curve are averaged and translated to complete the coarse registration transformation and obtain the trajectory center curve.

[0010] In one embodiment, automatically registering the image center curve and the trajectory center curve to generate a spatial transformation matrix for surgical navigation includes: The initial refined registration includes: obtaining the center curve of the trajectory data, and initially registering it with the image center curve based on the iterative closest point algorithm to obtain the spatial transformation relationship; Secondary refined registration involves re-segmenting the endoscopic trajectory data and performing high-precision scanning with a smaller step size to fit a more accurate image center curve. This is then initially registered with the image center curve using an iterative closest point algorithm to obtain a precise transformation relationship. When the registration is completed, the obtained spatial transformation matrix accurately maps the endoscope position acquired in real-time spatial pose to the corresponding medical image and pre-established three-dimensional model, thereby realizing real-time navigation of esophageal endoscopic surgery.

[0011] In a second aspect, a direct sparse odometry surgical navigation system is used to implement a direct sparse odometry surgical navigation method; the system comprises: Image center curve extraction module: used to process the patient's esophageal image data to extract the image center curve; 3D trajectory data pose vector acquisition module: used to process digestive endoscopy images and signal decomposition under sparse regularization constraints, determine the pose of the endoscope trajectory, pre-process the 3D trajectory data, and obtain the pose vector; Coarse registration module: used to associate the image center curve with the pose vector of the 3D trajectory data, and obtain the trajectory center curve through the rotation matrix; Fine-tuning registration module: obtains the final spatial transformation matrix through fine-tuning of the trajectory center curve and the image center curve; Mapping module: Through the final spatial transformation matrix, the endoscope position acquired in real-time spatial posture is accurately mapped to the corresponding medical image and pre-established three-dimensional model, thereby realizing real-time navigation of esophageal endoscopic surgery.

[0012] In a third aspect, an electronic device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor performs a surgical navigation method based on direct sparse odometry.

[0013] In a fourth aspect, a computer-readable storage medium stores a computer program, which, when executed on a computer, causes the computer to execute a surgical navigation method based on direct sparse odometry.

[0014] Compared with the prior art, the present invention has the following beneficial effects: (1) Based on the digestive endoscopy image and the signal decomposition under the sparse regularization constraint, the present invention determines the pose of the endoscope trajectory by simultaneously optimizing the photometric error and reprojection error, and simultaneously optimizing and updating the pose, photometric parameters, and inverse depth. The direct sparse odometry estimates the camera motion directly based on the pixel brightness information, and can completely avoid the calculation of key points and descriptors. Therefore, the direct method avoids both the feature calculation time and the problem of feature loss. As long as there are changes in light and dark in the scene, they can be gradual changes without forming local image features.

[0015] (2) The present invention uses medical images to obtain a set of spatial points through midline points, calculates feature points to obtain an image center curve containing the key morphological feature positions of the esophagus; it realizes the acquisition of the morphology of the internal organ such as the esophagus, and provides a basis for esophageal surgical navigation and registration; and allows certain point pairs to be lost during the acquisition process, and allows the setting of any device acquisition coordinate system, and can manually correct the specified feature point pairs and fine-tune the registration results.

[0016] (3) The present invention is free from the constraints of hardware such as NDI magnetic navigation equipment. It can complete the registration of the three-dimensional model and the digestive endoscopy image only on the basis of the existing digestive endoscopy image, providing the surgeon with more information on the original basis. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which: Figure 1 is a flow chart of the present invention; Figure 2 This is the DSO framework diagram; Figure 3 This is the virtual intestinal reconstruction image of the present invention; Figure 4 This is the DSO graph in the real intestinal data; Figure 5 This is the orb-slam3 graph in real intestinal data. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present invention clearer, the following Figures 1 to 5 The present invention is further described in detail. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0019] 1. Specific methods of this embodiment

[0020] This paper proposes an automatic registration and navigation method for esophageal endoscopic surgical trajectories and medical images, combining spatial pose reconstruction techniques (e.g., Direct Sparse Odometry (DSO)). The main principle is to use medical images to generate a central curve containing the locations of key morphological features of the esophagus. During the intraoperative registration phase, a complete digestive endoscopic image is collected. The DSO algorithm is used to reconstruct the motion trajectory and pose of the digestive endoscopic camera. The locations of key morphological features of the esophagus during the intraoperative period are calculated, and the intraoperative esophageal central curve is fitted. Automatic registration is then performed using two sets of curve point sets.

[0021] This method can map the esophageal endoscope pose to medical images. For endoscope trajectory generation, this method supports: (1) allowing certain point pairs to be lost during acquisition, (2) setting an arbitrary device acquisition coordinate system, and (3) manually correcting specified feature point pairs and fine-tuning the registration results.

[0022] 2.1 Medical Image Processing

[0023] like Figure 1 As shown in the figure, using the patient's CT scan data, the pixel coordinates of the esophagus region in the cross-sectional image sequence are averaged to determine the esophageal center of each cross-sectional area, and a continuous spatial point set is constructed by combining the Z-axis coordinate information. On the 2D CT cross section, for the mask image ,in Indicates that the pixel belongs to the esophagus area. If it does not belong to the esophagus, the center point of the esophagus The coordinate mean of all pixel points in the esophageal area can be calculated using formula (1-1) and formula (1-2): (1-1) (1-2) Based on the spatial point set, the esophageal central curve and its key morphological feature positions are extracted, including the esophageal starting point, end point, and a feature point with significant curvature change located near the third esophageal stenosis.

[0024] In order to improve the data quality, a sliding window average filter is performed on the point set to reduce the noise effect. , consider a window For all adjacent points within , use formula (1-3) to calculate their coordinate mean: (1-3) The selection of curvature feature points follows the following steps: (1) For a point set of the form Three consecutive points , the curvature can be estimated using formula (1-4) : , , (1-4) (2) The curvature values ​​of all points Sort and get an ordered curvature list ,in Indicates the sorted Large curvature.

[0025] (3) Set a threshold , calculate the adjacent differences of the curvature corresponding index after sorting .if , then it is believed that There is a spatial discontinuity between adjacent points of the same curvature value, which is used as the grouping boundary. According to the difference value grouping, multiple point set subintervals are formed. The internal curvature of each subinterval is relatively stable and spatially close.

[0026] (4) For each subinterval, calculate the degree of conformity of its representative point with the preset empirical model (e.g., the location of the third esophageal stenosis). Select the representative point of the subinterval that is closest to the empirical model as the feature point.

[0027] 2.2 Endoscopic trajectory DSO generation and registration preparation

[0028] The DSO digestive endoscopy trajectory generation process is as follows Figure 2 As shown: It is mainly divided into the following steps.

[0029] Data processing flow 2.1 Data Preprocessing

[0030] like Figures 1 to 5 As shown in Figure 2, before formally processing endoscopic image data, the DSO algorithm needs to perform a series of preliminary data preparation steps: 1) Image pyramid construction: For each input frame, the system first constructs an image pyramid, which is a multi-scale representation that allows image features to be analyzed at different resolutions.

[0031] 2) Image feature extraction: For each layer of the image, calculate its radiation intensity (brightness or grayscale value), as well as information such as the gradient in the x-axis and y-axis directions.

[0032] 3) Additional information calculation: In addition to the basic gradient value, the sum of the squares of the gradient values ​​is also calculated, which helps to enhance the descriptive ability of the features.

[0033] 4) Information integration: All the above calculation results are integrated and stored in a data structure called FrameHessian, preparing for subsequent feature matching and motion estimation.

[0034] 2.2.2 Special processing of the first two frames

[0035] The DSO algorithm requires at least two frames of images to start, of which the first frame is designated as the key frame. For these initial two frames, the algorithm performs the following steps: 1) Initialization Check and Execution: The system first checks whether it has been initialized. If not, it performs initialization processing on the first frame, including building a pyramid, selecting feature points, and recording necessary information.

[0036] 2) Second Frame Tracking and Optimization: Once initialization is complete, the newly arrived second frame image is used to calculate the residual and Jacobian matrix with the keyframe. This step uses triangulation to initially determine the locations of feature points in 3D space. The initial camera motion model is then estimated based on these points and the photometric error.

[0037] 3) Acceleration optimization: a) Inverse depth initialization strategy: Given that the inverse depth between adjacent frames does not change much, the inverse depth of all points in the new frame is initially set to the inverse depth of the points in the previous layer to accelerate the convergence process.

[0038] B) Inverse depth median update method: In the optimization loop, the inverse depth value of each feature point is updated to the median of the inverse depths of its neighboring points, achieving a smooth transition of the inverse depth value and enhancing robustness.

[0039] 2.2.3 Continuous Processing of Subsequent Frames

[0040] For each frame received thereafter, the DSO performs the following operations: 1) Pose Hypothesis and Verification: Based on the pose information of historical frames (stored in allFrameHistory), a hypothesis is made about the possible pose of the current frame. Pose prediction is performed using the known keyframes and their relative pose relationships.

[0041] 2) Keyframe matching and photometric error calculation: By matching the keyframe features, the photometric error is calculated to reflect the deviation between the current frame and the predicted pose. Based on these residuals and the corresponding Jacobian matrix, a fast local optimization method (such as incremental or local BA) is used to preliminarily optimize the camera pose. This process is usually performed on the front end to quickly obtain a relatively reasonable pose estimate. The photometric error model of the pixel in the algorithm is:

[0042] (2-1) Among them, u is the pixel number, is the weight coefficient, and are the i-th and j-th frame images, and They are used to correct the image and Bias parameter for photometry, and They are used to correct the image and The scaling parameter of the luminosity, and They are images and Exposure time photometric bias parameters, is an image Pixel u is projected onto The sequence number of the point in .

[0043] The final sliding window optimization photometric error calculation formula is: (2-2) Where F is the number of image frames, Ui is all pixels of the i-th frame image, and obs(u) is all other images that can see a pixel u in the i-th frame image.

[0044] 3) Gauss-Newton Optimization: The newly arrived frame is treated as a keyframe, and its pose estimate is passed to the backend. The backend applies a global Gauss-Newton method to optimize the assumed pose, reducing photometric error, and iterates until the preset convergence conditions are met. This simultaneously optimizes the pose of the current frame, the poses of all keyframes in the window, the 3D coordinates of map points, and certain photometric parameters (such as exposure time and brightness scaling factor).

[0045] During the iterative optimization process, all the residual terms can be pieced together into a large linear equation: (2-3) Where J is the Jacobian matrix, W is the weight coefficient matrix, r is the residual vector, the superscript T is the transpose of the matrix, and δx is the update amount of the variable in each iteration. The left side can be written as: , that is, the Hessian matrix.

[0046] By simultaneously optimizing the photometric error and reprojection error, pose, photometric parameters, and inverse depth are simultaneously optimized and updated. Unlike traditional bundle adjustment, which optimizes keyframes and feature points globally, sliding window optimization in DSO preserves a certain number of keyframes, requiring marginalization of excess frames. Marginalization in sliding window optimization involves updating the information matrix, preserving the information of deleted frames as prior information in the matrix. This reduces the computational complexity of the optimization while achieving good optimization results.

[0047] 4) Key frame generation criteria: A new key frame is generated under the following conditions: a) The visual field changes significantly, exceeding the preset threshold.

[0048] b) Camera translation causes scene occlusion changes.

[0049] c) The exposure conditions change dramatically.

[0050] 5) Sliding Window Management: Maintain a dynamic frame window (typically 5 to 7 frames), periodically remove old keyframes and related map points through marginalization techniques, and update the associated Hessian matrix to ensure efficient system memory utilization and real-time performance.

[0051] 6) Non-keyframe processing: For non-keyframes, the main focus is on pose estimation. These frames are used to update the depth information of immature map points and continuously improve the 3D reconstruction of the scene. If the average optical flow distance of the pixels in the image of the current frame relative to the previous frame is less than a threshold, the current frame is a non-keyframe; otherwise, the current frame is a keyframe.

[0052] The mainstream feature point-based visual odometry (ORB-SLAM3) has been used domestically and internationally to estimate camera trajectories in the digestive tract. However, due to the dim lighting and limited feature points in the digestive tract, the results have been limited. Direct methods, on the other hand, achieve better results in estimating camera trajectories for digestive endoscopy by optimizing matching through photometric error without extracting feature points.

[0053] 2.3.4 Acquisition and processing of digestive endoscopy trajectory In the DSO algorithm, allFrameHistory stores the poses of all frames. After scaling the pose-composed trajectory appropriately, the required pose set of the digestive endoscope is obtained. The data is in the form of The point set of . Distance to starting point Calculated using formula (1-5): (1-5) Set the threshold for consecutive decreases , if the conditions are met: , then it is believed that The mirror withdrawal phase begins, and the trajectory point set is divided accordingly.

[0054] The preprocessed trajectory data is associated with the esophageal center curve in the 3D reconstructed model, and a rough registration is performed first: use formula (1-3) to filter valid trajectory points for the two segments of trajectory data, apply similar data smoothing methods, and use formula (1-4) to find the feature points corresponding to the two segments of trajectory through similar steps and average them.

[0055] For a set of basic feature point pairs, that is, the starting point , curvature feature points ,end , forming the pose vector as shown in formula (1-6): , , , (1-6) In three-dimensional space, the rough transformation of the mirror trajectory to the model centerline can be achieved by the rotation matrix , and translation vectors Define the transformation relationship. For the model pose , , and trajectory description of endoscope posture , , , Describe Rotate parallel to The transformation of Describe Rotate parallel to Transformation, defining the basic operation of rotation As shown in formula (1-7). Describes the endoscope trajectory after the first two rotation transformations The feature point set in the model to the feature point set in the center line (starting point , curvature feature points ,end and more specified corresponding points). Finally, the coarse registration transformation is calculated by formula group (1-8):

[0056] in,

[0057]

[0058]

[0059]

[0060]

[0061] (1-7)

[0062]

[0063]

[0064]

[0065]

[0066] (1-8) 2.3 Central Curve Fitting and Refined Registration

[0067] In coarse registration Based on the above, the trajectory point set is transformed to develop along the -z direction, and then a step-by-step refinement strategy is used to fit the esophageal center curve in actual operation. First, based on formula (1-3), the trajectory data is preliminarily scanned along the -z direction with a large step size to obtain a rough center curve. ; Then, the iterative closest point (ICP) algorithm, that is, formula (1-9), is used to perform the initial registration of the coarse fitting center curve with the esophageal center curve extracted from the medical image to obtain the preliminary spatial transformation relationship .

[0068] (1-9) Further refine the registration process, re-segment the endoscopic trajectory data, and perform high-precision scanning with smaller step sizes to fit a more accurate esophageal center curve Finally, apply (1-9) again for fine registration until the preset registration accuracy is achieved, and obtain the fine registration transformation .

[0069] 2.4 Registration Completion and Application

[0070] When the registration is completed, the resulting spatial transformation matrix The endoscope position acquired from the real-time spatial posture can be accurately mapped to the corresponding medical image and the pre-established three-dimensional model, thereby realizing real-time navigation of esophageal endoscopic surgery.

[0071] A direct sparse odometry surgical navigation system is used to implement a direct sparse odometry surgical navigation method; the system comprises: Image center curve extraction module: used to process the patient's esophageal image data to extract the image center curve; 3D trajectory data pose vector acquisition module: used to process digestive endoscopy images and signal decomposition under sparse regularization constraints, determine the pose of the endoscope trajectory, pre-process the 3D trajectory data, and obtain the pose vector; Coarse registration module: used to associate the image center curve with the pose vector of the 3D trajectory data, and obtain the trajectory center curve through the rotation matrix; Fine-tuning registration module: obtains the final spatial transformation matrix through fine-tuning of the trajectory center curve and the image center curve; Mapping module: Through the final spatial transformation matrix, the endoscope position acquired in real-time spatial posture is accurately mapped to the corresponding medical image and pre-established three-dimensional model, thereby realizing real-time navigation of esophageal endoscopic surgery.

[0072] An electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, the steps of a surgical navigation method based on direct sparse odometry are implemented.

[0073] The electronic device may be a desktop computer, a laptop, a PDA, a cloud server, or other electronic device. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the figures are merely examples of electronic devices and do not limit the scope of the electronic device. The electronic device may include more, fewer, or different components than shown.

[0074] The processor can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0075] Memory can be an internal storage unit of an electronic device, such as its hard drive or memory. It can also be an external storage device, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card. Memory can also include both internal storage units and external storage devices. Memory is used to store computer programs and other programs and data required by the electronic device.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a portion of code, and the module, program segment or a portion of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0077] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0078] If the functions are implemented as software modules and sold or used as standalone products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. It should be noted that, in this document, relational terms such as first and second, etc., are used solely to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. Furthermore, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. Without further limitation, the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article, or device comprising the elements.

[0079] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, and the like made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention. It should be noted that similar reference numerals and letters denote similar items in the following figures. Therefore, once an item is defined in one figure, it need not be further defined or explained in subsequent figures.

[0080] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A surgical navigation method based on direct sparse odometry, characterized in that: Determine the image center curve of the esophagus based on the patient's esophageal imaging data and the curvature of the center point of the esophageal cross section; Determine the position of the endoscope trajectory based on the digestive endoscope image and signal decomposition under sparse regularization constraints; Determine the center curve of the endoscope trajectory according to the posture of the endoscope trajectory and the image center curve; The image center curve and trajectory center curve are automatically aligned to generate a spatial transformation matrix for surgical navigation.

2. The surgical navigation method based on direct sparse odometry according to claim 1, characterized in that: According to the digestive endoscopy image and the signal decomposition under the sparse regularization constraint, the pose of the endoscope trajectory is determined including: The frames of digestive endoscopy images are acquired and stored in a preset data structure. The first frame is designated as the key frame to complete initialization. The positions of feature points are determined based on the second frame and triangulation to create a motion model. Starting from the third frame, the predicted pose of each frame image is predicted based on the pose relationship between each frame image and its key frame; Each frame image is matched with the features of the key frame, and the photometric error is calculated to reflect the deviation between the current frame and the predicted pose to obtain the actual pose.

3. The surgical navigation method based on direct sparse odometry according to claim 2, characterized in that: After obtaining the actual pose, it also includes optimizing the actual pose: Perform initial optimization by optimizing the photometric error through a sliding window; The newly arrived frame is used as a keyframe, and its pose estimate will be passed to the backend. The backend applies a global Gauss-Newton method to optimize the assumed pose, reduce the photometric error, and iterate continuously until the preset convergence conditions are met.

4. The surgical navigation method based on direct sparse odometry according to claim 1, characterized in that: According to the patient's target image data and the curvature of the cross-sectional center point, the image center curve of the esophagus is determined including: Obtain the patient's esophageal imaging data and construct a continuous spatial point set based on the center point of the esophageal cross section; Extract the starting point, end point and image feature points of the esophagus from the spatial point set and connect them to form an image center curve containing the key morphological feature positions of the esophagus; Image feature point extraction includes determining curvature values ​​based on a set of spatial points, and arranging the curvature values ​​in descending order to obtain a curvature list; A threshold is set, and multiple spatial point subset intervals are obtained based on the curvature list and adjacent differences. The points that best match the preset esophageal empirical model are selected from the spatial point subset intervals as image feature points.

5. The surgical navigation method based on direct sparse odometry according to claim 4, characterized in that: According to the position and image center curve of the endoscope trajectory, the center curve of the endoscope trajectory is determined including: According to the pose calculation vector of the endoscope trajectory, the three-dimensional trajectory data is rotated to the coordinate system of the image center line, and the feature points of the trajectory data and the feature point set of the center curve are averaged and translated to complete the coarse registration transformation and obtain the trajectory center curve.

6. The surgical navigation method based on direct sparse odometry according to claim 1, 4 or 5, characterized in that: Automatically aligning the image center curve and the trajectory center curve to generate a spatial transformation matrix for surgical navigation includes: The initial refined registration includes: obtaining the center curve of the trajectory data, and initially registering it with the image center curve based on the iterative closest point algorithm to obtain the spatial transformation relationship; Secondary refined registration involves re-segmenting the endoscopic trajectory data and performing high-precision scanning with a smaller step size to fit a more accurate image center curve. This is then initially registered with the image center curve using an iterative closest point algorithm to obtain a precise transformation relationship. When the registration is completed, the obtained spatial transformation matrix accurately maps the endoscope position acquired in real-time spatial pose to the corresponding medical image and pre-established three-dimensional model, thereby realizing real-time navigation of esophageal endoscopic surgery.

7. A surgical navigation system based on direct sparse odometry, characterized in that: A surgical navigation method for implementing the direct sparse odometry method according to any one of claims 1 to 6; It includes: Image center curve extraction module: used to process the patient's esophageal image data to extract the image center curve; 3D trajectory data pose vector acquisition module: used to process digestive endoscopy images and signal decomposition under sparse regularization constraints, determine the pose of the endoscope trajectory, pre-process the 3D trajectory data, and obtain the pose vector; Coarse registration module: used to associate the image center curve with the pose vector of the 3D trajectory data, and obtain the trajectory center curve through the rotation matrix; Fine-tuning registration module: obtains the final spatial transformation matrix through fine-tuning of the trajectory center curve and the image center curve; Mapping module: Through the final spatial transformation matrix, the endoscope position acquired in real-time spatial posture is accurately mapped to the corresponding medical image and pre-established three-dimensional model, thereby realizing real-time navigation of esophageal endoscopic surgery.

8. An electronic device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, The processor is enabled to execute the surgical navigation method based on direct sparse odometry as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that The readable storage medium stores a computer program, and when the computer program is run on a computer, the computer is enabled to execute the surgical navigation method based on direct sparse odometry according to any one of claims 1 to 6.

Images