Navigation path multiplexing method and system

Abstract

The invention discloses a navigation path multiplexing method which comprises the following steps: S1, acquiring historical operation data which comprises a historical navigation path and a historical catheter posture; s2, acquiring a trachea three-dimensional model of a patient, and extracting a center line of the trachea three-dimensional model; s3, generating a current navigation path based on the historical navigation path and the trachea three-dimensional model; and S4, in the navigation process, performing real-time collision detection on the microscopic image by adopting the collision detection model, and adjusting the posture of the catheter according to a collision detection result. By adopting the method, the navigation path can be quickly obtained, rich operation data resources are fully utilized, the complexity of an existing path planning method is avoided, and the path planning efficiency is improved; the collision detection model is constructed through computer vision and is applied to real-time navigation of the catheter, so that the catheter can be effectively prevented from colliding with a tracheal wall or other lung tissues, and the navigation safety is improved.

Description

Technical Field

[0001] The present invention relates to the field of catheter navigation, and particularly to a navigation path multiplexing method and system. Background Art

[0002] In the field of modern medicine, Natural Orifice Transluminal Endoscopic Surgery (NOTES), as a minimally invasive surgical method, has gradually attracted attention and application. This surgery enters the body through natural body cavities (such as the oral cavity, nasal cavity, anus, etc.) for operation, and has significant advantages such as small trauma, fast recovery, and few postoperative complications. However, this surgical method also faces some technical challenges, especially in surgical path planning.

[0003] In Natural Orifice Transluminal Endoscopic Surgery, the navigation path planning of the endoscope or robotic catheter is one of the key factors for the success of the surgery. The surgical path needs to accurately avoid important tissues and organs while ensuring that the surgical instruments can smoothly reach the target lesion site. However, the current path planning methods have many deficiencies. For each patient, the interventional surgical equipment needs to re-plan the path. This "from scratch" planning method is not only time-consuming and laborious, but also increases the complexity of surgical preparation. The surgical team needs to manually design the navigation path according to the patient's individual anatomical structure, lesion location, and surgical objectives, which not only relies on the doctor's experience and judgment, but also requires comprehensive analysis with the help of various imaging examination results.

[0004] More importantly, the existing path planning methods cannot learn and accumulate experience from a large number of previous cases. Each surgical path planning is independent and cannot draw on previous successful experiences or avoid repeated mistakes. This mode lacking experience inheritance and data accumulation limits the further optimization and development of surgical path planning technology. When facing complex cases, doctors often can only rely on their limited experience for judgment, and cannot use big data analysis and artificial intelligence algorithms to improve the accuracy and safety of path planning.

[0005] In addition, with the continuous progress of medical technology, the intelligence and automation levels of surgical equipment are also constantly improving. However, the lag of path planning technology is in sharp contrast to the intelligent development of the equipment. This mismatch not only affects the surgical efficiency, but also may pose potential risks to the surgical safety of patients. Therefore, how to optimize the path planning technology of Natural Orifice Transluminal Endoscopic Surgery so that it can learn from previous cases and accumulate experience to achieve more efficient and accurate surgical navigation has become an urgent problem to be solved in the current field of medical engineering.

[0006] In summary, developing a path planning system that can combine big data analysis and artificial intelligence technology is of great significance for improving the safety and efficiency of natural orifice transluminal endoscopic surgery (NOTES). Such a system can not only simplify the preoperative preparation work, but also provide personalized and optimal surgical path planning solutions for each patient by learning and accumulating experience. Summary of the Invention

[0007] The purpose of the present invention is to provide a navigation path reuse method and system to solve the problems raised in the above background technology.

[0008] To achieve the above invention purpose, one aspect of the present invention provides a navigation path reuse method, including the following steps:

[0009] Step S1, obtaining historical surgical data, where the historical surgical data includes historical navigation paths and historical catheter postures;

[0010] Step S2, obtaining a three-dimensional model of the patient's trachea and extracting the centerline of the three-dimensional trachea model;

[0011] Step S3, generating a current navigation path based on the historical navigation path and the three-dimensional trachea model;

[0012] Step S4, during the navigation process, using a collision detection model to perform real-time collision detection on the endoscopic image and adjusting the catheter posture according to the collision detection result.

[0013] Further, the historical navigation path is the movement trajectory of the catheter in the trachea, which is composed of a sequence of position points; the historical catheter posture is the posture information of the catheter at each position, including the rotation angle and the bending degree, and is used to establish the matching relationship between the catheter and the three-dimensional trachea model.

[0014] Further, the method for extracting the centerline of the three-dimensional trachea model is: using a branch point recognition algorithm to extract the centerline of the trachea from the three-dimensional trachea model, and performing smoothing processing on the extracted centerline to remove abnormal data points to ensure its continuity and accuracy.

[0015] Further, step S3 includes:

[0016] Step S301, matching the three-dimensional trachea model with the historical navigation path to determine whether there is a historical navigation path with a matching degree not lower than the first threshold. If so, select the historical navigation path and execute steps S302 and S304; if not, execute steps S303 and S304;

[0017] Step S302: If the matching degree of the selected historical navigation path is not lower than the second threshold, select the historical navigation path as the current navigation path of the catheter; if the matching degree of the selected historical navigation path is lower than the second threshold, optimize the historical navigation path to generate the current navigation path of the catheter; the second threshold is greater than the first threshold.

[0018] Step S303: Automatically plan the catheter path according to the lesion location and the center line of the tracheal three-dimensional model to generate the current navigation path of the catheter.

[0019] Step S304: During the navigation process, adjust the catheter attitude in real time to ensure that the catheter advances along the planned path.

[0020] Further, the method for optimizing the historical navigation path in Step S302 is: using the A* algorithm, calculating the optimal path according to the tracheal three-dimensional model and its center line, and combining the catheter attitude information and the tracheal anatomical structure to generate a navigation path that meets the catheter operation requirements. The tracheal anatomical structure includes the bending degree and branching position of the trachea.

[0021] Further, the method for adjusting the catheter attitude in real time in Step S304 is: based on the real-time feedback data of the catheter, adjust the catheter attitude in real time through Kalman filtering to ensure that the catheter advances along the planned path; the real-time feedback data includes the catheter position, bending degree, and rotation angle.

[0022] Further, the first threshold is 85% and the second threshold is 95%.

[0023] Further, the collision detection model must be trained and tested before application. The method includes the following steps:

[0024] Step S401: Obtain the internal image of the patient's trachea through a bronchoscope or other endoscopic device.

[0025] Step S402: Preprocess the image, including denoising, enhancing contrast, and light equalization.

[0026] Step S403: Collect image data with real collision and non-collision scenarios from the preprocessed image. The image data includes the internal view of the trachea under different operating environments, catheter attitude, collision scenarios, tracheal anatomical structure, catheter type, and insertion depth.

[0027] Step S404: Manually or through an automated tool, annotate the image data to mark the areas where collisions may occur, as well as the current position and attitude of the catheter; manual annotation is performed by experts, and existing collision determination rules are used when annotating through an automated tool.

[0028] Step S405, using a regional convolutional neural network to analyze the image, extract features from the image, and identify possible collision risks;

[0029] Step S406, using the labeled training data set to train the collision detection model; during the training process, the collision and non-collision areas in the image are accurately distinguished by optimizing the loss function, and learning how to predict the probability of collision based on the features in the image;

[0030] Step S407, using different test sets to verify the generalization ability of the model, wherein the test sets include images with different anatomical features and catheter types; evaluating the performance of the model by calculating model indicators, wherein the model indicators include precision, recall rate and F1 value.

[0031] Furthermore, the method for performing real-time collision detection on the image under the mirror using the collision detection model comprises the following steps:

[0032] Step S411, during the navigation process, the bronchoscope is used to obtain the image under the bronchoscope as the input data for collision detection; each frame of the image is preprocessed, and collision detection is performed through the collision detection model;

[0033] Step S412, based on the collision detection model, analyze the relative position between the catheter and the surrounding tissue in the current image, calculate whether the catheter is close to or collides; output the collision probability, the higher the probability, the greater the risk of collision.

[0034] Furthermore, the method for adjusting the catheter posture according to the collision detection result includes automatic adjustment of the catheter posture and coordinated control, wherein:

[0035] The automatic adjustment of the catheter position is to automatically adjust the position and posture of the catheter by linking with the catheter control module when a collision risk is detected. The adjustment strategy includes fine-tuning the catheter direction and adjusting the catheter posture; the fine-tuning of the catheter direction is to calculate a small direction adjustment angle according to the direction of the collision, and change the direction of the catheter through the control system; the adjustment of the catheter posture is to adjust the curvature or angle of the catheter according to real-time image feedback to avoid collision if it is detected that the catheter contacts the tracheal wall or other tissues.

[0036] The collaborative control works in collaboration with the navigation system to automatically update the path planning of the catheter and dynamically adjust the path according to the collision detection results; when adjusting the path, the shortest path and the safest path are given priority to avoid the catheter colliding with the key parts of the trachea.

[0037] Another aspect of the present invention provides a navigation path reuse system, comprising a data acquisition module, a three-dimensional model module, a path generation module, and a collision detection module, wherein:

[0038] The data acquisition module is used to acquire historical surgical data, and the historical surgical data includes a historical navigation path and a historical catheter attitude;

[0039] The three-dimensional model module is used to acquire a three-dimensional model of a patient's trachea and extract the center line of the three-dimensional trachea model;

[0040] The path generation module is used to generate a current navigation path based on the historical navigation path and the three-dimensional trachea model;

[0041] The collision detection module is used to perform real-time collision detection on the endoscopic image by using a collision detection model during navigation, and adjust the catheter attitude according to the collision detection result.

[0042] Since the present system and method are adopted, compared with the prior art, the following advantages are achieved:

[0043] 1. The present invention utilizes historical surgical data and matches it with the three-dimensional model of the patient's trachea, and can quickly obtain a navigation path, making full use of rich surgical data resources, avoiding the cumbersome of existing path planning methods, and improving the efficiency of path planning;

[0044] 2. On the basis of reusing the navigation path, the present invention constructs a collision detection model by using computer vision and applies it to the real-time navigation of the catheter, which can effectively avoid the collision of the catheter with the tracheal wall or other lung tissues and improve the safety of navigation. Description of the Drawings

[0045] Figure 1 It is a flowchart of a navigation path reuse method.

[0046] Figure 2 It is a flowchart of a method for automatically finding a path through a historical path.

[0047] Figure 3 It is a flowchart of a method for training and testing a collision detection model.

[0048] Figure 4 It is a flowchart of a method for performing real-time collision detection on an endoscopic image by using a collision detection model. Detailed Embodiments

[0049] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0050] Such as Figure 1The following is a flowchart of the method of the present invention. An embodiment of the present invention provides a method for reusing a catheter navigation path, which can improve the efficiency and accuracy of path planning. The specific steps are as follows:

[0051] Step S1: Obtain historical surgical data, and extract the historical navigation path and the historical catheter pose from the historical surgical data.

[0052] The historical surgical data can come from past surgical records, surgical videos, or catheter paths manually marked by doctors. It includes path data and catheter pose data. The path data is the trajectory of the catheter moving in the trachea, usually represented by a sequence of position points; the catheter pose data is the pose information of the catheter at each position, including the rotation angle, bending degree, etc., which reflects the operating state of the catheter.

[0053] Extract the historical navigation path and catheter pose information from the historical data. Use clustering analysis to extract effective target paths from the historical data. During the historical navigation process, all positioning points, catheter directions, operation commands, etc. are recorded, and the starting position is marked; match the historical path information to the navigation path line, and remove path loops and invalid operation commands. The catheter pose information can also be used to establish a matching relationship between the catheter and the trachea model, helping to more accurately simulate the movement trajectory of the catheter.

[0054] Step S2: Obtain the three-dimensional model of the patient's trachea, and extract the centerline of the three-dimensional model of the trachea.

[0055] Scan the trachea area of the patient through medical imaging equipment (such as CT, MRI) to generate high-resolution three-dimensional image data, and reconstruct the three-dimensional model of the patient's trachea. Then, perform preprocessing such as denoising and filtering on the three-dimensional model to ensure the accuracy and integrity of the data.

[0056] Adopt a branch point recognition algorithm to extract the centerline information of the trachea from the three-dimensional model of the trachea. The centerline represents the geometric central axis of the tracheal duct and can provide an optimal path reference for the catheter. Smooth the extracted centerline to remove abnormal data points and ensure its continuity and accuracy, which serves as the basis for subsequent path planning.

[0057] Step S3: Generate the current navigation path based on the historical navigation path and the three-dimensional model of the trachea.

[0058] By comparing the similarity between the historical path and the current three-dimensional model of the patient's trachea, the system can screen out the historical path that is most suitable for the current patient.

[0059] After extracting the operation instructions and catheter attitude information of the effective target path from the historical surgical data, when switching the target path and detecting an existing historical path, automatic path finding is performed. The system can automatically identify and detect the historical path, and when the historical path is detected, it automatically finds the path and provides path recommendations for the doctor. The method of automatically finding the path through the historical path in step S3 is as Figure 2 shown and includes the following steps:

[0060] Step S301: Match the three-dimensional trachea model with the historical navigation path to determine whether there is a historical navigation path with a matching degree higher than 85%. If there is, select the i navigation path and execute step S302; if not, execute step S303.

[0061] Step S302: If the matching degree of the selected historical navigation path is not lower than 95%, directly select the historical navigation path as the current navigation path of the catheter; if the matching degree of the selected historical navigation path is lower than 95%, optimize the historical navigation path to generate the current navigation path of the catheter. Among them, the path calculation algorithm uses the A* algorithm to calculate the shortest path or the optimal path according to the three-dimensional model and centerline information of the trachea. By combining the catheter attitude information and the tracheal anatomical structure, the path planning algorithm can take into account factors such as the curvature of the trachea and the branch position to generate a path that conforms to the actual operation.

[0062] Step S303: Automatically plan the catheter path according to the lesion location and the centerline of the three-dimensional trachea model to generate the current navigation path of the catheter.

[0063] Step S304: During the navigation process, adjust the catheter attitude in real time to ensure that the catheter advances along the planned path.

[0064] The catheter attitude adjustment algorithm is based on the real-time feedback data of the catheter, including the catheter position, bending degree, and rotation angle, and adjusts the catheter attitude in real time through Kalman filtering to ensure that it advances smoothly along the planned path.

[0065] Step S4: During the navigation process, use a collision detection model to perform real-time collision detection on the microscopic image and adjust the catheter attitude according to the collision detection result.

[0066] Before using the collision detection model, it needs to be trained and tested. The specific method is as Figure 3 shown and includes the following steps:

[0067] Step S401: To train the collision detection model, collect a large amount of microscopic image data with real collision and non-collision scenarios. These images include the internal view of the trachea under different operating environments, different attitudes of the catheter, and possible collision scenarios, and also include various tracheal anatomical structures, catheter types, and different insertion depths.

[0068] Step S402: Preprocess the image, including denoising, enhancing contrast, and equalizing illumination. Among them:

[0069] Denoising is to use an image denoising algorithm (such as Gaussian filtering, median filtering, etc.) to remove the noise in the image and ensure the clarity of the image.

[0070] Enhancing contrast is to use an image enhancement algorithm (such as histogram equalization) to increase the contrast of the image and help the model better identify the boundary between the catheter and the surrounding tissue.

[0071] Illumination equalization is to process the uneven illumination problem in the image through local illumination equalization technology to ensure the overall quality of the image.

[0072] Step S403: Collect collision data. From the preprocessed images, collect image data with real collision and non - collision scenarios. These image data include the internal view of the trachea under different operating environments, different postures of the catheter, and possible collision scenarios, and include various tracheal anatomical structures, catheter types, and different insertion depths.

[0073] Step S404: Label the collected image data. Manually by expert doctors or use existing collision determination rules through an automated tool to label the microscopic images, marking the areas where collisions may occur (such as the tracheal wall, other tissues) and the current position and posture of the catheter.

[0074] Step S405: Use a region - based convolutional neural network to analyze the image. Extract features from the image to identify possible collision risks.

[0075] Step S406: Use the labeled training data set to train the collision detection model. During the training process, optimize the loss function so that the model can accurately distinguish between collision and non - collision regions in the image and learn how to predict the probability of a collision occurring based on the features in the image.

[0076] Step S407: Use different test sets (including images with different anatomical features and catheter types) to verify the generalization ability of the model. Evaluate its performance by calculating metrics such as the accuracy, recall rate, and F1 - value of the model to ensure that the model can work reliably in multiple environments.

[0077] Then use the collision detection model to perform real - time collision detection on the microscopic image. According to the collision detection results, adjust the position of the catheter, such as Figure 4 shown, including the following steps:

[0078] Step S411: During the operation, the bronchoscope image is transmitted to the system as input data for collision detection. The system processes each frame of the image and performs collision detection using a trained collision detection model.

[0079] Step S412: Based on the trained model, the system analyzes the relative position between the catheter and surrounding tissues (such as tracheal wall, branch points, etc.) in the current image and calculates whether the catheter is close to or collides with the catheter. The system will output the probability of collision, and the higher the probability, the greater the risk of collision.

[0080] When a collision risk is detected, the system automatically adjusts the position and posture of the catheter by linking with the catheter control module. The adjustment strategies include fine-tuning the catheter direction and adjusting the catheter posture. Fine-tuning the catheter direction is based on the direction of the collision. The system will calculate a small direction adjustment angle and change the direction of the catheter through the control system. Adjusting the catheter posture is to detect that a part of the catheter contacts the tracheal wall or other tissues. The system will adjust the curvature or angle of the catheter based on real-time image feedback to avoid collision. In addition, the catheter path planning can be automatically updated, and the path can be dynamically adjusted based on real-time collision detection results. When adjusting the path, the system gives priority to the shortest path and the safest path to avoid the catheter colliding with key parts of the trachea.

[0081] At the end, reverse path planning is performed to withdraw from the current catheter position to the starting position by automatically controlling the direction and retreating.

[0082] At the same time, the present invention also provides user interaction and feedback functions. Through a visual interface, the system displays the matching status of the catheter and the target path in real time, and prompts the doctor with possible collision areas through color coding or marking. For example, when the catheter is too close to the collision area, a red warning is displayed; if the risk is small, a yellow prompt is displayed.

[0083] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A navigation path multiplexing method, characterized in that, Including: Step S1: Obtain historical surgical data, where the historical surgical data includes a historical navigation path and a historical catheter pose; Step S2: Obtain a three-dimensional model of the patient's trachea and extract the centerline of the three-dimensional trachea model; Step S3: Generate a current navigation path based on the historical navigation path and the three-dimensional trachea model; Step S4: During navigation, use a collision detection model to perform real-time collision detection on the endoscopic image and adjust the catheter pose according to the collision detection result.

2. The navigation path multiplexing method according to claim 1, wherein The historical navigation path is the movement trajectory of the catheter in the trachea, which is composed of a sequence of position points; the historical catheter pose is the pose information of the catheter at each position, including the rotation angle and the degree of bending, and is used to establish a matching relationship between the catheter and the three-dimensional trachea model.

3. The navigation path multiplexing method according to claim 1, wherein The method for extracting the centerline of the three-dimensional trachea model is as follows: Use a branch point recognition algorithm to extract the centerline of the trachea from the three-dimensional trachea model, and perform smoothing processing on the extracted centerline to remove abnormal data points to ensure its continuity and accuracy.

4. The navigation path multiplexing method according to claim 1, characterized in that Step S3 includes: Step S301: Match the three-dimensional trachea model with the historical navigation path to determine whether there is a historical navigation path with a matching degree not lower than the first threshold. If there is, select the historical navigation path and execute Step S302 and Step S304; if not, execute Step S303 and Step S304; Step S302: If the matching degree of the selected historical navigation path is not lower than the second threshold, select the historical navigation path as the current navigation path of the catheter; if the matching degree of the selected historical navigation path is lower than the second threshold, optimize the historical navigation path to generate the current navigation path of the catheter; the second threshold is greater than the first threshold; Step S303: Automatically plan the catheter path according to the lesion location and the centerline of the three-dimensional trachea model to generate the current navigation path of the catheter; Step S304: During navigation, adjust the catheter pose in real time to ensure that the catheter advances along the planned path.

5. The navigation path multiplexing method according to claim 4, wherein The method for optimizing the historical navigation path in Step S302 is as follows: Use the A* algorithm to calculate the optimal path according to the three-dimensional trachea model and its centerline, and combine the catheter pose information and the tracheal anatomical structure to generate a navigation path that meets the catheter operation requirements. The tracheal anatomical structure includes the degree of bending and the branch position of the trachea.

6. The navigation path multiplexing method according to claim 4, wherein The method for adjusting the catheter pose in real time in Step S304 is as follows: Based on the real-time feedback data of the catheter, use Kalman filtering to adjust the catheter pose in real time to ensure that the catheter advances along the planned path; the real-time feedback data includes the catheter position, the degree of bending, and the rotation angle.

7. The navigation path multiplexing method according to any one of claims 4 to 6, characterized in that The first threshold is 85%, and the second threshold is 95%.

8. The navigation path multiplexing method according to claim 1, characterized in that, Before the collision detection model is applied, it must be trained and tested. The method includes the following steps: Step S401: Obtain the internal image of the patient's trachea through a bronchoscope or other endoscopic device; Step S402: Perform preprocessing on the image, including denoising, enhancing contrast, and light equalization; Step S403, collecting image data with real collision and non-collision scenes from the preprocessed images, the image data including endotracheal views under different operating environments, catheter postures, collision scenes, tracheal anatomical structures, catheter types, and insertion depths; Step S404, manually or through an automated tool, annotating the image data to mark areas where collisions may occur and the current position and posture of the catheter; manual annotation is performed by an expert, and the automated tool annotation uses existing collision determination rules; Step S405, using a regional convolutional neural network to analyze the image, extract features from the image, and identify possible collision risks; Step S406, using the labeled training data set to train the collision detection model; during the training process, the collision and non-collision areas in the image are accurately distinguished by optimizing the loss function, and learning how to predict the probability of collision based on the features in the image; Step S407, using different test sets to verify the generalization ability of the model, wherein the test sets include images with different anatomical features and catheter types; evaluating the performance of the model by calculating model indicators, wherein the model indicators include precision, recall rate and F1 value.

9. The navigation path multiplexing method according to claim 1, characterized in that, The method for performing real-time collision detection on the image under the mirror using the collision detection model comprises the following steps: Step S411, during the navigation process, the bronchoscope is used to obtain the image under the bronchoscope as the input data for collision detection; each frame of the image is preprocessed, and collision detection is performed through the collision detection model; Step S412, based on the collision detection model, analyze the relative position between the catheter and the surrounding tissue in the current image, calculate whether the catheter is close to or collides; output the collision probability, the higher the probability, the greater the risk of collision.

10. The navigation path multiplexing method according to claim 1, wherein The method for adjusting the catheter posture according to the collision detection result includes automatic adjustment of the catheter posture and cooperative control, wherein: The automatic adjustment of the catheter posture is to automatically adjust the position and posture of the catheter by linking with the catheter control module when a collision risk is detected. The adjustment strategy includes fine-tuning the catheter direction and adjusting the catheter posture; the fine-tuning of the catheter direction is to calculate a small direction adjustment angle according to the direction of the collision, and change the direction of the catheter through the control system; the adjustment of the catheter posture is to adjust the curvature or angle of the catheter according to real-time image feedback to avoid collision if it is detected that the catheter contacts the tracheal wall or other tissues; The collaborative control works in collaboration with the navigation system to automatically update the path planning of the catheter and dynamically adjust the path according to the collision detection results; when adjusting the path, the shortest path and the safest path are given priority to avoid the catheter colliding with the key parts of the trachea.

11. A navigation path multiplexing system, characterized in that, It includes data acquisition module, 3D model module, path generation module and collision detection module, among which: The data acquisition module is used to acquire historical surgical data, wherein the historical surgical data includes historical navigation paths and historical catheter postures; The three-dimensional model module is used to obtain a three-dimensional model of the patient's trachea and extract the center line of the three-dimensional trachea model; The path generation module is used to generate a current navigation path based on the historical navigation path and the trachea three-dimensional model; The collision detection module is used to perform real-time collision detection on the microscopic image using a collision detection model during navigation, and adjust the catheter attitude according to the collision detection results.

Images