A pulmonary lesion navigation path planning and target branch decision system

Abstract

The present application relates to a lung focus navigation path planning and target branch decision system, which adopts the following mode: pre-processing the input CT image, extracting the lung region CT image, segmenting the airway tree, arteriovenous and focus from the extracted lung region CT image; extracting the airway tree center line image, obtaining the key points, constructing the cube with the focus center as the center of gravity, screening the optimal operation point from the key points in the cube, and outputting the target path from the trachea entrance to the focus according to the optimal operation point; the present application comprehensively considers whether the bronchoscope can pass through the planned path and the blood vessel distribution in the operation point to focus puncture path, screens the point which meets the bronchoscope short walking distance and the operation point to focus puncture path without blood vessel distribution at the same time, and the present application improves the accuracy of path planning and is more in line with the actual clinical demand.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pulmonary lesion navigation, and more particularly to a pulmonary lesion navigation path planning and target branch decision system. BACKGROUND

[0002] Lung cancer is the most common primary lung malignancy, which is harmful to human health, and has become the cancer with the highest incidence and mortality rate in the world. Early diagnosis and treatment of lung micro-nodules (lesions) is a key measure to reduce lung cancer mortality. Bronchoscopy is the safest and most superior method for diagnosing lung peripheral micro-nodule lesions. At present, the most reliable bronchoscopy operation route map is obtained by observing the lesion area by doctors reading films and then relying on surgical experience to sketch. However, manual planning of the operation path is a labor-intensive task, which is time-consuming and laborious, and the sketched results differ greatly among doctors with different years of clinical experience. Therefore, a fully automatic, high-precision and repeatable path planning method is needed.

[0003] With the development of computer hardware resources and auxiliary diagnosis technology, fully automatic operation path planning methods have been greatly developed. Existing path planning methods include deep learning-based methods and learning-free methods. The deep learning-based method uses pairs of CT images and gold standard to train the network model, and learns to plan the operation route map for the unseen CT images. This method requires doctors to sketch a large number of gold standard samples in advance, and due to the differences between imaging devices and the diversity of lesion distribution among patients, the accuracy of the network model in actual application is difficult to guarantee. The learning-free method directly searches the path according to the bronchial topological structure. This method does not need to train the gold standard, and uses a breadth-first search or depth-first search algorithm to calculate the distance from the tracheal entrance to the operation point and the distance from the operation point to the lesion center point. The optimal path takes the shortest path from the tracheal entrance to the lesion as the target.

[0004] There are many invention patents and literatures about learning-free methods for surgical path planning using medical data visualization, optimizing path cost functions and virtual endoscopy technology, such as:

[0005] The Chinese invention patent document with publication date of July 31, 2020 and publication number CN114066906A discloses a navigation path planning method, system and readable storage medium. The method first segments the natural cavity in the CT image using the region growing method, and then takes the natural cavity distance transform volume data as the research object, with the minimum cost function as the target, to obtain the best navigation path from the starting point to the target point on the cavity center line, thereby improving the accuracy of navigation path planning. However, the invention needs to modify the target path, and the peak pixel points in the connected domain of the distance transform volume data are modified to path points to ensure that the target path is on the center line. It can be considered to directly plan the path on the airway tree center line image.

[0006] The Chinese invention patent document with publication date of July 13, 2021 and publication number CN113112609A discloses a navigation method and system for lung biopsy bronchoscopy. The method is based on a preset deep learning lung nodule recognition model and virtual endoscopy technology, and can realize intraoperative navigation through lesion positioning and lesion marking, with the advantages of accurate diagnosis and high efficiency. The virtual endoscopy technology can segment the airway tree from the CT image and generate a virtual route image from the airway entrance to the lesion using image rendering technology. The segmentation method is a fast region growing algorithm based on a queue structure. However, existing studies have shown that the segmentation method based on deep learning has a much better ability to preserve the topological structure of the airway tree than traditional methods (such as region growing and threshold-based methods). Traditional methods that rely on pixel intensity differences can successfully segment the trachea and main bronchus, but as the airway branches into finer bronchi, the intensity contrast between the airway lumen and the wall gradually weakens, and these methods usually cannot extract peripheral bronchi. The path planning method is the Dijkstra algorithm, a shortest path search algorithm based on the greedy strategy. When searching for the shortest path from the tracheal entrance to the lesion, it needs to calculate the shortest path from the starting node to all nodes, and construct a path tree by increasing the path length point by point until the target point is marked, involving a large number of irrelevant node operations.

[0007] It can be seen that these successful automatic surgical path planning methods are prone to problems such as missing lung airway tree segmentation and redundant computation. At the same time, these methods ignore the influence of factors such as whether the bronchoscope can pass through the selected path, the size relationship between the bronchoscope and the airway lumen cross section, and whether the selected surgical point is suitable for puncture, i.e. the blood vessel distribution in the puncture path from the surgical point to the lesion, which does not conform to the actual clinical situation. A lung lesion navigation path planning and target branch decision system is needed to overcome the above-mentioned defects. SUMMARY

[0008] The technical problem solved by the present application is to provide a path planning and target branch decision system for lung lesion navigation to solve the above-mentioned defects of the prior art.

[0009] The technical solution adopted by the present application to solve its technical problem is:

[0010] The present application provides a path planning and target branch decision system for lung lesion navigation, wherein the system comprises a data input unit, a data processing unit and a data output unit.

[0011] The data input unit is configured to receive an input CT image.

[0012] The data processing unit is configured to pre-process the input CT image, extract a lung region CT image from the pre-processed CT image, segment the airway tree, the artery and vein and the lesion from the extracted lung region CT image, extract a centerline image of the airway tree segmentation mask, extract key points from the airway tree centerline image according to a key point extraction algorithm, encode the key points, the key points being end points and branch points on the airway tree centerline, construct a cube with the center of the lesion as the center of gravity, delete key points that cannot be passed through by a bronchoscope in the cube, determine a first common parent node of the key points through the same part in the key point code, find a node with the shortest distance to the first common parent node and the center of the lesion on the branch segment connected by each key point and its parent node, and retain a set number of nodes as operation points; determine whether the multiple operation points remain unchanged after continuous expansion of the cube for a set number of times, stop expanding the cube if yes, otherwise continue expanding the cube according to a set rule, and repeat the operation on the newly included key points: delete key points that cannot be passed through by a bronchoscope in the cube, determine a first common parent node of the remaining key points, find a node with the shortest distance to the first common parent node and the center of the lesion on the branch segment connected by each key point and its parent node, and retain a set number of nodes as operation points; and select an optimal operation point according to the length of the walking distance and the distribution of blood vessels in the lesion puncture path of the finally determined multiple operation points, and obtain a target path from the tracheal entrance to the lesion according to the optimal operation point.

[0013] The data output unit is configured to output the obtained target path.

[0014] The path planning and target branch decision system for lung lesion navigation provided by the present application, wherein the lung region CT image is extracted from the pre-processed CT image by using a threshold method to convert the CT image into a binary image, wherein the lung region is set as the foreground and other tissue information is set as the background, and a plurality of connected domains are obtained.

[0015]

[0016] ​By constraining the volume and eccentricity of the connected domain, the two largest connected domains are taken as the masks of the left lung and the right lung;

[0017] The masks of the left lung and the right lung in the binary image are hole-filled using morphological operations to eliminate the interference of the noise in the lung region mask

[0018] The isolated pixel points at the boundary of the left lung and right lung masks in the binary image are merged using convex hull calculation to form a more complete lung region mask.

[0019] The preprocessed CT image and the lung region mask in the binary image are multiplied pixel by pixel to obtain a CT image containing only the lung region, and the tissue information outside the lung region is removed from the preprocessed CT image.

[0020] The path planning and target branch decision system for lung lesion navigation, wherein the airway tree, arteriovenous and lesion are segmented from the extracted lung region CT image, comprises:

[0021] The lung region CT image is cropped to the minimum bounding box of the lung as an effective input region;

[0022] The effective input region is cropped into multiple volume blocks in a sliding window manner, and the multiple volume blocks are input into a neural network for model training.

[0023] After the model training is completed, it can be directly applied to the clinic to segment the airway tree structure, lung arteriovenous structure and lung lesion structure in the preprocessed CT image.

[0024] The path planning and target branch decision system for lung lesion navigation, wherein the extracted airway tree centerline image adopts:

[0025] The segmented results are data-processed, specifically, the airway tree mask is extracted from the segmented results to obtain a binary image containing only the airway tree mask and the background, and a three-dimensional thinning algorithm is used to reduce the airway tree mask in the binary image to a single-voxel width to obtain an airway tree centerline image.

[0026] The path planning and target branch decision system for lung lesion navigation, wherein the key points are extracted from the airway tree centerline image according to the key point extraction algorithm, and the key points are encoded using:

[0027] According to the number of voxels in the 26-neighborhood of each node on the centerline, the nodes on the centerline are included as end points, edge points and bifurcation points, wherein the number of adjacent voxels of the end point is 1, the number of adjacent voxels of the edge point is 2, and the number of adjacent voxels of the bifurcation point is not less than 3.

[0028] The key points are encoded using a breadth-first search algorithm.

[0029] The key point information is saved as a dictionary structure, wherein the key is the key point coordinate, and the value is the corresponding code of the key point.

[0030] The lung lesion navigation path planning and target branch decision system, wherein a cube with the lesion center as the gravity center is constructed, the key points that cannot be passed through by the bronchoscope in the cube are deleted, the first common parent node of the key points is determined through the same part in the code of the remaining key points, the node with the minimum distance from the first common parent node and the lesion center on the branch segment connected with the parent node of each key point is found through iteration, and a set number of nodes are reserved, and the operation of marking the operation points is adopted:

[0031] The center point coordinates (X pn ,Y pn ,Z pn ) of the pulmonary nodule are calculated, the center point of the pulmonary nodule is taken as the gravity center of the cube, the edge length is increased by a preset length from 0, the cube is gradually expanded outward, until the key points are contained in the cube, and the key points in the cube are established as a set.

[0032] The average value of the minimum diameters of the airway lumen at the key points and the parent nodes is taken to calculate the cross-sectional area of the connected branch segment, if the area is smaller than the size of the bronchoscope, it is considered that the bronchoscope cannot pass through the branch, and the key point is removed from the set.

[0033] The first common parent node of the key points is determined through the same part in the code of the remaining key points, the node with the minimum distance from the first common parent node and the lesion center on the branch segment connected with the parent node of each key point is found through iteration.

[0034] The nodes are sorted from low to high according to the distance, the first set number of points are reserved, and the first set number is not enough, then the empty space is processed.

[0035] The lung lesion navigation path planning and target branch decision system, wherein whether the plurality of operation points currently determined remain unchanged after the cube is continuously expanded for a set number of times is judged, if yes, the cube expansion is stopped, otherwise the cube is continuously expanded according to the set rule, and the operation of repeating the above step on the newly included key points is adopted.

[0036] Continue to increase the size of the cube by the preset length, and repeat the operation on the newly included key points: delete the key points that cannot be passed through by the bronchoscope in the cube, determine the first common parent node of the remaining key points, find the node with the shortest distance to the first common parent node and the lesion center on the branch segment connected with the parent node, retain a set number of nodes as operation points, compare the path length of the first set number of new operation points with the first set number of operation points generated in the last stage, replace or retain the operation points in the last stage; if the first set number of operation points determined continuously for the second set number of times of increasing the size of the cube do not change, stop expanding the cube outward. The average value of the key points in the cube and the minimum diameter of the airway lumen at the parent node is taken to calculate the cross-sectional area of the connected branch segment:

[0037] The airway tree mask binary image is subjected to dilation morphological processing.

[0038] The voxel values of the foreground and background are exchanged.

[0039] The distance transformation operation is performed to obtain the Euclidean distance image of each zero voxel in the airway tree to the nearest non-zero airway wall.

[0040] The distance image is multiplied by the airway tree centerline image to obtain the nearest distance from the airway tree centerline to the airway wall.

[0041] The double nearest distance is taken as the minimum diameter of the airway lumen corresponding to any point, which can be used to calculate the cross-sectional size of the lumen, and the average value of the cross-sectional size of the lumen at the key points and the parent node is the cross-sectional area of the connected branch segment.

[0042] The lung lesion navigation path planning and target branch decision system of the application, wherein,

[0043] The optimal operation point is screened out according to the length of the path and the blood vessel distribution in the puncture path to the lesion, and the target path from the tracheal entrance to the lesion is output according to the optimal operation point.

[0044] Whether there is a pulmonary artery and vein in the puncture path of each operation point to the lesion is calculated to measure the blood vessel distribution of the operation point to the lesion.

[0045] The following decisions are made in combination with the length of the path and the blood vessel distribution in the puncture path:

[0046] 1) If there is no blood vessel distribution in the puncture path of the operation point corresponding to the shortest path in the first set number of paths to the lesion, the point corresponding to the shortest path is taken as the optimal operation point, and the corresponding three-dimensional space coordinates are output.

[0047] 2) If the shortest path of the first set number of paths corresponds to a puncture path from the operation point to the lesion with blood vessel distribution, a new point without blood vessel distribution in the puncture path to the lesion is calculated from the back to the front on the branch segment where the operation point is located, and according to the triangular three-side relationship, the new point walking path is further shortened; if there is no point without blood vessel distribution in the puncture path to the lesion on the branch, the operation point corresponding to the second shortest path is observed, if there is no blood vessel distribution in the puncture path from the operation point to the lesion, the three-dimensional space coordinates of the operation point are directly output, otherwise the same searching operation is performed on the branch segment of the second shortest path: a new point without blood vessel distribution in the puncture path is calculated from the operation point to the front, and so on, until the operation point that meets the walking path short and the puncture path to the lesion without blood vessel distribution is selected, and the three-dimensional space coordinates are output;

[0048] After obtaining the three-dimensional space coordinates of the optimal operation point, all coordinate points of the target branch are sequentially output from the back to the front to form a list; the list is reversed to obtain the coordinate information of each node on the path from the tracheal entrance to the optimal operation point.

[0049] The beneficial effects of the present application are that the technical scheme of the present application proposes a full-automatic, learning-free, simple and efficient path planning and target branch decision method, which adheres to the principle that the shortest operation path is not the optimal path, and also considers whether the bronchoscope can pass through the planned path and the blood vessel distribution in the puncture path from the operation point to the lesion, so a series of operation points meeting the conditions are obtained, forming a plurality of branch paths from the tracheal entrance to the operation point, the path that the bronchoscope cannot pass through can be easily excluded, and then the optimal operation point that meets the bronchoscope walking path short and the puncture path from the operation point to the lesion without blood vessel distribution is screened out. The present application improves the accuracy of path planning and is more in line with the actual clinical needs. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the present application will be further described below with reference to the drawings and embodiments. The drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creating any inventive labor:

[0051] Figure 1 is the flow chart of the path planning and target branch decision system for lung lesion navigation of the preferred embodiment of the present application;

[0052] Figure 2 is the decision process flow chart of the path planning and target branch decision system for lung lesion navigation of the preferred embodiment of the present application;

[0053] Figure 3The principle block diagram of a path planning and target branch decision system for pulmonary lesion navigation is a preferred embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the following will make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the protection scope of the present application.

[0055] The path planning and target branch decision system for pulmonary lesion navigation is a preferred embodiment of the present application, as shown in Figure 1 , and referring to Figure 2 and Figure 3 , the system comprises a data input unit 100, a data processing unit 101 and a data output unit 102.

[0056] The data input unit 100 is configured to receive an input CT image.

[0057] The data processing unit 101 is configured to pre-process the input CT image, extract a lung region CT image from the pre-processed CT image, segment airway trees, arteries and veins and lesions from the extracted lung region CT image, extract a centerline image of the airway tree segmentation mask, extract key points from the airway tree centerline image according to a key point extraction algorithm, encode the key points, the key points being end points and bifurcation points on the airway tree centerline, construct a cube with the center of the lesion as the center of gravity, delete key points in the cube that cannot be passed through by a bronchoscope, determine a first common parent node of the key points through the same part in the key point coding, find a node with the shortest distance to the first common parent node and the center of the lesion on a branch segment connected by each key point and its parent node, and retain a set number of nodes as operation points. It is judged whether the multiple operation points determined currently remain unchanged after the cube is continuously expanded for a set number of times, if yes, the cube expansion is stopped, otherwise the cube is continuously expanded according to a set rule. The operation of deleting key points in the cube that cannot be passed through by a bronchoscope, determining a first common parent node of the remaining key points, finding a node with the shortest distance to the first common parent node and the center of the lesion on a branch segment connected by each key point and its parent node, and retaining a set number of nodes as operation points is repeated for the newly included key points. The optimal operation point is screened out according to the length of the walking distance and the blood vessel distribution in the lesion puncture path of the finally determined multiple operation points, and the target path from the tracheal entrance to the lesion is obtained according to the optimal operation point.

[0058] The data output unit 102 is configured to output the obtained target path.

[0059] The technical scheme of the present application provides a full-automatic, learning-free, simple and efficient path planning and target branch decision method, while adhering to the principle that the shortest surgical path is not the optimal path, also considering whether the bronchoscope can pass through the planned path and the blood vessel distribution in the puncture path from the surgical point to the lesion, so a series of surgical points meeting the conditions are obtained, forming multiple branch paths from the tracheal entrance to the surgical point, the paths that the bronchoscope cannot pass through can be easily excluded, and then the optimal surgical point that meets the bronchoscope's short travel distance and the absence of blood vessel distribution in the puncture path from the surgical point to the lesion is screened out, the present application improves the accuracy of path planning and is more in line with actual clinical needs.

[0060] Specifically, the method steps adopted by the system are as follows:

[0061] S01: Preprocessing the input CT image, and extracting a lung region CT image from the preprocessed CT image;

[0062] S02: Segmenting the airway tree, arteries and veins, and lesions from the extracted lung region CT image;

[0063] S03: Extracting an airway tree centerline image, extracting key points from the airway tree centerline image according to a key point extraction algorithm, and encoding the key points; the key points are end points and branch points on the airway tree centerline;

[0064] The airway tree centerline is used to represent the center axis of the airway lumen in medical imaging, which is a single-pixel representation form similar to the airway contour, and can reflect the main direction of the airway. Once the airway tree centerline is extracted, doctors can use it to measure the diameter, length, and curvature of the airway, to help locate lesions and evaluate the extent of the lesions. The airway tree centerline is obtained by data processing on the segmented airway tree mask, specifically: using a three-dimensional thinning algorithm to process from the outer contour of the airway tree mask to the center, checking in the 26-neighborhood of each voxel, and iteratively deleting external voxels until the airway tree structure is reduced to a single-voxel width.

[0065] S04: Constructing a cube with the lesion center as the center of gravity, deleting the key points in the cube that the bronchoscope cannot pass through, determining the first common parent node of the remaining key points, finding the node with the shortest distance to the first common parent node and the lesion center on the branch segment connected by each key point and its parent node, and retaining a certain number of nodes as surgical points;

[0066] Generally, the airway tree of a human being has about 24 branches from the first level to the alveoli, which is like an inverted tree, so it can be extended to the concept of tree structure in data structure. The trachea is divided into left and right main bronchi from the bifurcation, and the tracheal entrance can be regarded as a root node.

[0067] The lesion is an irregular three-dimensional shape, and the lesion is limited in a smallest cube which just surrounds the lesion, and the geometric center of the cube is the center of the lesion.

[0068] S05: determining whether the plurality of surgical points currently determined remain unchanged after the cube is continuously expanded for a set number of times, if yes, stopping the expansion of the cube, otherwise continuing to expand the cube according to the set rule, and repeating the operation for the newly included key points each time: deleting the key points in the cube which cannot be passed through by the bronchoscope, determining the first common parent node of the remaining key points, finding the node with the minimum distance to the first common parent node and the center of the lesion on the branch segment connected with the parent node for each key point, and retaining a set number of nodes as surgical points;

[0069] S06: screening the optimal surgical point according to the length of the walking distance and the distribution of blood vessels in the puncture path to the lesion from the plurality of surgical points finally determined, and outputting the target path from the tracheal entrance to the lesion according to the optimal surgical point;

[0070] It should be noted that in step one, the input CT image is preprocessed, including adjusting the window level and window width, and numerical normalization, but not including voxel resampling operation. Although the actual spatial size of a single voxel in different input images may not be consistent, resampling will lead to discontinuity and incompleteness of the segmentation mask in the gold standard, which is not conducive to the neural network to learn the effective representation of the slender tubular structure.

[0071] In step one, the lung region CT image is extracted. Considering the effective airway tree, arteries and veins, the lesion is mainly limited in the lung tissue region, and it is reasonable to only take the lung as the region of interest for analyzing the anatomical information.

[0072] The lung region can be extracted in the following ways: 1) using threshold method to convert the CT image into a binary image, wherein the lung region is set as foreground and other tissue information is set as background, and a plurality of connected domains are obtained; 2) by constraining the volume and eccentricity of the connected domain, the two largest connected domains are taken as the mask of the left lung and the right lung; 3) using morphological operation to fill the holes in the mask of the left lung and the right lung in the binary image to eliminate the interference of noise in the lung region mask; 4) using convex hull calculation to merge the isolated pixel points at the boundary of the left lung and right lung mask in the binary image to form a more complete lung region mask; 5) multiplying the preprocessed CT image and the lung region mask in the binary image pixel by pixel to obtain a CT image containing only the lung region, and the tissue information outside the lung region is removed from the preprocessed CT image; and cropping the lung region CT image to the minimum bounding box of the lung as the effective input region.

[0073] In step two: the effective area is cropped into smaller volume blocks in a sliding window manner and input into the neural network for model training. By considering the topological relationship of the airway tree, pulmonary artery and vein, i.e. the voxels of the three are mutually exclusive and do not contain each other. The introduction of topological prior can avoid the neural network from learning incorrect feature representation, ensuring the accuracy of the extracted airway tree, artery and vein data, especially the topological correctness of the airway tree.

[0074] In step three: after the neural network training is completed, the network model is used to segment the airway tree, artery and vein and lesions in the preprocessed CT image to obtain the coordinate information in the three-dimensional space. The segmented airway tree mask is processed, and a three-dimensional thinning algorithm is used to reduce the binary airway tree mask image to a single voxel width (specifically, check within the 26-neighborhood of each voxel and iteratively delete the external voxels until the voxel width no longer changes), which is used as the airway tree centerline image.

[0075] For the extraction of key points in the airway tree centerline image: according to the number N of voxels in the 26-neighborhood of each node on the centerline, the nodes on the centerline are defined as end points (N = 1), edge points (N = 2) and bifurcation points (N ≥ 3).

[0076] The key points (end points and bifurcation points) are coded using a breadth-first search algorithm, in which the tracheal starting point is set to 1, its child node, which is also the first bifurcation point, is set to 1.1, the child nodes of the first bifurcation point are set to 1.1.1, 1.1.2, … 1.1.M, where M represents the number of child nodes of the first bifurcation point. Each level down, the coding length increases by one bit, indicating the level number of the key point. By finding the same part in the coding of multiple key points, their first common parent node can be easily found (assuming key points 1.1.1, 1.1.1.1, 1.1.2.1, compare each bit of the coding from left to right, if a bit is different, the common part is from the left to that position left bit. After comparison, it is found that the third bit coding is different, so the common part is from the left to the second bit coding, and it is known that the first common parent node of them is 1.1). The key point information is saved as a dictionary structure, in which the key is the key point coordinate and the value is the key point corresponding coding ({(X n ,Y n ,Z n ):‘1.1.1’}) Mastering all key point information before path planning will make the task of sampling multiple lesions easier without the need for repeated coding.

[0077] In calculating the path length, the Euclidean distance formula is used:

[0078]

[0079] where P iand P j The distance between two adjacent points is calculated in turn, and after each calculation, the adjacent point at this moment is taken as the starting point at the next moment, and the distance between it and the new adjacent point is calculated, until the adjacent point at a certain moment is the first common parent node, and the calculation is ended.

[0080] After summing up all the calculation results, the distance from a certain node to the first common parent node is obtained (when finding the adjacent points of a certain node, first determine whether the certain point exists in the dictionary, that is, whether it belongs to an endpoint or a branching point. If not, according to the setting of the coordinate origin, only need to add or subtract 1 in the Z direction each time, and calculate the 9-neighborhood in the new plane. When the starting node is an endpoint, first traverse its 26-neighborhood to find out whether the adjacent points are in the plane away from or close to the coordinate origin, and then add or subtract 1 in the Z direction each time until the first branching point is encountered. For a branching point, first check whether there is an adjacent point in its 8-neighborhood, if there is an adjacent point, the same check is performed, and if there is no adjacent point in the same plane, determine whether the adjacent points are in the plane away from or close to the coordinate origin, and add or subtract 1 in the Z direction each time until the next branching point is encountered).

[0081] Since the first common parent node of multiple nodes can be known, it is only necessary to calculate the distance from each node to the first common parent node from back to front, and it is not necessary to calculate the distance from the first common parent node to the starting point of the bronchus 1, because this is the path that must be passed through by the bronchoscope, greatly reducing the amount of calculation.

[0082] In step four:

[0083] According to the airway tree centerline and the lesion position, the path is planned. The center point coordinates (X pn ,Y pn ,Z pn ) of the pulmonary nodule are calculated, and the center of the cube is taken as the center, starting from 0 and increasing the edge length by 2δ d , the cube gradually expands outward, until the cube contains the key points, and these points are established as a set (the three coordinates of the center are ±δ d , respectively, because the key point information of the bronchus has been saved as a dictionary in the previous step, it can be detected whether there is a key point in the range of (X pn ±δ d ,Y pn ±δ d ,Z pn ±δ d ).

[0084] The airway tree mask binary image is dilated morphologically, i.e. the bronchus is expanded outward by one voxel width, and the dilated part can be used as the airway wall. Then the foreground and background voxel values are exchanged (before the exchange, the airway tree is the foreground and the voxel value is 1, and the rest is the background and the voxel value is 0. After the exchange, the airway tree voxel value becomes 0, and the rest of the voxel value becomes 1), and then distance transformation operation is performed, so that the Euclidean distance image of each zero voxel in the airway tree to the nearest non-zero airway wall can be obtained. On this basis, the distance image is multiplied by the airway tree centerline image, and the nearest distance from the airway tree centerline to the airway wall can be obtained. Simply doubling the nearest distance as the minimum diameter of the airway lumen corresponding to any point, the cross-sectional size of the lumen is calculated, and by comparing with the size of the bronchoscope, it can be determined whether the bronchoscope can pass through the point;

[0085] The cross-sectional area of the branch segment connected by each key point and the average value of the minimum diameter of the lumen at the parent node is calculated. If the area is less than the size of the bronchoscope, it is considered that the bronchoscope cannot pass through the branch, and the key points on the branch are removed from the set.

[0086] The first common parent node of the remaining key points in the set is calculated, and the node with the minimum distance from each key point to the first common parent node and the lesion center on the branch segment connected with the parent node is calculated. It can be an end point, an edge point or a bifurcation point (the distance from these key points to the nearest point to the first common parent node and the lesion center on the branch segment connected with the child nodes is not calculated, because the branch segment will increase the distance of the bronchoscope, the cross-sectional area of the airway lumen will be smaller than the upper level, the bronchoscope cannot pass through, and the calculation amount of the distance from the node to the two target points is reduced), and these nodes are marked as operation points.

[0087] The multiple operation points currently determined are sorted from low to high according to the distance, and the first 5 (the number can be adjusted as needed) operation points are retained, and less than 5 is treated as a vacancy. At this time, the number of operation points is less than or equal to 5.

[0088] After the above steps are completed, the 2δ d Continue to increase the size of the cube, and repeat the operation for the newly included key points: remove the key points that the bronchoscope cannot pass through in the cube, determine the first common parent node of the remaining key points, traverse to find the node with the minimum distance from each key point to the first common parent node and the lesion center on the branch segment connected with the parent node, retain 5 nodes, mark them as operation points, and compare the 5 new operation points with the 5 operation points generated in the last stage in terms of distance length, replace or retain the operation points in the last stage. If the size of the cube is increased for 3 (the number can be adjusted as needed) times in a row, and the 5 operation points do not change, stop expanding the cube outward.

[0089] In step five:

[0090] Since the pulmonary artery and vein have been segmented in the previous steps, the next step is to measure the blood vessel distribution of each surgical point by calculating whether there is a blood vessel in the puncture path from each surgical point to the lesion center, and then combining the length of the surgical point path to make the following decisions:

[0091] 1) If the shortest path of the five paths does not have a blood vessel in the puncture path from the surgical point to the lesion, the surgical point corresponding to the shortest path is selected as the optimal surgical point, and the corresponding three-dimensional coordinate is output; 2) If the shortest path of the five paths has a blood vessel in the puncture path from the surgical point to the lesion, a new point is calculated from the back to the front on the branch segment where the surgical point corresponding to the shortest path is located, and the new point has no blood vessel in the puncture path to the lesion. According to the relationship of the three sides of a triangle, the new point further shortens the walking path; if there is no point on this branch that has no blood vessel in the puncture path to the lesion, the surgical point corresponding to the second shortest path is examined. If the surgical point has no blood vessel in the puncture path to the lesion, the three-dimensional coordinate of the surgical point is directly output, otherwise the same search operation is performed on the branch segment of the second shortest path: a new point is calculated from the surgical point to the front, which has no blood vessel in the puncture path, and so on, until a surgical point that meets both the short walking path and the puncture path to the lesion without blood vessels is selected, and the three-dimensional coordinate is output.

[0092] After obtaining the three-dimensional coordinate of the optimal surgical point, all coordinate points of the target branch are output in order from the back to the front, forming a list (since all key point coordinate information has been recorded, each key point can be traced back to the tracheal entrance direction, and multi-process operation can improve calculation efficiency and shorten calculation time). Reverse the list to obtain the coordinate information of each node on the path from the tracheal entrance to the optimal surgical point, and realize the function of guiding the bronchoscope to the lesion for sampling.

[0093] It should be understood that those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes shall fall within the protection scope of the appended claims of the present application.

Claims

1. A path planning and target branch decision-making system for pulmonary lesion navigation, characterized in that, The system includes a data input unit, a data processing unit, and a data output unit; The data input unit is used to receive input CT images; The data processing unit is used to preprocess the input CT images, extract lung region CT images from the preprocessed CT images; segment the airway tree, arteries and veins, and lesions from the extracted lung region CT images; extract the centerline image of the airway tree segmentation mask, extract key points from the airway tree centerline image according to the key point extraction algorithm, and encode the key points; the key points are the endpoints and bifurcation points on the airway tree centerline; construct a cube with the center of the lesion as the centroid, and delete key points within the cube that the bronchoscope cannot pass through; The first common parent node of these key points is determined by identifying the common parts in the remaining key point codes; the node with the smallest sum of distances from the first common parent node to the lesion center is found by traversing the branch segments connected to each key point and its parent node, and a set number of nodes are retained and recorded as surgical points; it is determined whether the multiple surgical points currently determined remain unchanged after the cube is continuously expanded a set number of times, if so, the cube expansion is stopped, otherwise the cube expansion continues according to the set rules, and the operation is repeated for newly included key points: key points in the cube that cannot be passed by the bronchoscope are deleted, the first common parent node of the remaining key points is determined, the node with the smallest sum of distances from the first common parent node to the lesion center is found by traversing the branch segments connected to each key point and its parent node, and a set number of nodes are retained and recorded as surgical points; the optimal surgical point is selected from the multiple surgical points finally determined according to the length of the travel distance and the distribution of blood vessels along the way to the lesion puncture, and the target path from the tracheal inlet to the lesion is obtained based on the optimal surgical point; The data output unit is used to output the obtained target path.

2. The path planning and target branch decision-making system for lung lesion navigation according to claim 1, characterized in that, Extracting lung regions from preprocessed CT images using: The CT image was converted into a binary image using a thresholding method, where the lung region was set as the foreground and other tissue information was set as the background, resulting in multiple connected components. By constraining the volume and eccentricity of connected components, the two largest connected components are used as masks for the left and right lungs. Morphological operations are used to fill holes in the masks of the left and right lungs in the binary image to eliminate noise interference in the lung region mask; Convex hull calculation is used to merge isolated pixels at the boundaries of the left and right lung masks in the binary image to form a more complete lung region mask; The lung region mask in the preprocessed CT image and the binary image are multiplied pixel by pixel to obtain a CT image containing only the lung region, while tissue information outside the lung region is removed from the preprocessed CT image.

3. The path planning and target branch decision-making system for lung lesion navigation according to claim 1 or 2, characterized in that, The airway tree, arteries and veins, and lesions were segmented from the extracted lung region CT images, including: The smallest bounding box of the lung region CT image is cropped to the lung as the effective input region; The effective input region is cropped into multiple volume blocks using a sliding window, and these volume blocks are then input into the neural network for model training. After the model is trained, it can be directly applied in clinical practice to segment the airway tree structure, pulmonary arteriovenous structure, and pulmonary lesion structure in the preprocessed CT images.

4. The path planning and target branch decision-making system for lung lesion navigation according to claim 1, characterized in that, The method used to extract the airway tree centerline image is: The segmentation results are processed by extracting the airway tree mask from the segmentation results to obtain a binary image containing only the airway tree mask and the background. A three-dimensional thinning algorithm is then used to reduce the airway tree mask in the binary image to the width of a single pixel to obtain the airway tree centerline image.

5. The path planning and target branch decision-making system for lung lesion navigation according to claim 1, characterized in that, The key point extraction algorithm is used to extract key points from the airway tree centerline image, and the key points are encoded using the following method: Based on the number of voxels in the 26 neighborhoods of each node on the center line, the nodes on the center line are divided into endpoints, edge points, and bifurcation points. The number of adjacent voxels of an endpoint is 1, the number of adjacent voxels of an edge point is 2, and the number of adjacent voxels of a bifurcation point is not less than 3. Key points are encoded using a breadth-first search algorithm; Save the key point information as a dictionary structure, where the key is the key point coordinate and the value is the corresponding code of the key point.

6. The path planning and target branch decision-making system for lung lesion navigation according to claim 1, characterized in that, Construct a cube with the lesion center as its centroid. Remove key points within the cube that the bronchoscope cannot pass through. Determine the first common parent node of these key points by identifying the common parts in their codes. Iterate through each key point and find the node with the smallest sum of distances from its parent node to the first common parent node and the lesion center. Retain a predetermined number of these nodes, denoted as the surgical points. Calculate the coordinates of the center point of the lung nodule (X). pn ,Y pn Z pn Using the center point of the lung nodule as the centroid of the cube, starting from 0, the side length is gradually increased by a preset length, and the cube gradually expands outward until the key points are contained inside the cube. The key points inside the cube are then established as a set. The average value of the minimum diameter of the airway lumen at the key point and its parent node is used to calculate the cross-sectional area of ​​the connected branch segment. If the area is smaller than the size of the bronchoscope, it is considered that the bronchoscope cannot pass through the branch, and the key point is removed from the set. Determine the first common parent node of these key points by identifying the common parts in the remaining key point encodings, and traverse to find the node with the smallest sum of distances from the first common parent node and the lesion center on the branch segments connected to these key points and their parent nodes. Sort the nodes from shortest to longest path, retain the first set number of nodes and record them as surgical points; otherwise, treat them as empty slots.

7. The path planning and target branch decision-making system for lung lesion navigation according to claim 6, characterized in that, The cross-sectional area of ​​the connected branch segments is calculated by taking the average of the minimum diameter of the airway lumen at the key point within the cube and its parent node: Perform dilation morphological processing on the binary image of the airway tree mask; Swap the voxel values ​​of the foreground and background; Perform a distance transformation operation to obtain the Euclidean distance image from each zero-value voxel in the airway tree to the nearest non-zero airway wall; Multiply the distance image with the airway tree centerline image to obtain the shortest distance from the airway tree centerline to the airway wall; The minimum diameter of the airway lumen corresponding to any point is taken as twice the nearest distance. The cross-sectional area of ​​the lumen is calculated, and the average cross-sectional area of ​​the lumen at the key point and its parent node is taken as the cross-sectional area of ​​the connected branch segment.

8. The path planning and target branch decision-making system for lung lesion navigation according to claim 6 or 7, characterized in that, Determine whether the currently identified surgical points remain unchanged after the cube is continuously expanded a set number of times. If so, stop the cube expansion; otherwise, continue expanding the cube according to the set rules. Repeat the previous step for each newly included key point. Continue increasing the size of the cube according to the preset length, and repeat the operation for newly included key points: delete key points in the cube that the bronchoscope cannot pass through, determine the first common parent node of the remaining key points, traverse and find the node with the smallest sum of distances from the first common parent node to the lesion center on the branch segment connected to each key point and its parent node, retain a set number of nodes, and record them as surgical points. Compare the first set number of new surgical points with the first set number of surgical points generated in the previous stage in terms of path length, and replace or retain the surgical points of the previous stage; if the size of the cube increases for a second set number of consecutive times and the currently determined first set number of surgical points no longer changes, then stop the cube from expanding outward.

9. The path planning and target branch decision-making system for pulmonary lesion navigation according to any one of claims 1, 2, 4-7, characterized in that, The optimal surgical point is selected from the multiple determined surgical points based on the distance and the distribution of blood vessels along the puncture route to the lesion. Based on the optimal surgical point, the target path from the tracheal inlet to the lesion is output using the following method: Calculate whether there are pulmonary arteries and veins along the puncture path from each surgical point to the lesion, in order to measure the distribution of blood vessels from the surgical point to the lesion. Based on the length of the puncture site and the distribution of blood vessels along the route, the following decision should be made: 1) If there are no blood vessels distributed along the puncture route from the surgical point corresponding to the shortest path in the first set number of paths to the lesion, then the point corresponding to the shortest path is taken as the optimal surgical point, and the corresponding three-dimensional spatial coordinates are output. 2) If there are blood vessels along the puncture route from the surgical point to the lesion on the shortest path in the first set number of paths, then on the branch segment where the surgical point is located, calculate from back to front a new point where there are no blood vessels along the puncture route to the lesion. According to the triangle relationship, the travel distance of the new point is further shortened. If there are no points on the branch where there are no blood vessels along the puncture route to the lesion, then examine the surgical point corresponding to the second shortest path. If there are no blood vessels along the puncture route from the surgical point to the lesion, then directly output the three-dimensional spatial coordinates of the surgical point. Otherwise, perform the same search operation on the branch segment of the second shortest path: calculate new points where there are no blood vessels along the puncture route from the surgical point to the lesion, and so on, until a surgical point that satisfies both short travel distance and no blood vessels along the puncture route to the lesion is selected, and output the three-dimensional spatial coordinates. After obtaining the three-dimensional spatial coordinates of the optimal surgical point, output all the coordinate points of the target branch from back to front to form a list; reverse the list to obtain the coordinate information of each node on the path from the tracheal inlet to the optimal surgical point.

Images