A method, apparatus, device, and medium for segmenting lower respiratory tract images

Abstract

The present invention provides a method, apparatus, device and medium for segmenting lower respiratory tract images. The method includes: S1, extracting trachea and main bronchial branch images from the original lung images; S2, segmenting the lung vascular images to generate a lung vascular segmentation result; according to the lung vascular segmentation result, extracting the main trunk image and branch images of the blood vessels in the lungs; S3, traversing each blood vessel branch, performing segmentation within the range where the blood vessel branch is located to generate small tracheal branch images; S4, splicing all the small tracheal branch images and the tracheal main trunk image to generate a complete lower respiratory tract tracheal image; S5, performing bronchial leakage detection and elimination on the lower respiratory tract tracheal image to generate an accurate bronchial image. This method is used to improve the detection rate of small bronchial branches.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly to a method, device, equipment and medium for segmenting lower respiratory tract images. Background Art

[0002] Lung biopsy based on endoscopy is an important means for lung cancer screening. Before endoscopy, doctors first need to perform three-dimensional computed tomography (CT) scans on patients to obtain the morphological positions of tissues such as the lungs, lower respiratory tract, lung nodules, and pulmonary blood vessels, and then plan the optimal navigation path based on the above morphological positions. Currently, the bronchial order visible on CT with higher resolution is about 1 mm, and the passing diameter of an ultra-thin bronchoscope in the airway is about 2 mm. Planning the optimal path along the lung bronchial branches and performing instrument navigation can effectively improve the biopsy success rate while reducing surgical trauma.

[0003] When taking CT, due to CT artifacts, human movement, or volume effects, bronchial degradation will occur on the CT image. The wall of smaller bronchi is relatively thin in the CT image and visually connected to the lungs, resulting in bronchial leakage, breakage, or even disappearance, which easily leads to a decrease in diagnostic accuracy and thus affects the bronchus-based surgical planning and surgical success rate. Therefore, there is an urgent need for a new method, device, equipment and medium for segmenting lower respiratory tract images to improve the above problems. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, device, equipment and medium for segmenting lower respiratory tract images, and this method is used to improve the detection rate of small bronchial branches.

[0005] In a first aspect, the present invention provides a method for segmenting lower respiratory tract images, including: S1, extracting trachea and main bronchial branch images from the original lung image; S2, segmenting the pulmonary blood vessel image to generate a pulmonary blood vessel segmentation result; according to the pulmonary blood vessel segmentation result, extracting the main blood vessel image and blood vessel branches of the lungs; S3, traversing each blood vessel branch and performing segmentation within the range where the blood vessel branch is located to generate small tracheal branch images; S4, splicing all small tracheal branch images and the tracheal main image to generate a complete lower respiratory tract trachea image; S5, performing bronchial leakage detection and removal on the lower respiratory tract trachea image to generate an accurate bronchial image.

[0006] The beneficial effects of the method of the present invention are as follows: By traversing each blood vessel branch and performing segmentation within the range where the blood vessel branch is located, small tracheal branch images are generated; all small tracheal branch images and the tracheal main trunk image are stitched together to generate a complete lower respiratory tract trachea image, which can improve the detection rate of small bronchial branches, reduce bronchial leakage and false detection rates, and is beneficial to improving the diagnostic accuracy. By obtaining the main bronchial branch image; performing bronchial leakage detection and elimination on the lower respiratory tract trachea image to generate an accurate bronchial image, it can provide more accurate guidance for lung surgery planning and navigation, reduce surgical trauma, and improve the surgical success rate.

[0007] Optionally, extracting the trachea and main bronchial branch images from the original lung image includes: S11, obtaining a lung CT image, performing a derivative operation to obtain the second derivative of the image; constructing a new diffusion coefficient based on the second derivative of the image; updating the anisotropic diffusion formula according to the new diffusion coefficient to obtain a smoothed image; S12, determining bronchial seed points according to the smoothed image, updating the pixel difference ratio each time an iteration is performed, and performing iterative loops to obtain the main bronchial branch image.

[0008] Optionally, determining bronchial seed points according to the smoothed image, updating the pixel difference ratio each time an iteration is performed, and performing iterative loops to obtain the main bronchial branch image includes: S111, taking the average of the pixel values of the previous segmentation result to obtain an average pixel value, and stopping the iteration when the average pixel value is greater than the maximum average pixel value; S112, finding the set of edge pixel points of the previous segmentation result, traversing each target point in the 26-neighborhood of each pixel in the set of edge pixel points, taking the difference between the pixel value of the current target point and the average pixel value of the previous iteration, and setting the current target point as a valid point if the difference is less than the pixel difference ratio diffT; S113, traversing the 6-neighborhood of each valid point, taking the difference between the pixel value of each point in the 6-neighborhood and the average pixel value, and setting the current valid point as a point on the bronchus if all are less than the pixel difference ratio diffT; S114, calculating the number of bronchial pixels segmented in the previous iteration and the number of bronchial pixels segmented in this iteration, taking the difference between the two and dividing by the number of bronchial pixels segmented in the previous iteration to obtain a count difference ratio, and stopping the iteration when the count difference ratio is greater than the maximum iteration difference ratio, and using the previous segmentation result as the final result of bronchial segmentation; S115, updating the pixel difference ratio based on the current pixel difference ratio plus the pixel difference ratio step size, and stopping the iteration when it is determined that the updated pixel difference ratio is greater than the maximum pixel difference ratio diffTmax, and using the previous segmentation result as the final result of bronchial segmentation.

[0009] Optionally, determining bronchial seed points based on the smoothed image includes: starting from the head direction of the smoothed image, selecting a two-dimensional image from the tracheal inlet to the area outside the lungs as the first image; using Otsu's threshold method to separate the lung tissue in the first image to obtain a second image; after adding the edge region based on the second image, performing flood filling, and inverting the filled image to obtain a third image; removing the edge region of the third image to obtain a fourth image; obtaining the pixel coordinates of the pixels in the fourth image whose values are equal to the intermediate trachea as the coordinates of the seed points in the smoothed image.

[0010] Optionally, segmenting the pulmonary vascular image to generate a pulmonary vascular segmentation result includes: S21, selecting a two-dimensional image at the central position in the z direction of the CT image to calculate pulmonary seed points, performing bilateral lung region growing to obtain a bilateral lung segmentation result with bronchi; S22, when the main bronchial branch image is in the lung segmentation result, subtracting the main bronchial branch image from the bilateral lung segmentation result with bronchi to obtain a lung segmentation result; S23, based on the lung segmentation result, performing a closing operation on the lungs, filling the bilateral lung segmentation image regions with holes into bilateral lung solid regions, and subtracting the lung segmentation result from the solid lung image region to obtain a pulmonary vascular segmentation result.

[0011] Optionally, segmenting within the range where the vascular branches are located to generate a small tracheal branch image includes: S31, traversing each branch of the blood vessel, connecting the starting point and the ending point of the currently traversed branch as a line segment as the main axis of the cylinder, making a cylinder with a radius of 12 pixels, and removing the part outside the cylinder on the CT image. Traversing each two-dimensional image I in the cross-section, sagittal plane, and coronal plane of the cylinder, locating the possible two-dimensional tracheal cross-section SecC through binarization and region growing, and counting the number of pixels NsecC of each two-dimensional tracheal cross-section; S32, removing the two-dimensional tracheal cross-sections with a similarity to the bronchus lower than the similarity threshold to retain the possible two-dimensional tracheal cross-sections that may be bronchi with a probability greater than the probability threshold; S33, setting a threshold for the small tracheal branches, and obtaining the edge EdgeSecC of each possible two-dimensional tracheal cross-section, where the edge points are the points among the four-connected points of all points on the two-dimensional tracheal cross-section that do not belong to the points on the two-dimensional tracheal cross-section; S34, scoring each possible two-dimensional tracheal cross-section, and the scoring score Score(SecC) satisfies:

[0012]

[0013] where x and y are the point coordinates in the two-dimensional image, and z is the layer number of the two-dimensional image; the scoring score is greater than 0, and the higher the scoring score, the greater the probability that it is a bronchial cross-section branch; S35, retaining the M high-score cross-sections SecC2 with the highest scores, and fitting an ellipse to each high-score cross-section SecC2 to locate the tracheal cross-section in the cylinder.

[0014] Optionally, it further includes: S36, projecting the fitted ellipses of the three high-score cross-sections onto a vertical plane, and determining that it is required that at least part of these three projections overlap, and the distance from the midpoints of the three ellipses to the regression line is lower than the first distance threshold, and determining whether the high-score cross-section is a measure of a bronchial cross-section, satisfying:

[0015] measure1(SecC) = distance(C i-1 ,C i ,C i+1 )

[0016] measure2(SecC)

[0017] = midValue{Score(SecC i-1 ), Score(SecC i ), Score(SecC i+1 )}

[0018] *overlap(SecC i-1 , SecC i , SecC i+1 )

[0019] wherein, distance is the sum of the distances from the centers of the fitted ellipses of the three high-score cross-sections to the regression line. midValue is the median, Score(SecC i ) is the scoring score for the i-th two-dimensional tracheal cross-section, SecC i is the i-th two-dimensional tracheal cross-section, i is a positive integer, overlap is the intersection-over-union ratio of the areas of the projections of the fitted ellipses of the three high-score cross-sections, measure1 is the preliminary measurement value when initially selecting the upper and lower cross-sections, and measure2 is the subsequent measurement value when subsequently selecting the upper and lower cross-sections; according to the measurement values measure1 and measure2, the cross-sections that do not belong to the trachea are removed from the high-score cross-section SecC2i to form the target cross-section Sec, and the target cross-section Sec contains all the ellipses projected onto the corresponding vertical planes; the target cross-sections are connected to form the small tracheal branch image.

[0020] Optionally, the connecting of the target cross-sections to form the bronchiole branch image includes: S37, connecting L adjacent target cross-sections to form a segmentation block; classifying the unconnected cross-sections, searching for the distance from the unconnected cross-section to the elliptical center of the connected cross-section and the connected cross-section with the smallest included angle between the regression line of the unconnected cross-section and its adjacent cross-section and the regression line of the connected cross-section as the classification target; incorporating the currently unconnected cross-section into the segmentation block to which the classification target belongs; S38, pairwise determining whether the segmentation blocks meet the mergeable conditions, where the mergeable conditions include that the central distance of the elliptical cross-sections at both ends that can be connected, the included angle between the regression line and the connecting line of the midpoints of the ellipses, and the included angle between the regression lines are the smallest and less than a set threshold, and then completing the trachea at both ends that can be connected; S39, retaining the segmentation block with the smallest Hausdorff distance to the points on the blood vessel branch and less than the distance threshold as the bronchial branch corresponding to the current blood vessel branch.

[0021] Optionally, stitching all the bronchiole branch images and the tracheal trunk image to generate a complete lower respiratory tract trachea image includes: S41, performing region growing on the tracheal trunk after the image stitching to merge some bronchiole branches with the main bronchus branches; S42, first connecting the remaining unmerged tracheal branches between the branches and then connecting them to the tracheal trunk; the conditions for connecting between the branches include that the distance between the midpoints of the two ellipses at both ends of the connection is less than a second distance value and the connection direction of the two endpoints and the included angle between the regression lines of the two ellipses at both ends are less than an included angle threshold, and completing the trachea between the two ellipses.

[0022] Optionally, performing bronchial leakage detection and removal on the lower respiratory tract trachea image includes: S51, refining the bronchial segmentation result to obtain the bronchial trunk, traversing the points on the trunk starting from the main airway entrance to form a bronchial tree; the bronchial tree includes all bronchial branches, and each branch includes two branch nodes and a branch path; when there is more than one branch path between two nodes, it is confirmed that there is a leakage in the segmentation; S52, calculating the center point of the leaking trunk and fitting an ellipse to obtain the elliptical center point, the major and minor axes, performing dilation operation based on an elliptical spherical kernel on the elliptical center to obtain the leakage area, and removing the leakage area from the lower respiratory tract trachea image to obtain an accurate bronchial image.

[0023] Second aspect, the present invention provides a segmentation device for lower respiratory tract images, used for the method described in any one of the first aspects, including: a segmentation unit for segmenting the pulmonary vascular image to generate a pulmonary vascular segmentation result; an extraction unit for extracting the trachea and main bronchial branch images from the original pulmonary image; according to the pulmonary vascular segmentation result, extracting the main pulmonary vascular image and vascular branches; traversing each vascular branch, performing segmentation within the range where the vascular branch is located to generate small tracheal branch images; a synthesis unit for stitching all small tracheal branch images and the main tracheal image to generate a complete lower respiratory tract tracheal image; a correction unit for performing bronchial leakage detection and elimination on the lower respiratory tract tracheal image to generate an accurate bronchial image.

[0024] Optionally, the correction unit is further configured to obtain a pulmonary CT image, perform a derivative operation to obtain the second derivative of the image; construct a new diffusion coefficient according to the second derivative of the image; update the anisotropic diffusion formula according to the new diffusion coefficient to obtain a smoothed image.

[0025] Third aspect, the present invention provides an electronic device, including a memory and a processor, where a program that can run on the processor is stored on the memory. When the program is executed by the processor, the electronic device implements the method described in any one of the first aspects.

[0026] Fourth aspect, the present invention provides a readable storage medium, where a program is stored in the readable storage medium. When the program is executed, the method described in any one of the first aspects is implemented. Description of the Drawings

[0027] Figure 1 It is a schematic flowchart of a segmentation method for bronchial images provided by the present invention;

[0028] Figure 2 It is the segmentation result of the trachea and main bronchial branches in a certain test example of the present invention;

[0029] Figure 3 It is the vascular segmentation result in a certain test example of the present invention;

[0030] Figure 4 It is the main pulmonary vessel extracted in a certain test example of the present invention;

[0031] Figure 5 It is a partial bronchial branch and vascular accompanying diagram in a certain test example of the present invention;

[0032] Figure 6 It is a partial bronchial leakage and leakage elimination diagram in a certain test example of the present invention;

[0033] Figure 7 It is the lower respiratory tract segmentation result in a certain test example of the present invention;

[0034] Figure 8 Schematic structural diagram of a segmentation device for bronchial images provided by the present invention;

[0035] Figure 9 Schematic structural diagram of an electronic device provided by the present invention.

[0036] Reference numerals in the figure:

[0037] 101, segmentation unit; 102, extraction unit; 103, synthesis unit; 104, correction unit;

[0038] 201, processor; 202, memory. Specific embodiments

[0039] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts shall fall within the protection scope of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein shall have the ordinary meanings understood by those of ordinary skill in the art in the field to which the present invention belongs. The words such as "including" used herein are intended to mean that the elements or items appearing before this word cover the elements or items listed after this word and their equivalents, without excluding other elements or items.

[0040] Regarding the problems existing in the prior art, as Figure 1 shown, the first embodiment provides a method for segmenting lower respiratory tract images, including: S1, extracting trachea and main bronchial branch images from the original lung images; S2, segmenting the lung vascular images to generate lung vascular segmentation results; according to the lung vascular segmentation results, extracting the main trunk images and branch images of the blood vessels in the lungs; S3, traversing each blood vessel branch and performing segmentation within the range where the blood vessel branch is located to generate small bronchial branch images; S4, splicing all the small bronchial branch images and the trachea main trunk image to generate a complete lower respiratory tract trachea image; S5, performing bronchial leakage detection and elimination on the lower respiratory tract trachea image to generate accurate bronchial images.

[0041] In this embodiment, by obtaining the main bronchial branch image, traversing each vascular branch, and performing segmentation within the range where the vascular branch is located to generate the small tracheal branch image, and then stitching all the small tracheal branch images and the tracheal trunk image together to generate the complete lower respiratory tract trachea image, the detection rate of small bronchial branches can be improved, the bronchial leakage and false detection rate can be reduced, which is beneficial to improving the diagnostic accuracy. Detecting and removing bronchial leakage from the lower respiratory tract trachea image to generate an accurate bronchial image can provide more precise guidance for lung surgery planning and navigation, reduce surgical trauma, and improve the surgical success rate.

[0042] In some embodiments, extracting the trachea and main bronchial branch image from the original lung image includes: S11, obtaining a lung CT image, performing a derivative operation to obtain the second derivative of the image, constructing a new diffusion coefficient according to the second derivative of the image, and updating the anisotropic diffusion formula according to the new diffusion coefficient to obtain a smoothed image; S12, determining bronchial seed points according to the smoothed image, updating the pixel difference ratio in each iteration, and iterating cyclically to obtain the main bronchial branch image.

[0043] Specifically, for thin-slice CT images with relatively high resolution, such as the unit recognition size being less than 0.6x0.6x0.6mm, the CT image includes the complete double lungs, that is, starting from the collarbone of the subject to the middle and upper abdominal cavity. Denoising the CT image before extracting the main bronchial branch can ensure that the edge information is not lost as much as possible and filter out the redundant noise. In this embodiment, a denoising method that preserves the edge, the curvature anisotropic diffusion method, is used to generate a smoothed image. In this embodiment, the curvature or the second derivative of the image is introduced into the anisotropic diffusion formula to construct a new diffusion coefficient, removing noise while retaining as much image detail as possible, and then performing Laplacian sharpening on the smoothed image to enhance the image edge information.

[0044] In some embodiments, bronchial seed points are determined based on the smoothed image, and the pixel difference ratio is updated iteratively each time. The loop iteration is performed to obtain the bronchial main branch image, including: S111, taking the average of the pixel values of the last segmentation result to obtain the average pixel value, and stopping the iteration when the average pixel value is greater than the maximum average pixel value; S112, finding the set of edge pixel points of the last segmentation result, traversing each target point in the 26-neighborhood of each pixel in the set of edge pixel points, taking the difference between the pixel value of the current target point and the average pixel value of the last iteration. If the difference is less than the pixel difference ratio diffT, set the current target point as a valid point; S113, traversing the 6-neighborhood of each valid point, taking the difference between the pixel value of each point in the 6-neighborhood and the average pixel value. If all are less than the pixel difference ratio diffT, set the current valid point as a point on the bronchus; S114, calculating the number of bronchial pixels segmented in the last iteration and the number of bronchial pixels segmented in this time, taking the difference between the two and dividing by the number of bronchial pixels segmented in the last time to obtain the count difference ratio. When the count difference ratio is greater than the maximum iteration difference ratio, stop the iteration and use the last segmentation result as the final result of bronchial segmentation; S115, updating the pixel difference ratio based on the current pixel difference ratio plus the pixel difference ratio step size. When it is determined that the updated pixel difference ratio is greater than the maximum pixel difference ratio diffTmax, stop the iteration and use the last segmentation result as the final result of bronchial segmentation.

[0045] Exemplarily, the pixel difference ratio diffT is 0.05, the maximum pixel difference ratio diffTmax is 0.11, the maximum average pixel value meanmax is -850 Hounsfield Unit (HU), and the maximum iteration difference ratio diffI is 0.2. In a certain test case, the segmentation results of the trachea and bronchial main branches are as Figure 2 shown.

[0046] It should be noted that in S114, calculating the number of bronchial pixels segmented in the last iteration and the number of bronchial pixels segmented in this time, taking the difference between the two and dividing by the number of bronchial pixels segmented in the last time to obtain the count difference ratio includes: obtaining the number of bronchial pixels P1 segmented in the last iteration and the number of bronchial pixels P2 segmented in this time. Calculating the difference in the number of pixels between the two segmentations: the difference obtained in the first subtraction is P1 - P2, and the difference obtained in the second subtraction is P2 - P1. Taking the absolute value of the two differences: that is, |P1 - P2| and |P2 - P1|. Dividing by the number of bronchial pixels segmented in the last time: taking the larger value of the above two absolute values and then dividing by P1. After dividing the larger difference by P1, the ratio of the change in the bronchial structure between the two segmentations, that is, the count difference ratio, is obtained.

[0047] In some embodiments, determining bronchial seed points based on the smoothed image includes: starting from the head direction of the smoothed image, selecting a two-dimensional image from the tracheal inlet to the area outside the lungs as the first image; using Otsu's threshold method to separate the lung tissue in the first image to obtain a second image; after adding an edge area based on the second image, performing flood filling, and inverting the filled image to obtain a third image; removing the edge area of the third image to obtain a fourth image; and obtaining the pixel coordinates in the fourth image that are equal to the intermediate trachea as the coordinates of the seed points in the smoothed image.

[0048] Exemplarily, select the fifth CT image from the head direction downwards of the CT image as the initial two-dimensional image. In some other embodiments, estimate the CT area from the tracheal inlet to the area outside the lungs and select an image as the initial two-dimensional image. Use Otsu's threshold method to separate the lungs from the bone and fat tissues in the first image to obtain a second image. Based on the second image, through a padding operation, add an edge area of 3 pixels. Use the upper left corner point as the seed point for flood filling, with the filled pixel value being 255. Invert the filled image, and remove the previously added edge area from the inverted image to obtain a fourth image with the pixel value of the intermediate trachea being 255 and the background pixel value being 0 as the initial segmentation result as Figure 3 shown. Obtain the coordinates of the points with pixel value 255 in the fourth image in the CT image as the coordinates of the seed points.

[0049] In some embodiments, segmenting the pulmonary vascular image to generate a pulmonary vascular segmentation result includes: S21, selecting a two-dimensional image at the central position in the z direction of the CT image to calculate pulmonary seed points, performing dual-lung region growing to obtain a dual-lung segmentation result with bronchi; S22, when the main bronchial branch image is in the lung segmentation result, subtracting the main bronchial branch image from the dual-lung segmentation result with bronchi to obtain a lung segmentation result; S23, based on the lung segmentation result, performing a closing operation on the lungs to fill the dual-lung segmentation image region with holes into a dual-lung solid region, and subtracting the lung segmentation result from the solid lung image region to obtain a pulmonary vascular segmentation result.

[0050] Exemplarily, when calculating the pulmonary seed points, perform dual-lung region growing with a threshold of -800 to obtain a dual-lung segmentation result. Since there may be lung leakage of the bronchi on the CT image, the segmented dual lungs may contain bronchi. In some other examples, an adaptive region growing can be performed from the aortic inlet on the CT image from top to bottom to obtain a pulmonary artery vascular segmentation result.

[0051] In some other examples, extracting the main pulmonary vascular image includes: refining the pulmonary vascular segmentation result to extract the main pulmonary vascular trunk, as Figure 4As shown. Find all nodes and branches in the main pulmonary blood vessels. The node is the 26-neighborhood sum of the node greater than or equal to 3, or the 26-neighborhood sum of the node equal to 1. The above branch is a point on the main blood vessel connecting two nodes.

[0052] In some other examples, the arterial blood vessel runs parallel to the bronchus as Figure 5 shown. The heart tissue is connected to the arterial blood vessel, the arterial blood vessel runs parallel to the bronchus and both are connected to the lung tissue. Traverse each blood vessel branch and perform segmentation within the range where the blood vessel branch is located to generate a small bronchial branch image, including: connecting the starting point and the ending point of the currently traversed branch as a line segment as the main axis of the cylinder, generating a cylinder with a radius of 12 pixels, and removing the part outside the cylinder on the CT image. Perform small bronchus segmentation on the remaining part corresponding to the cylinder on the CT image, including: positioning the tracheal cross-section in the cylinder; connecting the cross-sections to segment the small bronchial branches.

[0053] Specifically, the segmentation within the range where the blood vessel branch is located to generate a small bronchial branch image includes: S31, traverse each two-dimensional image I of the cross-section, sagittal plane, and coronal plane in the cylinder, locate the possible two-dimensional tracheal cross-section SecC through binarization and region growing, and count the number of pixels NsecC of each two-dimensional tracheal cross-section; S32, remove the two-dimensional tracheal cross-sections with a similarity lower than the similarity threshold to the bronchus to retain the possible two-dimensional tracheal cross-sections with a probability greater than the probability threshold that may be the bronchus; S33, set the threshold of the small bronchial branch, and obtain the edge EdgeSecC of each possible two-dimensional tracheal cross-section, where the edge point is the point among the four-connected points of all points on the two-dimensional tracheal cross-section that does not belong to the points on the two-dimensional tracheal cross-section; S34, score each possible two-dimensional tracheal cross-section, and the scoring score Score(SecC) satisfies:

[0054]

[0055] where x and y are the point coordinates in the two-dimensional image, and z is the layer number of the two-dimensional image; the scoring score is greater than 0 and the higher the scoring score, the greater the probability that it is a bronchial cross-section branch; S35, retain the M high-score cross-sections SecC2 with the highest scores, and fit an ellipse to each high-score cross-section SecC2 to locate the tracheal cross-section in the cylinder.

[0056] In a more specific embodiment, it further includes: S36, project the fitted ellipses of the three high-score cross-sections onto the vertical plane, and judge that it is required that at least part of these three projections overlap, and the distance from the midpoint of the three ellipses to the regression line is lower than the first distance threshold. The measure for judging whether the high-score cross-section is a bronchial cross-section satisfies:

[0057] measure1(SecC)=distance(C i-1 ,Ci , C i+1 )

[0058] measure2(SecC)

[0059] = midValue{Score(SecC i-1 ), Score(SecC i ), Score(SecC i+1 )}

[0060] * overlap(SecC i-1 , SecC i , SecC i+1 )

[0061] where distance is the sum of the distances from the centers of the fitted ellipses of the three high - score cross - sections to the regression line. The regression line is generated by the least - squares line fitting method. midValue is the median, Score(SecC i ) is the scoring score for the i - th two - dimensional tracheal cross - section, SecC i is the i - th two - dimensional tracheal cross - section, i is a positive integer, overlap is the intersection - over - union of the areas of the projections of the fitted ellipses of the three high - score cross - sections, measure1 is the preliminary measurement value during the preliminary selection of the upper and lower cross - sections, measure2 is the subsequent measurement value during the subsequent selection of the upper and lower cross - sections; according to the measurement values measure1 and measure2, the cross - sections that do not belong to the trachea are removed from the high - score cross - section SecC2i to form the target cross - section Sec, and the target cross - section Sec contains all the ellipses projected onto the corresponding vertical planes; the target cross - sections are connected to form the small tracheal branch image.

[0062] In some other embodiments, to improve the calculation efficiency, first, three upper and lower cross-sections with the smallest distance from the center of the ellipse of the current cross-section are found according to the cross-sections of the upper and lower CT layers where the current cross-section is located. If there are no paired upper and lower cross-sections, the current cross-section is removed. Otherwise, continue with the measure2 metric calculation to score the current cross-section as a bronchial cross-section. For the current cross-section, select the cross-section score that maximizes measure2, and retain the ellipse projected from this cross-section onto the corresponding vertical plane, the upper and lower cross-sections of this cross-section, and the ellipses projected from the upper and lower cross-sections onto the corresponding vertical planes. The regression line can be generated by at least one of Least Squares (LS), Least Absolute Deviations (LAD), Bessel's Correction (BC), Weighted Least Squares (WLS), Ridge Regression (RR), Principal Component Regression (PCR), and Partial Least Squares (PLS).

[0063] In some embodiments, the connecting the target cross-sections to form the small tracheal branch image includes: S37, connecting L adjacent target cross-sections to form a segmentation block; classifying the unconnected cross-sections, searching for the distance from the unconnected cross-section to the center of the ellipse of the connected cross-section, and the connected cross-section with the smallest angle between the regression lines of the unconnected cross-section and its adjacent cross-section and the regression line of the connected cross-section as the classification target; incorporating the current unconnected cross-section into the segmentation block to which the classification target belongs; S38, pairwise determining whether the segmentation blocks meet the mergeable conditions, where the mergeable conditions include that the central distance between the ellipse cross-sections at both ends that can be connected, the angle between the regression line and the connecting line of the midpoint of the ellipse, and the angle between the regression lines are the smallest and less than a set threshold, and then completing the trachea at both ends that can be connected; S39, retaining the segmentation block with the smallest Hausdorff distance from the points on the blood vessel branch and less than the distance threshold in the segmentation block as the bronchial branch corresponding to the current blood vessel branch.

[0064] Specifically, connect each target cross-section, the 2 cross-sections above the target cross-section, and the 2 cross-sections below the target cross-section to form a segmentation block containing 5 cross-sections. Search for the distance from each unconnected cross-section to the center of the ellipse of the connected cross-section, and the connected cross-section with the smallest angle between the regression lines of the unconnected cross-section and its adjacent cross-section and the regression line of the connected cross-section, and use the segmentation block to which this cross-section belongs as the segmentation block where the current cross-section is located.

[0065] In some embodiments, all small bronchial branch images and the tracheal main stem image are stitched together to generate a complete lower respiratory tract tracheal image, including: S41, after the images are stitched together, region growing is performed on the tracheal main stem, and some small bronchial branches are merged with the main bronchial branches; S42, the remaining unmerged bronchial branches are first connected between branches, and then connected to the tracheal main stem; the conditions for connecting between branches include that the distance between the midpoints of the ellipses at both ends is less than a second distance value and the included angle between the connection direction of the two endpoints and the regression line of the ellipses at both ends is less than an included angle threshold, and the trachea between the two ellipses is complemented.

[0066] In other embodiments, for performing the connection of the main stem, the branch endpoints with a distance less than a certain value from the main stem are found, and the similarity between the branch direction and the direction of the nearest point on the main stem is judged. If the similarity is greater than the similarity threshold, then the branch and the main stem are connected. It is also possible to set the position and scale to complement the bronchial main stem according to a cylinder. After completing the connection of the bronchial main stem, the largest connected component processing is performed to eliminate the branches not in the lower respiratory tract.

[0067] In some embodiments, bronchial leakage detection and elimination are performed on the lower respiratory tract tracheal image, including: S51, the bronchial segmentation result is refined to obtain the bronchial main stem, and the points on the main stem are traversed starting from the main airway entrance to form a bronchial tree; the bronchial tree includes all bronchial branches, and each branch includes two branch nodes and a branch path; when there is more than one branch path between two nodes, it is confirmed that there is a leakage in the segmentation; S52, the center point of the leaking main stem is calculated and an ellipse is fitted to obtain the ellipse center point and the major and minor axes. An inflation operation based on an elliptical spherical kernel is performed on the ellipse center to obtain a leakage area as shown on the left side; the leakage area is eliminated from the lower respiratory tract tracheal image to obtain an accurate bronchial image as shown on the right side. After eliminating the leakage area, the entire lower respiratory tract segmentation result is as shown. Figure 6 as shown on the left side; the leakage area is eliminated from the lower respiratory tract tracheal image to obtain an accurate bronchial image as shown on the right side. Figure 6 After eliminating the leakage area, the entire lower respiratory tract segmentation result is as shown. Figure 7 as shown.

[0068] In other embodiments, if there is a leakage in the segmentation, then there are ring-shaped or grid-shaped points on the main stem, and there is more than one branch path between two nodes. In this embodiment, it is default that the number of nodes of a ring-shaped or grid-shaped point composed of one leakage does not exceed 4, which is beneficial to reducing the calculation amount and improving the leakage detection efficiency. It should be noted that the default number of nodes of a ring-shaped or grid-shaped point composed of one leakage does not exceed X, where X is any positive integer.

[0069] as Figure 8As shown in the figure, the second embodiment provides a device for segmenting lower respiratory tract images, which is used for the method described in any one of the first aspects, and includes: a segmentation unit 101, configured to segment a pulmonary vascular image to generate a pulmonary vascular segmentation result; an extraction unit 102, configured to extract trachea and main bronchial branch images from an original pulmonary image; according to the pulmonary vascular segmentation result, extract a main pulmonary vascular image and vascular branches; traverse each vascular branch, and perform segmentation within the range where the vascular branch is located to generate small tracheal branch images; a synthesis unit 103, configured to splice all small tracheal branch images and the main tracheal image to generate a complete lower respiratory tract tracheal image; a correction unit 104, configured to perform bronchial leakage detection and removal on the lower respiratory tract tracheal image to generate an accurate bronchial image.

[0070] Specifically, the segmentation unit 101, the extraction unit 102, the synthesis unit 103, and the correction unit 104 may be provided on the same processor. In some other specific embodiments, the segmentation unit 101, the extraction unit 102, the synthesis unit 103, and the correction unit 104 may be provided as independent chips respectively.

[0071] In some embodiments, the correction unit 104 is further configured to obtain a pulmonary CT image, perform a derivative operation to obtain the second derivative of the image; construct a new diffusion coefficient according to the second derivative of the image; update the anisotropic diffusion formula according to the new diffusion coefficient to obtain a smoothed image.

[0072] As Figure 9 shown in the figure, the third embodiment provides an electronic device, including a memory 202 and a processor 201. A program that can run on the processor 201 is stored on the memory 202. When the program is executed by the processor 201, the electronic device implements the method described in any one of the first aspects.

[0073] It should be understood that the processor in this embodiment may be a central processing unit (CPU) or a graphics processing unit (GPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0074] It can be understood that the memory in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include but not be limited to these and any other suitable types of memory.

[0075] The fourth embodiment provides a readable storage medium, in which a program is stored. When the program is executed, the method described in any one of the first aspects is implemented.

[0076] It is worth noting that if the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions for causing an electronic device to execute all or part of the steps of the method described in each embodiment of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories, random access memories, magnetic disks, or optical discs, etc., which can store program codes.

[0077] This embodiment also provides a program product, which implements the method described in any of the above method embodiments when executed by an electronic device.

[0078] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a program product. The program product includes one or more instructions. When the instructions are loaded and executed on an electronic device, the processes or functions described in this embodiment are generated in whole or in part. The instructions can be stored in a readable storage medium or transmitted from one readable storage medium to another. For example, the instructions can be transmitted from a website, server, or data center to another website, server, or data center in a wired manner (such as coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wirelessly (such as infrared, Bluetooth, microwave, etc.). The readable storage medium can be any available medium accessible by the electronic device or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a Digital Video Disc (DVD) with high density), or a semiconductor medium (such as a Solid State Disk (SSD)), etc.

[0079] Although the embodiments of the present invention have been described in detail above, it is obvious to those skilled in the art that various modifications and changes can be made to these embodiments. However, it should be understood that such modifications and changes are all within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein can have other embodiments and can be implemented or realized in various ways.

Claims

1. A method for segmenting lower respiratory tract images, characterized in that, Including: S1: Extracting the trachea and main bronchial branch images from the original lung image; S2: Segmenting the lung vascular image to generate a lung vascular segmentation result; according to the lung vascular segmentation result, extracting the main lung vascular image and vascular branches; S3: Traversing each vascular branch and performing segmentation within the range where the vascular branch is located to generate small tracheal branch images; including: S31: Traversing each branch of the blood vessel, connecting the starting point and ending point of the currently traversed branch as a line segment as the cylinder main axis, making a cylinder with a radius of 12 pixels, and removing the part outside the cylinder on the CT image; traversing each two-dimensional image I in the cross-section, sagittal plane, and coronal plane of the cylinder, locating the possible two-dimensional tracheal cross-section SecC through binarization and region growing, and counting the number of pixels NsecC of each two-dimensional tracheal cross-section; S32: Removing the two-dimensional tracheal cross-sections with a similarity to the bronchus lower than the similarity threshold to retain the two-dimensional tracheal cross-sections that may be the bronchus with a probability greater than the probability threshold; S33: Setting the threshold for small tracheal branches and obtaining the edge EdgeSecC of each possible two-dimensional tracheal cross-section, where the edge points are the points among the four-connected points of all points on the two-dimensional tracheal cross-section that do not belong to the points on the two-dimensional tracheal cross-section; S34, score each possible two-dimensional tracheal cross-section, and the scoring score satisfies: where x and y are the point coordinates in the two-dimensional image, and z is the layer number of the two-dimensional image; the scoring score is greater than 0, and the higher the scoring score, the greater the probability of being a bronchial cross-section branch; S35: Retaining the M high-score cross-sections SecC2 with the highest scores, and fitting an ellipse to each high-score cross-section SecC2 to locate the tracheal cross-section in the cylinder; S4: Merging all small tracheal branch images and main bronchial branch images, and performing region growing on the tracheal trunk after the image merging to merge some small tracheal branches with the main tracheal branches; first connecting the remaining unmerged tracheal branches between branches, and then connecting the tracheal trunks; the conditions for connecting between branches include that the distance between the midpoints of the two ends of the connection is less than the second distance value and the included angle between the connection direction of the midpoints of the two ellipses and the regression line of the two ends of the ellipses is less than the included angle threshold, and supplementing the trachea between the two ellipses to generate a complete lower respiratory tract trachea image; S5: Performing bronchial leakage detection and removal on the lower respiratory tract trachea image to generate an accurate bronchial image.

2. The method according to claim 1, characterized in that, The extraction of the trachea and main bronchial branch images from the original lung image includes: S11: Obtaining a lung CT image, performing a derivative operation to obtain the second derivative of the image; constructing a new diffusion coefficient according to the second derivative of the image; updating the anisotropic diffusion formula according to the new diffusion coefficient to obtain a smoothed image; S12: Determining bronchial seed points according to the smoothed image, and updating the pixel difference ratio each time an iteration is performed, and performing cyclic iteration to obtain the main bronchial branch image.

3. The method according to claim 2, characterized in that, Determining bronchial seed points according to the smoothed image, and updating the pixel difference ratio each time an iteration is performed, and performing cyclic iteration to obtain the main bronchial branch image, including: S111: Taking the average of the pixel values of the previous segmentation result to obtain the average pixel value, and stopping the iteration when the average pixel value is greater than the maximum average pixel value; S112, find the edge pixel set of the last segmentation result, traverse each target point in the 26 neighborhoods of each pixel of the edge pixel set, and make a difference between the pixel value of the current target point and the average pixel value of the last iteration. If the difference is less than the pixel difference ratio diffT, set the current target point as a valid point; S113, traversing the 6 neighborhoods of each of the valid points, subtracting the pixel value of each point in the 6 neighborhoods from the average pixel value, and if all of them are less than the pixel difference ratio diffT, setting the current valid point as a point on the bronchus; S114, calculating the number of bronchial pixels of the last iterative segmentation and the number of pixels of the current bronchial segmentation, and performing a subtraction between the two and dividing by the number of pixels of the last bronchial segmentation to obtain a count difference ratio. When the count difference ratio is greater than the maximum iterative difference ratio, the iteration is stopped, and the last segmentation result is used as the final result of the bronchial segmentation. S115, based on the current pixel difference ratio plus the pixel difference ratio step, the pixel difference ratio is updated, and when it is determined that the updated pixel difference ratio is greater than the maximum pixel difference ratio diffTmax, the iteration is stopped, and the last segmentation result is used as the final result of bronchial segmentation.

4. The method according to claim 2, characterized in that, Determining bronchial seed points according to the smoothed image includes: Starting from the head direction of the smoothed image, a two-dimensional image from the tracheal entrance to the area outside the lungs is selected as the first image; the lung tissue in the first image is separated by the Otsu threshold method to obtain the second image; after adding the edge area based on the second image, flood filling is performed, and the filled image is inverted to obtain the third image; the edge area of ​​the third image is eliminated to obtain the fourth image; the pixel point coordinates in the fourth image whose pixel value is equal to the middle trachea are obtained as the coordinates of the seed point in the smoothed image.

5. The method according to claim 1, characterized in that, The segmenting of the pulmonary blood vessel image to generate a pulmonary blood vessel segmentation result includes: S21, selecting the two-dimensional image at the central position in the z direction in the CT image to calculate the lung seed point, performing double lung region growth, and obtaining the double lung segmentation result with bronchi; S22, when the bronchial main branch image is in the lung segmentation result, subtract the bronchial main branch image from the double lung segmentation result with bronchi to obtain a lung segmentation result; S23, based on the lung segmentation result, a closing operation is performed on the lungs, the double lung segmentation image area with holes is filled into a double lung solid area, and the lung segmentation result is subtracted from the solid lung image area to obtain a pulmonary blood vessel segmentation result.

6. The method according to claim 1, characterized in that, Also includes: S36, projecting the three fitted ellipses of the high-resolution cross sections onto a vertical plane, and determining whether the high-resolution cross section is a measure of a bronchial cross section, requiring that the three projections at least partially overlap, and the distances from the midpoints of the three ellipses to the regression line are lower than a first distance threshold, and satisfying: Among them, is the sum of the distances from the centers of the fitted ellipses of the three high-score cross-sections to the regression line, is the median, is the scoring score for the i-th two-dimensional tracheal cross-section, is the i-th two-dimensional tracheal cross-section, where i is a positive integer, is the intersection over union of the areas of the projections of the fitted ellipses of the three high-score cross-sections, is the preliminary measurement value during the preliminary selection of the upper and lower cross-sections, is the subsequent measurement value during the subsequent selection of the upper and lower cross-sections; According to the measurement value 1 and 2, cross-sections that do not belong to the trachea are removed from the high-score cross-section SecC2i to form a target cross-section Sec, and the target cross-section Sec contains all the ellipses projected onto the corresponding vertical planes; the target cross-sections are connected to form the small tracheal branch image.

7. The method according to claim 6, wherein The step of connecting the target cross sections to form the bronchial branch image comprises: S37. Connect the adjacent L target cross-sections to form a segmentation block; classify the unconnected cross-sections, and search for the distance from the unconnected cross-section to the center of the ellipse of the connected cross-section and the connected cross-section with the smallest included angle between the regression line of the unconnected cross-section and its adjacent cross-section and the regression line of the connected cross-section as the classification target; incorporate the currently unconnected cross-section into the segmentation block to which the classification target belongs. S38. Judge pairwise whether the segmentation blocks meet the mergeable conditions. The mergeable conditions include that the central distance between the center points of the two elliptical cross-sections that can be connected, the included angle between the regression line and the connecting line of the midpoints of the ellipses, and the included angle between the regression lines are the smallest and less than the set threshold, and then complete the airways at both ends that can be connected. S39. Retain the segmentation block with the smallest Hausdorff distance to the points on the blood vessel branch and less than the distance threshold in the segmentation block as the bronchial branch corresponding to the current blood vessel branch.

8. The method according to claim 1, wherein Perform bronchial leakage detection and removal on the lower respiratory tract airway image, including: S51. Refine the bronchial segmentation result to obtain the bronchial main trunk, traverse the points on the main trunk starting from the main airway entrance to form a bronchial tree; the bronchial tree includes all bronchial branches, and each branch includes two branch nodes and a branch path; when there is more than one branch path between two nodes, it is confirmed that there is a leakage in the segmentation. S52. Calculate the center point of the leaking main trunk and fit an ellipse to obtain the center point of the ellipse and the major and minor axes. Perform dilation operation based on the elliptical spherical kernel on the center of the ellipse to obtain the leakage area, and remove the leakage area from the lower respiratory tract airway image to obtain an accurate bronchial image.

9. A segmentation device for lower respiratory tract images, for the method according to any one of claims 1 to 8, wherein Including: A segmentation unit for segmenting the pulmonary vascular image to generate a pulmonary vascular segmentation result. An extraction unit for extracting the trachea and bronchial main branch images from the original pulmonary image; according to the pulmonary vascular segmentation result, extracting the pulmonary vascular main trunk image and vascular branches; traversing each vascular branch and performing segmentation within the range where the vascular branch is located to generate small tracheal branch images. A synthesis unit for mosaicking all the small tracheal branch images and the bronchial main branch images to generate a complete lower respiratory tract airway image. A correction unit for performing bronchial leakage detection and removal on the lower respiratory tract airway image to generate an accurate bronchial image.

10. The device according to claim 9, wherein The correction unit is further configured to obtain a pulmonary CT image, perform a derivative operation to obtain the second derivative of the image; construct a new diffusion coefficient according to the second derivative of the image; update the anisotropic diffusion formula according to the new diffusion coefficient to obtain a smoothed image.

11. An electronic device, wherein Including a memory and a processor, and a program is stored on the memory and can run on the processor. When the program is executed by the processor, the electronic device implements the method according to any one of claims 1 to 8.

12. A readable storage medium, wherein the readable storage medium stores a program, and wherein When the program is executed, the method according to any one of claims 1 to 8 is implemented.

Images