A method and system for airway segmentation based on fractal dimension coefficient

Abstract

The present invention discloses a method and system for airway segmentation based on fractal dimension coefficients. The method calculates the fractal dimension and / or airway density value of each pixel using airway labels. This method then assigns weights to each pixel of the training image based on the fractal dimension and / or airway density value to guide the training of a three-dimensional convolutional network, resulting in a deep learning model capable of more precise airway segmentation. The present invention accurately and efficiently segmentes lung airways using a deep neural network based on fractal dimension coefficients, particularly enabling more in-depth and precise airway segmentation results. Furthermore, the present invention can achieve fully automatic airway segmentation.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to an airway segmentation method and system based on fractal dimension coefficients. Background Art

[0002] With the development of medical imaging technology, CT scanning has become an important tool for clinical diagnosis and treatment. In the diagnosis process based on chest CT images, obtaining accurate airway segmentation is of great value in treating common lung diseases such as chronic obstructive pulmonary disease (COPD), asthma, and lung cancer. However, traditional airway segmentation methods mainly rely on manual operations, which are inefficient and require improved accuracy. Therefore, the development of an automatic and accurate computer-aided airway segmentation system is of great value.

[0003] In recent years, deep learning technology has achieved remarkable results in the field of medical image processing, providing a new solution for airway segmentation. Existing deep learning-based airway segmentation algorithms have shown improved segmentation accuracy compared to traditional segmentation techniques. However, due to the high complexity of CT images, the small diameter of the terminal bronchi, and the complex surrounding tissue environment, which are easily affected during the segmentation process, accurate segmentation of bronchioles has always been a challenge in the field of airway segmentation. Summary of the Invention

[0004] The purpose of the present invention is to address the deficiencies of the existing technology and provide an airway segmentation method and system based on fractal dimension coefficients for airway tree segmentation, aiming to improve the segmentation accuracy of bronchioles.

[0005] The technical solutions adopted in the present invention are as follows:

[0006] An airway segmentation method based on fractal dimension coefficients comprises the following steps:

[0007] Acquire a lung CT image to be segmented;

[0008] The lung CT image to be segmented is input into the trained three-dimensional neural network, and the lung airway segmentation result is obtained as output;

[0009] The trained three-dimensional neural network is obtained by the following training method:

[0010] Acquire a data set, wherein each sample of the data set includes a lung CT image, an airway label, and a fractal dimension weight and / or an airway density value of each pixel calculated based on the airway label; the airway density value of each pixel is a ratio of the sum of airway pixel values ​​in a window centered on the pixel to the total number of pixels in the window;

[0011] A three-dimensional neural network is constructed, and the lung CT image of each sample in the data set is used as input. The three-dimensional neural network outputs the predicted segmentation result; the three-dimensional neural network is trained by minimizing the loss function to obtain a trained three-dimensional neural network; the loss function represents the difference between the segmentation result predicted by all pixels and the airway label, and the weight of each pixel in the loss function is set according to the fractal dimension weight and / or airway density value of the pixel. The larger the fractal dimension weight and / or airway density value of the pixel, the greater the proportion of the pixel in the loss function.

[0012] Furthermore, the lung CT image is an image whose values ​​are distributed in [0, 1].

[0013] Furthermore, the lung CT image is obtained by processing as follows:

[0014] Obtain chest CT images;

[0015] Extract lung regions from chest CT images and remove redundant regions;

[0016] In the image with redundant areas removed, the HU values ​​in areas below the second HU threshold are set to 0, and the HU values ​​in areas above the third HU threshold are set to 1. The HU values ​​in the area between the second and third HU thresholds are uniformly scaled to the range of 0-1 according to the HU values ​​to obtain a lung CT image.

[0017] Furthermore, the structure of the three-dimensional neural network adopts 3DUNet, nnUNet, TransUNet or AirwayNet.

[0018] Furthermore, the fractal dimension weight of each pixel is calculated by the following method:

[0019] On the airway label, the local fractal dimension within the window centered on each airway pixel is calculated; the pixels in the overlapping areas of different windows are averaged and normalized to obtain the fractal dimension weight of each pixel.

[0020] Furthermore, the loss function is expressed as follows:

[0021] Loss=ω1*Loss_F D+ω2*Loss_D

[0022] ω1 and ω2 are weights, Loss_FD and Loss_D represent the first loss function and the second loss function constructed based on the fractal dimension weight of pixels and the airway density value, respectively.

[0023] Furthermore, the first loss function constructed based on the fractal dimension weight of the pixel is expressed as follows:

[0024]

[0025] Where, ω if represents the fractal dimension weight of the i-th pixel, g i represents the segmentation result output by the i-th pixel network, l i represents the airway label of the i-th pixel, α is a hyperparameter, and M represents the number of pixels in the lung CT image;

[0026] The second loss function constructed based on the airway density value of the pixel is expressed as follows:

[0027]

[0028] Where, ω id Represents the airway density value of the i-th pixel.

[0029] Furthermore, it also includes: using a sliding window to perform multiple predictions on the lung CT image to be segmented, and finally counting the segmentation results predicted for each pixel in different sliding windows, and determining that the pixels predicted as airway pixels are greater than a threshold number as airway pixels.

[0030] Furthermore, the method further includes: calculating the sizes of connected areas of the predicted image segmentation result, and selecting the second largest connected area as the final segmentation result.

[0031] An airway segmentation system based on fractal dimension coefficients includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the airway segmentation method based on fractal dimension coefficients is implemented.

[0032] The present invention has the following beneficial effects:

[0033] 1. The present invention adopts deep learning as the segmentation method and constructs a three-dimensional fully convolutional neural network, which can realize fully automatic segmentation of the lung airway and reduce labor costs.

[0034] 2. Accurate segmentation of bronchiolar segments has always been a difficult point in airway segmentation. The present invention uses fractal dimension coefficients to help the model segment the airways. The fractal dimension of each pixel in the airway label is calculated through the airway label morphology, and the weight of the pixels used in the training sample is set by the fractal dimension of the pixels. The fractal dimension is used to guide the model training, which can ultimately improve the segmentation depth and segmentation accuracy of the bronchiolar segments, providing doctors with more accurate diagnosis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The flowchart of the airway segmentation method based on fractal dimension coefficient;

[0036] Figure 2This is a diagram of the three-dimensional convolutional neural network structure. DETAILED DESCRIPTION

[0037] The present invention provides a lung airway segmentation method based on fractal dimension coefficient, which can more accurately segment the terminal bronchioles of the airway. Specifically, Figure 1 As shown, the following steps are included:

[0038] Step 1: Obtain the lung CT image to be segmented;

[0039] Step 2: Input the lung CT image to be segmented into the trained 3D neural network and output the lung airway segmentation result;

[0040] The three-dimensional neural network of the present invention uses the fractal dimension of pixels to set the weights of pixels in the training sample, and uses the fractal dimension to guide model training. The resulting three-dimensional neural network can improve the segmentation depth and segmentation accuracy of the bronchiolar segment, providing doctors with more accurate diagnosis results. Specifically, the training includes the following steps:

[0041] Step 1: Acquire a dataset: Each sample of the dataset includes a lung CT image, an airway label, and a fractal dimension weight and / or airway density value of each pixel calculated based on the airway label;

[0042] In the present invention, the lung CT image refers to a chest CT image that removes non-lung areas and only contains the lung area. Generally, it can be obtained by a series of operations such as HU threshold screening, connected area calculation, and erosion and expansion. Specifically, the operation method is as follows:

[0043] Lung region extraction from chest CT images: a first HU threshold is set. Pixels in the chest CT image larger than the threshold are marked as non-lung candidate regions and can be distinguished by a number. Regions smaller than the threshold are marked as lung candidate regions and are marked with another number.

[0044] Based on a series of operations such as connected area calculation and corrosion and expansion, redundant areas are removed to obtain the final lung area extraction result.

[0045] In a preferred embodiment, the lung CT image is an image with values ​​distributed in [0, 1], which is convenient for neural network processing. Specifically, the operation method is as follows:

[0046] In the image with redundant areas removed, the HU values ​​in areas below the second HU threshold are set to 0, and the HU values ​​in areas above the third HU threshold are set to 1. The HU values ​​in the area between the second and third HU thresholds are uniformly scaled to the interval 0-1 according to the HU values, thus obtaining a lung CT image with a value distribution in the range [0,1].

[0047] In the present invention, the airway label refers to a labeled image of the airway area in the chest CT image. Generally, professionals can be invited to perform the labeling.

[0048] In the present invention, the fractal dimension weight of each pixel is calculated as follows:

[0049] On the airway label, calculate the local fractal dimension within the window centered on each airway pixel. Generally, the box counting method can be used to calculate the fractal dimension FD. The formula is as follows:

[0050]

[0051] Where r represents the radius of the counting box, and several counting boxes are used to cover the airway image corresponding to the window. N(r) represents the minimum number of boxes required to cover the airway.

[0052] Then, the pixels in the overlapping areas of different windows are averaged and normalized to obtain the fractal dimension weight of each pixel. The larger the fractal dimension of the pixel, the greater the impact on the training. The formula for averaging the pixels in the overlapping areas of different windows is as follows:

[0053]

[0054] Where i represents the number of times the current pixel is covered by different windows, and FD(i) represents the value of the fractal dimension of each window covering the current pixel.

[0055] In the present invention, the airway density value of each pixel is the ratio of the sum of the airway pixel values ​​in the window centered on the pixel to the total number of pixels in the window; the formula is as follows:

[0056]

[0057] where p i represents the value of the i-th airway pixel in the window, N represents the number of airway pixels in the window, and N(i) represents the total number of pixels in the window.

[0058] Step 2: Construct a 3D neural network, taking the lung CT image of each sample in the dataset as input, and outputting the predicted segmentation result. Train the 3D neural network by minimizing the loss function to obtain a trained 3D neural network.

[0059] In the present invention, the structure of the three-dimensional neural network can adopt networks such as 3DUNet, nnUNet, TransUNet, and AirwayNet.

[0060] In the present invention, the loss function characterizes the difference between the predicted segmentation results of all pixels and the airway labels, and can be expressed by the average error of all pixels, relative deviation, absolute deviation, absolute error, the ratio of the sum of the products of the predicted segmentation results of each pixel and the airway label to the sum of the predicted segmentation results of all pixels and the airway labels, etc., wherein the larger the fractal dimension weight and / or airway density value of the pixel, the greater the proportion of the pixel in the loss function.

[0061] Taking the ratio representation as an example, the first loss function constructed based on the pixel fractal dimension weight is expressed as follows:

[0062]

[0063] Where, ω if represents the fractal dimension weight of the i-th pixel, g i represents the segmentation result output by the i-th pixel network, l i represents the airway label of the i-th pixel, α is a hyperparameter, and M represents the number of pixels in the lung CT image.

[0064] The second loss function constructed based on the airway density value of the pixel is expressed as follows:

[0065]

[0066] Where, ω id Represents the airway density value of the i-th pixel.

[0067] The loss function can use the fractal dimension weight or the airway density value to guide the optimization alone, or the fractal dimension weight and the airway density value to guide the optimization at the same time. The loss function can be expressed as follows:

[0068] Loss=ω1*Loss_F D+ω2*Loss_D

[0069] ω1 and ω2 are weights, Loss_FD and Loss_D represent the first loss function and the second loss function constructed based on the fractal dimension weight of pixels and the airway density value, respectively.

[0070] By setting the weights of pixels in the training sample based on the fractal dimension of the pixels and guiding the model training based on the fractal dimension, the segmentation depth and the segmentation accuracy of the bronchiolar segment can be improved, providing doctors with more accurate diagnosis results. The effects of the present invention will be further explained in conjunction with a specific embodiment below:

[0071] An airway segmentation method based on fractal dimension coefficients includes the following steps:

[0072] Step 1: Acquire a dataset: Each sample of the dataset includes a lung CT image, an airway label, and a fractal dimension weight and / or airway density value of each pixel calculated based on the airway label;

[0073] Step 1 specifically includes the following sub-steps:

[0074] Step 101: Obtain chest CT images of several patients and invite professionals to annotate the CT images to obtain airway labels;

[0075] Step 102: Extract the lung region from the chest CT image: Set a first HU threshold of -600. Pixels in the chest CT image larger than the threshold are marked as non-lung candidate regions, which can be distinguished by a number. In this embodiment, the number is set to 2. Regions smaller than the threshold are marked as lung candidate regions. In this embodiment, the number is set to 1.

[0076] Step 103: Based on a series of operations such as connected area calculation and corrosion and expansion, redundant areas are removed to obtain the final lung area extraction result.

[0077] Step 104: Adjust the window width and window level of the extracted chest CT image. Set the image regions with HU values ​​below the second HU threshold of -1200 to 0, and the image regions with HU values ​​above the third HU threshold of 600 to 1. The image regions with HU values ​​between -1200 and 600 are uniformly scaled to the range of 0-1 based on the HU values.

[0078] Furthermore, a window with a size of 128*128*64 (x*y*z) is used to crop the chest CT image, and the cropped slices are used as lung CT images.

[0079] Step 105: Calculate the fractal dimension of the airway based on the annotated image. Centered on each airway pixel, use a 64*8*8 (x*y*z) window to calculate the local fractal dimension according to formula (1-1). Finally, average the overlapping regions according to formula (1-2) and normalize them to obtain the final airway fractal dimension weight map. Pixels with larger fractal dimensions have a greater impact on training.

[0080] Step 106: Based on the ratio of the sum of the airway pixel values ​​in a 7*7*7 window centered on the current pixel to the total number of pixels in the window, use formula (1-3) to calculate the airway density of the current area, which is used to represent the density value of the current pixel and generate an airway density weight map. The denser the pixel, the greater the impact on training.

[0081] Furthermore, part of the chest CT image and the annotated image is used as training data, and part is used as test data. In this embodiment, the annotated CT image is divided into two parts according to the ratio of 8:2, the larger part is used for training, and the smaller part is used for testing the segmentation result.

[0082] Step 2: Construct a three-dimensional neural network, take the lung CT image of each sample in the data set as input, and output the predicted segmentation result of the three-dimensional neural network; train the three-dimensional neural network by minimizing the loss function to obtain a trained three-dimensional neural network; the three-dimensional neural network described in this embodiment adopts a three-dimensional fully convolutional neural network with 3DUnet as the skeleton; Figure 2 As shown in the figure, this network consists of a series of downsampling and upsampling layers. The downsampling layers perform four downsampling cycles, with the output channels of each layer being 32, 64, 128, and 256, respectively. The upsampling layers perform four upsampling cycles, with the output channels of each layer being 128, 64, 32, and 1, respectively. Each downsampling and upsampling layer consists of a convolutional layer and a pooling layer. The convolutional layer includes convolution, batch normalization, and ReLU activation, while the pooling layer uses average pooling with a stride of 2. Skip connections are implemented between each corresponding downsampling and upsampling layer, and the concatenation is performed along the channel dimension.

[0083] Step 302: Take the lung CT image of each sample in the dataset as input and calculate the model loss based on the model segmentation results, the fractal dimension and density distribution weight calculated in step 2, and the airway tree label provided by the doctor in step 1. The model loss specifically includes the following two parts:

[0084] According to the network segmentation result g of each pixel i , airway label i and fractal dimension weight ω if The product of the network segmentation result g of each pixel i , airway label i The ratio between the sum of is used to determine the first component Loss_FD according to formula (1-4), where the hyperparameter α is set to 0.5 in this experiment.

[0085] According to the network segmentation result g of each pixel i , airway label i and airway density distribution weight ω id The product of the network segmentation result g of each pixel i , airway label i The ratio between the sum of the two components is used to determine the second component Loss_D according to formula (1-5), where the hyperparameter α is set to 0.5 in this experiment.

[0086] The two components are added with weights of 0.5 and 0.5 to obtain the final loss function:

[0087] Loss=0.5*Loss_FD+0.5*Loss_D

[0088] The Adam optimizer was used as the network optimizer, and the learning rate was set to 0.00001. The network parameters were initialized using the He Kaiming initialization method to obtain a trained three-dimensional neural network.

[0089] The trained three-dimensional neural network is tested using a test set. In this embodiment, a sliding window is used to perform multiple predictions on the test sample. Finally, the difference in the prediction results of each pixel using different sliding windows is counted and a comprehensive judgment is made. The details are as follows:

[0090] A sliding window of 128*128*64 (x*y*z) was used to predict the test data samples with a step size of 64*64*32 (x*y*z), and a threshold of 0.5 was set to distinguish the foreground (airway pixels) from the background (non-airway pixels).

[0091] During the sliding window prediction process, each pixel will be predicted eight times by different sliding windows, generating eight prediction results. In this embodiment, pixels that are predicted as foreground six or more times are identified as foreground, and the remaining pixels are identified as background to obtain the final segmentation result.

[0092] The output results are post-processed by calculating the size of the connected areas, selecting the second largest connected area as the final segmentation result, and removing redundant false positive areas and broken parts.

[0093] In this example, the dice coefficient, precision (pre), branch length detection rate (TD), and branch number detection rate (BD) are used to measure the final performance of the model. The calculation formula for TD is as follows:

[0094]

[0095] Where T det represents the total length of all correctly segmented branches in the airway segmentation result, T lab Represents the total length of all branches in the airway label.

[0096] The calculation formula for BD is as follows:

[0097]

[0098] Among them B det Represents the number of correctly segmented branches in the airway segmentation result, representing B lab The total number of branches in the airway label.

[0099] The final segmentation performance of the deep learning model trained based on the fractal dimension coefficient in this example is shown in Table 1:

[0100] Table 1: Performance evaluation indicators of the airway segmentation model of the present invention

[0101]

[0102] It can be seen that this method can obtain airway segmentation results with high airway segmentation completeness and high airway segmentation depth while maintaining a relatively high overall segmentation accuracy, thus providing assistance for subsequent treatment.

[0103] Corresponding to the aforementioned embodiment of an airway segmentation method based on fractal dimension coefficients, the present invention also provides an embodiment of an airway segmentation system based on fractal dimension coefficients.

[0104] An embodiment of the present invention provides an airway segmentation system based on fractal dimension coefficients, including one or more processors for implementing an airway segmentation method based on fractal dimension coefficients in the above embodiment.

[0105] The embodiment of the airway segmentation system based on fractal dimension coefficients of the present invention can be applied to any device with data processing capability, such as a computer or other device.

[0106] The device embodiments may be implemented through software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a logically defined device, a processor of any device with data processing capabilities reads the corresponding computer program instructions from a non-volatile memory into the memory and executes them. From a hardware perspective, this includes a processor, memory, a network interface, and a non-volatile memory. In addition, any device with data processing capabilities in which the device in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0107] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0108] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0109] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the airway segmentation method based on fractal dimension coefficients in the above embodiment is implemented.

[0110] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0111] Finally, according to the above description, the above embodiments of the present invention are for illustration rather than limitation, and ordinary technicians in this field can make various changes and modifications based on these embodiments, but these changes and modifications do not essentially depart from the spirit and scope of the present invention.

Claims

1. An airway segmentation method based on fractal dimension coefficient, characterized in that: The following steps are involved: Acquire a lung CT image to be segmented; The lung CT image to be segmented is input into the trained three-dimensional neural network, and the lung airway segmentation result is obtained as output; The trained three-dimensional neural network is obtained by the following training method: Acquire a data set, wherein each sample of the data set includes a lung CT image, an airway label, and a fractal dimension weight and an airway density value of each pixel calculated based on the airway label; the airway density value of each pixel is a ratio of the sum of airway pixel values ​​in a window centered on the pixel to the total number of pixels in the window; A three-dimensional neural network is constructed, and the lung CT image of each sample in the data set is used as input. The three-dimensional neural network outputs a predicted segmentation result. The three-dimensional neural network is trained by minimizing a loss function to obtain a trained three-dimensional neural network. The loss function represents the difference between the segmentation result predicted by all pixels and the airway label. The weight of each pixel in the loss function is set based on the fractal dimension weight and airway density value of the pixel. The larger the fractal dimension weight and airway density value of the pixel, the greater the proportion of the pixel in the loss function. The fractal dimension weight of each pixel is calculated by the following method: On the airway label, the local fractal dimension within the window centered on each airway pixel is calculated; the pixels in the overlapping areas of different windows are averaged and normalized to obtain the fractal dimension weight of each pixel.

2. The method according to claim 1, characterized in that The lung CT image is an image whose values ​​are distributed in [0, 1].

3. The method according to claim 2, characterized in that The lung CT image is obtained by the following processing method: obtaining a chest CT image; Extract lung regions from chest CT images and remove redundant regions; In the image with redundant areas removed, the HU values ​​in areas below the second HU threshold are set to 0, and the HU values ​​in areas above the third HU threshold are set to 1. The HU values ​​in the area between the second and third HU thresholds are uniformly scaled to the range of 0-1 according to the HU values ​​to obtain a lung CT image.

4. The method according to claim 1, wherein The structure of the three-dimensional neural network adopts 3DUNet, nnUNet, TransUNet or AirwayNet.

5. The method according to claim 1, wherein The loss function is expressed as follows: Loss=ω1*Loss_F D+ω2*Loss_D; ω1 and ω2 are weights, Loss_FD and Loss_D represent the first loss function and the second loss function constructed based on the fractal dimension weight of pixels and the airway density value, respectively.

6. The method according to claim 5, characterized in that The first loss function constructed based on the pixel fractal dimension weight is expressed as follows: Where, ω if represents the fractal dimension weight of the i-th pixel, g i represents the segmentation result output by the i-th pixel network, l i represents the airway label of the i-th pixel, α is a hyperparameter, and M represents the number of pixels in the lung CT image; The second loss function constructed based on the airway density value of the pixel is expressed as follows: Where, ω id Represents the airway density value of the i-th pixel.

7. The method according to claim 1, characterized in that Also includes: The lung CT image to be segmented is predicted multiple times using a sliding window, and finally the segmentation results of each pixel predicted in different sliding windows are counted, and the pixels predicted as airway pixels whose number of times is greater than a threshold are determined to be airway pixels.

8. The method according to claim 1, characterized in that Also includes: The segmentation result of the image prediction is calculated by calculating the size of the connected area, and the second largest connected area is selected as the final segmentation result.

9. An airway segmentation system based on fractal dimension coefficients, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the airway segmentation method based on fractal dimension coefficients according to any one of claims 1 to 8 is implemented.

Images