Lung image segmentation method, device, storage medium and computer equipment

Abstract

The present invention relates to the fields of artificial intelligence and digital medical technology, and provides a lung image segmentation method, apparatus, storage medium, and computer equipment. The method comprises: constructing a lung nodule segmentation model based on a fully convolutional neural network model using a densely connected hybrid dilated convolution module and a hierarchical attention mechanism; training the lung nodule segmentation model using lung parenchyma image samples as input and lung nodule segmentation images of the lung parenchyma image samples as output to obtain a trained lung nodule segmentation model; obtaining a lung image to be processed, segmenting the lung image using a lung parenchyma segmentation algorithm to obtain a lung parenchyma image; and obtaining a lung nodule segmentation result for the lung image using the trained lung nodule segmentation model based on the lung parenchyma image. The above method can effectively improve the segmentation accuracy of lung nodules.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and digital medical technology, and in particular to a lung image segmentation method, device, storage medium and computer equipment. Background Art

[0002] In recent years, due to various reasons such as the deterioration of air quality, the deepening harm of secondhand smoke, and the influence of occupational factors, the incidence of lung diseases has gradually shown a high incidence trend. Among them, lung nodules are one of the early signs of lung cancer, and their segmentation methods have become one of the hot topics widely studied in the industry.

[0003] In the existing technology, a fully convolutional neural network model is usually used as the basic model for building a lung nodule segmentation model. However, when using a fully convolutional neural network model to downsample lung images, a lot of contextual information is often lost, making it difficult for the upsampling process to completely restore the detailed information of the target to be segmented and its corresponding spatial dimensions, resulting in unclear upsampling results, which in turn leads to low lung nodule segmentation accuracy. In addition, the dilated convolution used in the traditional fully convolutional neural network model is prone to grid problems, and the appearance of grid problems will make the sampling results unable to cover all image features and will also lose the continuity of image features, which further reduces the accuracy of lung nodule segmentation results. Summary of the Invention

[0004] In view of this, the present application provides a lung image segmentation method, apparatus, storage medium and computer equipment, the main purpose of which is to solve the technical problem of low accuracy in lung nodule segmentation.

[0005] According to a first aspect of the present invention, a lung image segmentation method is provided, the method comprising:

[0006] Based on a fully convolutional neural network model, a lung nodule segmentation model is constructed through densely connected hybrid dilated convolution modules and a hierarchical attention mechanism;

[0007] Taking lung parenchyma image samples as input and lung nodule segmentation images of the lung parenchyma image samples as output, the lung nodule segmentation model is trained to obtain a trained lung nodule segmentation model;

[0008] Acquire a lung image to be processed, segment the lung image using a lung parenchyma segmentation algorithm, and obtain a lung parenchyma image;

[0009] According to the lung parenchyma image, the lung nodule segmentation result of the lung image is obtained through the trained lung nodule segmentation model.

[0010] According to a second aspect of the present invention, a lung image segmentation device is provided, comprising:

[0011] A model building module for constructing a lung nodule segmentation model based on a fully convolutional neural network model, using densely connected hybrid dilated convolution modules and a hierarchical attention mechanism;

[0012] a model training module, configured to train a lung nodule segmentation model using lung parenchyma image samples as input and lung nodule segmentation images of the lung parenchyma image samples as output, thereby obtaining a trained lung nodule segmentation model;

[0013] An image segmentation module is used to obtain a lung image to be processed, and segment the lung image using a lung parenchyma segmentation algorithm to obtain a lung parenchyma image;

[0014] The image processing module is used to obtain the lung nodule segmentation result of the lung image based on the lung parenchyma image through the trained lung nodule segmentation model.

[0015] According to a third aspect of the present invention, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned lung image segmentation method is implemented.

[0016] According to a fourth aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned lung image segmentation method when executing the program.

[0017] The present invention provides a lung image segmentation method, device, storage medium and computer equipment. First, based on a fully convolutional neural network model, a lung nodule segmentation model is constructed through a densely connected mixed dilated convolution module and a hierarchical attention mechanism. Then, the lung parenchyma image samples are used as input and the manually labeled lung nodule image samples are used as output to train the lung nodule segmentation model, and then the lung image to be processed is segmented to obtain a lung parenchyma image. Finally, the lung image is processed by the trained lung nodule segmentation model to obtain a lung nodule segmentation result. The above method obtains a lot of contextual information in the lung image through the up and down sampling structure and hierarchical attention mechanism of the fully convolutional neural network model, and also selectively amplifies valuable feature channels and suppresses useless feature channels, so that the lung nodule segmentation model can better capture the key feature information of the lung nodules, thereby improving the segmentation accuracy of the lung nodules. In addition, the above method can also extract the features of the multi-level receptive field in the lung image by adding a densely connected mixed dilated convolution module to the fully convolutional neural network model, and can avoid the network grid problem, further improving the segmentation accuracy of the lung nodules.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 A schematic diagram of a process flow of a lung image segmentation method provided by an embodiment of the present invention is shown;

[0021] Figure 2 A schematic diagram of a scenario of performing binarization processing on a lung image provided by an embodiment of the present invention is shown;

[0022] Figure 3 A schematic diagram of a scenario for performing three-dimensional connectivity reconstruction on a lung image provided by an embodiment of the present invention is shown;

[0023] Figure 4 A schematic diagram of a scenario of clustering lung images provided by an embodiment of the present invention is shown;

[0024] Figure 5 A schematic diagram of a scenario of performing an expansion operation on a lung image according to an embodiment of the present invention is shown;

[0025] Figure 6 A schematic diagram of a scenario of performing mask operation on a lung image according to an embodiment of the present invention is shown;

[0026] Figure 7 A schematic structural diagram of a lung image segmentation device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0028] In one embodiment, Figure 1As shown, a lung image segmentation method is provided, and the method is applied to a computer device such as a server as an example for illustration. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The above method includes the following steps:

[0029] 101. Based on the fully convolutional neural network model, a lung nodule segmentation model is constructed through densely connected hybrid dilated convolution modules and hierarchical attention mechanism.

[0030] Specifically, convolutional neural networks (CNNs) are often used as the basic network for model building in image processing. Generally speaking, convolutional neural networks are mainly used to process 2D images. In this embodiment, a full convolutional neural network (V-Net network or U-Net network, etc.) is mainly used as the basis for building a lung nodule model. The full convolutional neural network is a deformation of the convolutional neural network. It adopts the short-circuit connection method of ResNet and replaces the upsampling and downsampling pooling layers in the convolutional neural network with convolution layers. In the structure of the full convolutional neural network, one side is a gradually compressed path composed of an encoder, and the other side is a gradually decompressed path composed of a decoder. In this embodiment, the full convolutional neural network has a better effect on feature extraction of lung images than ordinary convolutional neural networks.

[0031] Furthermore, on the basis of the full convolutional neural network, a densely connected hybrid dilated convolutional network module and a hierarchical attention mechanism can be added to construct an initial lung nodule segmentation model. Among them, the hierarchical attention mechanism refers to the weight distribution of each layer of the encoder and decoder of the full convolutional neural network, so as to highlight the key features of the target to be detected; the dilated convolutional network refers to a network that dilates the convolution kernel of the convolutional network according to the dilation coefficient, that is, the dilated convolution kernel is obtained by zero-filling the area not occupied by the original convolution kernel. In this embodiment, the dilated convolutional network can expand the receptive field of the convolution kernel without increasing the amount of calculation, and can also avoid the loss of detail features in the downsampling process. However, ordinary dilated convolutional networks can easily cause grid problems. Therefore, this embodiment uses hybrid dilated convolution to replace dilated convolution and add it to the full convolutional neural network. The dilation rate feature of the hybrid dilated convolution itself can be used to avoid the grid problem of the dilated convolution. Moreover, the densely connected structure of the hybrid dilated convolution can also reduce the training time of the network parameters.

[0032] In this embodiment, the encoder and decoder of the fully convolutional neural network can extract the features of the multi-level receptive field in the lung image through a densely connected hybrid dilated convolutional network, and can avoid the grid problem. The hierarchical attention mechanism can reduce unnecessary features and highlight key features, thereby better capturing the key information in the original image. Therefore, based on the fully convolutional neural network model, the model constructed by densely connected hybrid dilated convolution modules and the hierarchical attention mechanism can effectively improve the performance of the lung nodule segmentation model, thereby improving the accuracy of lung nodule segmentation.

[0033] 102. The lung nodule segmentation model is trained with the lung parenchyma image sample as input and the lung nodule segmentation image of the lung parenchyma image sample as output to obtain a trained lung nodule segmentation model.

[0034] The lung parenchyma image sample refers to an image sample containing grayscale information obtained by segmenting the lung image, and the lung nodule segmentation image of the lung parenchyma image sample refers to an image sample obtained by manually annotating the lung parenchyma image sample. In this embodiment, each lung parenchyma image sample corresponds to a labeled lung nodule segmentation image. In addition, the lung parenchyma image sample and its corresponding lung nodule segmentation image sample can be obtained from a database or other means, which is not specifically limited in this embodiment.

[0035] Specifically, the computer device can first obtain a large number of lung parenchyma image samples and the lung nodule segmentation image corresponding to each lung parenchyma image sample, and then use each lung parenchyma image sample as input and the lung nodule segmentation image corresponding to the sample as output to continuously iterate the model constructed in step 101 to finally obtain a trained lung nodule segmentation model.

[0036] 103. Obtain a lung image to be processed, segment the lung image using a lung parenchyma segmentation algorithm, and obtain a lung parenchyma image.

[0037] The lung image refers to an image obtained by computed tomography of the lungs. In this embodiment, the lung image can be processed sequentially using a binarization algorithm, a three-dimensional connectivity algorithm, a clustering algorithm, a dilation algorithm, and a masking algorithm to obtain a lung parenchyma image. The binarization algorithm refers to the operation of binarizing pixels in an image, i.e., setting the grayscale values ​​of pixels on the image to 0 or 255, thereby making the entire image appear distinctly black and white. Common binarization algorithms include the grayscale average method, the bimodal method, the OTSU method, and the Niblack method. The three-dimensional connectivity algorithm refers to a three-dimensional image reconstruction method based on a connected domain algorithm, i.e., marking the binarized image pixels and performing a three-dimensional connectivity reconstruction to obtain a three-dimensional image. The clustering algorithm is an algorithm that segments an image into several target regions of different significance, and mainly includes fuzzy clustering, K-means clustering, and C-means clustering. The dilation algorithm is a morphological algorithm that merges all background points that are in contact with an object into the object, expanding the boundary outward. The masking algorithm refers to an algorithm that multiplies the pixels of the processed image with the original image. In this embodiment, the algorithm names used by the binarization algorithm, three-dimensional connectivity algorithm, clustering algorithm, expansion algorithm, and masking algorithm can be selected according to actual conditions, and this embodiment does not make any specific restrictions here.

[0038] 104. According to the lung parenchyma image, the lung nodule segmentation result of the lung image is obtained through the trained lung nodule segmentation model.

[0039] Specifically, after a lung parenchyma image is obtained through a series of algorithms, it can be input into a trained lung nodule segmentation model, and the lung nodules in the lung parenchyma image can be segmented by the lung nodule model to obtain a feature expression of the lung nodule segmentation result. In this embodiment, the feature information output by the lung nodule model can also be input into a nonlinear activation function, and then the voxel values ​​in the feature information are binarized by the nonlinear activation function to obtain the final lung nodule segmentation result.

[0040] The lung image segmentation method provided in this embodiment first constructs a lung nodule segmentation model based on a fully convolutional neural network model through a densely connected hybrid dilated convolution module and a hierarchical attention mechanism. Then, the lung nodule segmentation model is trained with lung parenchyma image samples as input and manually annotated lung nodule image samples as output, and then the lung image to be processed is segmented to obtain a lung parenchyma image. Finally, the lung image is processed by the trained lung nodule segmentation model to obtain a lung nodule segmentation result. The above method obtains a lot of contextual information in the lung image through the up and down sampling structure and hierarchical attention mechanism of the fully convolutional neural network model, and also selectively amplifies valuable feature channels and suppresses useless feature channels, so that the lung nodule segmentation model can better capture the key feature information of the lung nodules, thereby improving the segmentation accuracy of the lung nodules. In addition, the above method can also extract the features of the multi-level receptive field in the lung image by adding a densely connected hybrid dilated convolution module to the fully convolutional neural network model, and can avoid the network grid problem, thereby further improving the segmentation accuracy of the lung nodules.

[0041] In one embodiment, optionally, step 103 can be implemented by the following method: first, binarizing the lung image using a binarization algorithm to obtain a binarized lung image; then, performing connectivity reconstruction on the binarized lung image using a three-dimensional connectivity algorithm to obtain a three-dimensional lung image; then, clustering the three-dimensional lung image using a mean clustering algorithm to obtain a binarized three-dimensional connected lung parenchyma image; then, performing a dilation operation on the binarized three-dimensional connected lung parenchyma image to obtain a three-dimensional lung parenchyma contour; and finally, performing a masking operation on the three-dimensional lung parenchyma contour and the lung image to obtain a lung parenchyma image. This embodiment processes the lung image using a series of algorithms, including a binarization algorithm, a three-dimensional connectivity algorithm, a clustering algorithm, a dilation operation, and a masking operation. Compared to a single lung parenchyma segmentation algorithm in the prior art, this embodiment can effectively remove the chest cavity contour in the lung image and retain pleural adherent lung nodules, thereby effectively reducing the probability of missed lung nodules.

[0042] In one embodiment, optionally, the method for performing connectivity reconstruction on a binarized lung image can be implemented by the following steps: first, using a three-dimensional connectivity algorithm, the image pixels in the binarized lung image are connected in six three-dimensional neighborhoods (i.e., upper neighborhood, lower neighborhood, left neighborhood, right neighborhood, front neighborhood, and back neighborhood) to obtain multiple connected areas of the binarized lung image, and then the multiple connected areas of the binarized lung image are reconstructed in a predetermined order to obtain a three-dimensional image of the lungs.

[0043] In one embodiment, optionally, clustering processing of the three-dimensional lung image can be achieved by the following steps: first, using the distance between the center points of multiple connected regions of the three-dimensional lung image and the center point of the three-dimensional lung image as a clustering feature, clustering the three-dimensional lung image using a mean clustering algorithm to obtain multiple three-dimensional connected regions of the three-dimensional lung image, then retaining the binarized three-dimensional connected regions of the three-dimensional lung image, and setting the voxel values ​​of the non-binarized three-dimensional connected regions of the three-dimensional lung image to 1, to obtain a binarized three-dimensional connected image of the lung parenchyma. In this embodiment, the multiple three-dimensional connected regions include binarized three-dimensional connected regions and non-binarized three-dimensional connected regions, wherein the binarized three-dimensional connected regions refer to three-dimensional connected regions composed of binarized lung parenchyma, and the non-binarized three-dimensional connected regions refer to other three-dimensional connected regions other than the binarized three-dimensional connected regions.

[0044] In one embodiment, in order to facilitate the specific description of the implementation process of step 103, this embodiment uses a specific example to illustrate the implementation process of a series of algorithms such as binarization algorithm, three-dimensional connectivity algorithm, clustering algorithm, dilation operation and mask operation. In this embodiment, the OTSU binarization algorithm is first used to binarize the lung image to obtain a binary lung CT serial slice image. Among them, the OTSU binarization algorithm is a binarization algorithm based on the maximum inter-class variance method. It can divide the image into two parts, background and target, according to the grayscale characteristics of the image. The greater the inter-class variance between the background and the target, the greater the difference between the two parts of the image. When part of the target is mistakenly classified as the background or part of the background is mistakenly classified as the target, the difference between the two parts will become smaller. Therefore, the segmentation accuracy with the maximum inter-class variance is higher, wherein the image binarization diagram is as shown in FIG. Figure 1 Then, the image pixels in the binary lung CT serial slice image can be connected in the six areas of up, down, left, right, front and back in the three-dimensional direction through the three-dimensional connectivity algorithm to obtain multiple connected areas. Finally, the connected areas are reconstructed in a certain order to obtain a three-dimensional image of the lung. The schematic diagram of the connected reconstruction is shown in FIG. Figure 2 shown.

[0045] Furthermore, after obtaining the three-dimensional image of the lungs, the C-means clustering algorithm can be used to cluster the three-dimensional image of the lungs, wherein the C-means clustering algorithm is also called the K-means clustering algorithm, which is a distance-based clustering algorithm. In this embodiment, the distance between the center point of the connected region and the center point of the three-dimensional image of the lungs can be used as the clustering feature, and the three-dimensional image of the lungs can be processed based on the C-means clustering algorithm. In the C-means clustering algorithm, the closer the distance between two objects, the greater their similarity. Furthermore, objects with similar clustering features can be compacted into clusters, and finally the compact and independent clusters can be used as different clustering targets. This embodiment uses the C-means clustering algorithm to conveniently and accurately distinguish between the lung parenchyma, background, chest contour, noise and other parts in the three-dimensional lung image, and remove other areas by setting the voxel value of other areas to 1, retaining only the three-dimensional connected area composed of the binarized lung parenchyma, wherein the cluster segmentation diagram is as shown below. Figure 3 shown.

[0046] Furthermore, after obtaining the binarized lung parenchyma, the three-dimensional connected region formed by the binarized lung parenchyma can be expanded by a dilation operation, wherein the structural element size of the dilation operation is 3. In this embodiment, the lung parenchyma region can be expanded by a dilation operation, so that the pleural adhesion nodules that may be mistakenly deleted due to the deletion of the sternum part during the clustering segmentation process can be retained. Finally, the expanded lung parenchyma image and the lung CT image can be subjected to a mask operation, that is, the pixels of the expanded lung parenchyma image are correspondingly multiplied with the pixels of the lung CT image to obtain a lung parenchyma image containing grayscale information. Among them, the schematic diagrams of the expansion budget and the mask operation are shown as follows. Figure 4 and Figure 5 shown.

[0047] In one embodiment, optionally, step 104 can be implemented by the following method: first, the lung parenchyma image is input into a trained lung nodule segmentation model to obtain characteristic information of the lung image, and then the characteristic information of the lung image is input into a nonlinear activation function, and the characteristic information of the lung image is binarized by the nonlinear activation function to obtain a lung nodule segmentation result of the lung image. The nonlinear activation function can specifically be a sigmoid function. In this embodiment, the information output by the model is further input into a nonlinear activation function for binarization processing, thereby further improving the segmentation accuracy of the lung nodules.

[0048] In one embodiment, optionally, the above step 101 can be implemented by the following method: first, construct an encoder and a decoder of a fully convolutional neural network model, wherein the encoder consists of a densely connected mixed dilated convolution module, a first batch of normalization layers and a first activation layer, and the decoder consists of a densely connected deconvolution module, a second batch of normalization layers and a second activation layer. Then, a hierarchical attention mechanism is introduced between the encoder and the decoder to construct a lung nodule segmentation model, wherein the feature layers of the encoder and the decoder under the hierarchical attention mechanism are opposite to each other, and the feature pixels in the two-to-two feature layers are multiplied respectively.

[0049] In one embodiment, optionally, the densely connected hybrid dilated convolution module in the lung nodule segmentation model may include multiple first three-dimensional convolution layers, wherein each two first three-dimensional convolution layers of the multiple first three-dimensional convolution layers are connected by a dense structure. Furthermore, the densely connected deconvolution module in the lung nodule segmentation model may include multiple second three-dimensional convolution layers, wherein each two second three-dimensional convolution layers of the multiple second three-dimensional convolution layers are connected by a dense structure. In this embodiment, the number of first three-dimensional convolution layers and second three-dimensional convolution layers is the same, and in order to take into account both the accuracy and processing efficiency of the model, the number of first three-dimensional convolution layers and second three-dimensional convolution layers can be set to about 3-5.

[0050] In one embodiment, in order to facilitate the specific description of the implementation process of step 101, this embodiment is combined with a specific example model creation process. In this embodiment, the pulmonary nodule segmentation model is constructed based on the V-Net network and consists of two parts: an encoder and a decoder. Among them, the encoder part includes a dense mixed dilated convolution module, a batch normalization layer (BN) and an activation layer (ReLU). The dense mixed dilated convolution module can be composed of multiple 3D convolution layers. In this embodiment, the dense mixed dilated convolution module is composed of 5 3D convolution layers. In addition, the convolution kernel size is 3×3×3, and the convolution layers are connected in a dense structure. The expansion rate can be set to [1, 2, 5, 2, 1]. The decoder part includes a deconvolution module, a batch normalization layer and an activation layer. Among them, the deconvolution module can be composed of multiple 3D convolution layers. In this embodiment, the deconvolution module is composed of 5 3D convolution layers. In addition, the convolution kernel size is also 3×3×3, and the convolution layers are also connected in a dense structure. Among them, the encoder can continuously extract the key features of the image and compress the important information extracted from the low layer to the high layer, and the decoder can reconstruct the final output feature map from the feature map compressed by the encoder.

[0051] Furthermore, the encoder and decoder can be set to a depth of 7 layers, where the encoder's dense hybrid dilated convolution module can be set between the batch normalization layer and the activation layer, and the decoder's deconvolution module can also be set between the batch normalization layer and the activation layer. With this configuration, the encoder and decoder can extract features from the multi-level receptive field of the image through a densely connected structure and dilated convolutional networks, avoiding grid issues, improving model accuracy, and reducing network parameters and training time.

[0052] Furthermore, in this embodiment, a hierarchical attention mechanism is introduced between the encoder and decoder. Specifically, the encoder and decoder in each layer of the lung nodule model in this embodiment are downsampled according to the hierarchical attention mechanism, and then the pixels output by the encoder and the feature pixels output by the decoder of the corresponding layer are multiplied one by one to highlight the key features. In this embodiment, the specific process of applying the hierarchical attention mechanism to the lung nodule segmentation model is as follows:

[0053] The first layer features of the encoder are downsampled so that the feature size of the first layer of the encoder is consistent with the feature size of the seventh layer, and the first layer feature pixels of the encoder are multiplied one by one with the feature pixels of the seventh layer. The second layer features of the encoder are downsampled so that the feature size of the second layer of the encoder is consistent with the feature size of the sixth layer, and the second layer feature pixels of the encoder are multiplied one by one with the feature pixels of the sixth layer. The third layer features of the encoder are downsampled so that the feature size of the third layer of the encoder is consistent with the feature size of the fifth layer, and the third layer feature pixels of the encoder are multiplied one by one with the feature pixels of the fifth layer. The seventh layer features of the decoder are downsampled so that the feature size of the seventh layer of the decoder is consistent with the feature size of the first layer, and the seventh layer feature pixels of the decoder are multiplied one by one with the feature pixels of the first layer. The sixth layer features of the decoder are downsampled so that the feature size of the sixth layer of the decoder is consistent with the feature size of the second layer, and the sixth layer feature pixels of the decoder are multiplied one by one with the feature pixels of the second layer. The fifth layer features of the decoder are downsampled so that the feature size of the fifth layer of the decoder is consistent with the feature size of the third layer, and the fifth layer feature pixels of the decoder are multiplied one by one with the feature pixels of the third layer. This embodiment introduces a hierarchical attention mechanism into the V-Net network, which can reduce unnecessary features and highlight key features, thereby better capturing the key information of the original image.

[0054] Furthermore, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the implementation process of this embodiment, a lung image segmentation method is provided, such as Figure 2 As shown, the method includes the following steps:

[0055] Further, as Figure 1The specific implementation of the method shown in this embodiment provides a lung image segmentation device, such as Figure 3 As shown, the device includes: a model building module 21, a model training module 22, an image segmentation module 23 and an image processing module 24.

[0056] A model building module 21 can be used to build a lung nodule segmentation model based on a fully convolutional neural network model through densely connected hybrid dilated convolution modules and a hierarchical attention mechanism;

[0057] The model training module 22 can be used to train the lung nodule segmentation model using lung parenchyma image samples as input and lung nodule segmentation images of the lung parenchyma image samples as output to obtain a trained lung nodule segmentation model;

[0058] The image segmentation module 23 can be used to obtain a lung image to be processed, and segment the lung image using a lung parenchyma segmentation algorithm to obtain a lung parenchyma image;

[0059] The image processing module 24 can be used to obtain a lung nodule segmentation result of the lung image based on the lung parenchyma image using a trained lung nodule segmentation model.

[0060] In a specific application scenario, the image segmentation module 23 can be specifically used to binarize the lung image through a binarization algorithm to obtain a binarized lung image; perform connectivity reconstruction on the binarized lung image through a three-dimensional connectivity algorithm to obtain a three-dimensional lung image; perform clustering on the three-dimensional lung image through a mean clustering algorithm to obtain a binarized three-dimensional connected image of the lung parenchyma; perform an expansion operation on the binarized three-dimensional connected image of the lung parenchyma to obtain a three-dimensional lung parenchyma contour; and perform a mask operation on the three-dimensional lung parenchyma contour and the lung image to obtain a lung parenchyma image.

[0061] In a specific application scenario, the image segmentation module 23 can be used to connect the image pixels in the binarized lung image in six neighborhoods in the three-dimensional direction through a three-dimensional connectivity algorithm to obtain multiple connected areas of the binarized lung image; and reconstruct the multiple connected areas of the binarized lung image in a predetermined order to obtain a three-dimensional lung image.

[0062] In a specific application scenario, the image segmentation module 23 can be specifically used to use the distance between the center point of multiple connected areas of the lung three-dimensional image and the center point of the lung three-dimensional image as a clustering feature, and cluster the lung three-dimensional image through the mean clustering algorithm to obtain multiple three-dimensional connected areas of the lung three-dimensional image, wherein the multiple three-dimensional connected areas include binary three-dimensional connected areas and non-binary three-dimensional connected areas; the binary three-dimensional connected areas of the lung three-dimensional image are retained, and the voxel value of the non-binary three-dimensional connected areas of the lung three-dimensional image is set to 1 to obtain a binary three-dimensional connected image of the lung parenchyma.

[0063] In a specific application scenario, the image processing module 24 can be used to input the lung parenchyma image into a trained lung nodule segmentation model to obtain the characteristic information of the lung image; input the characteristic information of the lung image into a nonlinear activation function, binarize the characteristic information of the lung image, and obtain the lung nodule segmentation result of the lung image.

[0064] In a specific application scenario, the model construction module 21 can be used to construct the encoder and decoder of a fully convolutional neural network model, wherein the encoder consists of a densely connected mixed dilated convolution module, a first batch of normalization layers and a first activation layer, and the decoder consists of a densely connected deconvolution module, a second batch of normalization layers and a second activation layer; a hierarchical attention mechanism is introduced between the encoder and the decoder to construct a lung nodule segmentation model, wherein the feature layers of the encoder and decoder under the hierarchical attention mechanism are opposite to each other, and the feature pixels in the two-to-two feature layers are multiplied respectively.

[0065] In a specific application scenario, the densely connected hybrid dilated convolution module includes multiple first three-dimensional convolution layers, and each two first three-dimensional convolution layers of the multiple first three-dimensional convolution layers are connected using a dense structure; the densely connected deconvolution module includes multiple second three-dimensional convolution layers, and each two second three-dimensional convolution layers of the multiple second three-dimensional convolution layers are connected using a dense structure.

[0066] It should be noted that for other corresponding descriptions of the functional units involved in the lung image segmentation device provided in this embodiment, please refer to Figure 1 The corresponding description in will not be repeated here.

[0067] Based on the above Figure 1 The method shown in FIG. 1 is a method for performing the above-mentioned operation. Accordingly, this embodiment further provides a storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned Figure 1 The lung image segmentation method shown.

[0068] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product. The software product to be identified can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0069] Based on the above Figure 1 The method shown, and Figure 7 In order to achieve the above-mentioned purpose, the present embodiment further provides a physical device for lung image segmentation, which can be a personal computer, a server, a smart phone, a tablet computer, a smart watch, or other network devices, etc. The physical device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 The method shown.

[0070] Optionally, the physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface), etc.

[0071] Those skilled in the art will understand that the physical device structure for lung image segmentation provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0072] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the physical device hardware and the software resources to be identified, supporting the execution of the information processing program and other software and / or programs to be identified. The network communication module is used to enable communication between components within the storage medium and with other hardware and software in the physical information processing device.

[0073] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. By applying the technical solution of the present application, firstly, based on the fully convolutional neural network model, a lung nodule segmentation model is constructed through densely connected hybrid dilated convolution modules and hierarchical attention mechanisms. Then, the lung parenchyma image samples are used as input and the manually labeled lung nodule image samples are used as output to train the lung nodule segmentation model, and then the lung image to be processed is segmented to obtain the lung parenchyma image. Finally, the lung image is processed by the trained lung nodule segmentation model to obtain the lung nodule segmentation result. Compared with the prior art, the above method obtains a lot of contextual information in the lung image through the up and down sampling structure and hierarchical attention mechanism of the fully convolutional neural network model, and also selectively amplifies valuable feature channels and suppresses useless feature channels, so that the lung nodule segmentation model can better capture the key feature information of the lung nodules and improve the segmentation accuracy of the lung nodules. In addition, by adding a densely connected hybrid dilated convolution module to the fully convolutional neural network model, the above method can also extract the features of the multi-level receptive field in the lung image, avoid the network grid problem, and further improve the segmentation accuracy of lung nodules.

[0074] Those skilled in the art will understand that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application. Those skilled in the art will understand that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more devices different from the implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0075] The serial numbers of the above application are for descriptive purposes only and do not represent the advantages or disadvantages of the implementation scenarios. The above disclosure only discloses several specific implementation scenarios of the present application, but the present application is not limited thereto. Any changes that can be conceived by those skilled in the art should fall within the scope of protection of the present application.

Claims

1. A lung image segmentation method, characterized in that: The method comprises: Based on a fully convolutional neural network model, a lung nodule segmentation model is constructed through a densely connected hybrid dilated convolution module and a hierarchical attention mechanism, wherein the method for constructing the lung nodule segmentation model includes: constructing an encoder and a decoder of the fully convolutional neural network model, wherein the encoder is composed of a densely connected hybrid dilated convolution module, a first batch of normalization layers and a first activation layer, and the decoder is composed of a densely connected deconvolution module, a second batch of normalization layers and a second activation layer; between the encoder and the decoder, a hierarchical attention mechanism is introduced to construct the lung nodule segmentation model, wherein the feature layers of the encoder and the decoder under the hierarchical attention mechanism are opposite to each other, and the feature pixels in the two-to-two opposite feature layers are multiplied correspondingly; Taking lung parenchyma image samples as input and taking lung nodule segmentation images of the lung parenchyma image samples as output, the lung nodule segmentation model is trained to obtain a trained lung nodule segmentation model; Acquiring a lung image to be processed, and segmenting the lung image using a lung parenchyma segmentation algorithm to obtain a lung parenchyma image; According to the lung parenchyma image, a lung nodule segmentation result of the lung image is obtained by using the trained lung nodule segmentation model.

2. The method according to claim 1, characterized in that The step of segmenting the lung image using a lung parenchyma segmentation algorithm to obtain a lung parenchyma image includes: performing binarization processing on the lung image by a binarization algorithm to obtain a binarized lung image; Performing connectivity reconstruction on the binarized lung image using a three-dimensional connectivity algorithm to obtain a three-dimensional lung image; performing clustering processing on the three-dimensional lung image by using a mean value clustering algorithm to obtain a binarized three-dimensional connected image of the lung parenchyma; performing a dilation operation on the binarized three-dimensional connected image of the lung parenchyma to obtain a three-dimensional lung parenchyma contour; A mask operation is performed on the three-dimensional lung parenchyma contour and the lung image to obtain a lung parenchyma image.

3. The method according to claim 2, characterized in that The method of performing connectivity reconstruction on the binarized lung image using a three-dimensional connectivity algorithm to obtain a three-dimensional lung image includes: Connecting the image pixels in the binarized lung image on six neighborhoods in a three-dimensional direction using a three-dimensional connectivity algorithm to obtain multiple connected regions of the binarized lung image; The multiple connected regions of the binarized lung image are reconstructed in a predetermined order to obtain the three-dimensional lung image.

4. The method according to claim 3, characterized in that The clustering process is performed on the three-dimensional lung image by using a mean value clustering algorithm to obtain a binarized three-dimensional connected image of the lung parenchyma, including: performing clustering processing on the three-dimensional lung image using a mean value clustering algorithm, using distances between center points of multiple connected regions of the three-dimensional lung image and the center point of the three-dimensional lung image as clustering features, to obtain multiple three-dimensional connected regions of the three-dimensional lung image, wherein the multiple three-dimensional connected regions include binary three-dimensional connected regions and non-binary three-dimensional connected regions; The binarized three-dimensional connected region of the three-dimensional lung image is retained, and the voxel value of the non-binarized three-dimensional connected region of the three-dimensional lung image is set to 1 to obtain the binarized three-dimensional connected image of the lung parenchyma.

5. The method according to claim 1, wherein The step of obtaining a lung nodule segmentation result of the lung image by using the trained lung nodule segmentation model according to the lung parenchyma image includes: Inputting the lung parenchyma image into the trained lung nodule segmentation model to obtain feature information of the lung image; The characteristic information of the lung image is input into a nonlinear activation function, and the characteristic information of the lung image is binarized to obtain a lung nodule segmentation result of the lung image.

6. The method according to claim 1, wherein The densely connected hybrid dilated convolution module includes multiple first three-dimensional convolution layers, and each two first three-dimensional convolution layers of the multiple first three-dimensional convolution layers are connected using a dense structure; the densely connected deconvolution module includes multiple second three-dimensional convolution layers, and each two second three-dimensional convolution layers of the multiple second three-dimensional convolution layers are connected using a dense structure.

7. A lung image segmentation device, characterized in that: The device comprises: A model construction module is used to construct a lung nodule segmentation model based on a fully convolutional neural network model through a densely connected hybrid dilated convolution module and a hierarchical attention mechanism, wherein the method for constructing the lung nodule segmentation model includes: constructing an encoder and a decoder of the fully convolutional neural network model, wherein the encoder is composed of a densely connected hybrid dilated convolution module, a first batch of normalization layers and a first activation layer, and the decoder is composed of a densely connected deconvolution module, a second batch of normalization layers and a second activation layer; between the encoder and the decoder, a hierarchical attention mechanism is introduced to construct the lung nodule segmentation model, wherein the feature layers of the encoder and the decoder under the hierarchical attention mechanism are opposite to each other, and the feature pixels in the two-to-two opposite feature layers are multiplied correspondingly; a model training module, configured to train the lung nodule segmentation model using lung parenchyma image samples as input and lung nodule segmentation images of the lung parenchyma image samples as output, thereby obtaining a trained lung nodule segmentation model; An image segmentation module is used to obtain a lung image to be processed, and segment the lung image using a lung parenchyma segmentation algorithm to obtain a lung parenchyma image; The image processing module is used to obtain a lung nodule segmentation result of the lung image based on the lung parenchyma image using the trained lung nodule segmentation model.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images