Automatic segmentation and texture extraction method of abnormal lung regions

Abstract

The present invention provides a method for automatic segmentation and texture extraction of abnormal lung regions, comprising: identifying abnormal lung regions from lung CT images using a generator neural network; generating a segmented image of the abnormal lung region based on the abnormal lung region; calculating the difference between the generated segmented image of the abnormal lung region and the segmented mask image of the actual abnormal lung region using a discriminator neural network to obtain a network loss value; and optimizing and updating the generator neural network and the discriminator neural network based on the loss value to obtain an accurate segmented image of the abnormal lung region. Simultaneously, the operation of the generator neural network and the discriminator neural network enhances the texture details of the abnormal lung region in the accurate segmented image of the abnormal lung region and enables automatic texture extraction.

Description

Technical Field

[0001] The present invention relates to image segmentation technology, and in particular to a method for automatic segmentation and texture extraction of abnormal lung areas. Background Art

[0002] Lung abnormalities include pulmonary nodules, pneumonia, and tuberculosis. Automatic segmentation of abnormal lung regions on CT images can provide important information for assisting in their assessment, classification, and diagnosis. Existing methods for segmenting abnormal lung regions are limited by their variable locations, complex morphological structures, and uneven grayscale.

[0003] Currently, methods for segmenting abnormal lung regions based on CT images struggle to simultaneously achieve automatic image segmentation and texture extraction. Existing segmentation methods using region growing can obtain the contours of abnormal lung regions, but they require determining initial growth seed points and cannot simultaneously capture the texture of the abnormal lung regions. Automatic detection of lung CT signs using neural networks aims to extract a rectangular box surrounding the abnormal lung region, making it difficult to accurately segment the boundaries of the abnormal lung region. Summary of the Invention

[0004] To address the above problems, the present invention uses a generator neural network and a discriminator neural network to work together to generate accurate segmentation images of abnormal lung areas while extracting texture information of abnormal lung areas, thereby providing automatic segmentation and texture extraction results of abnormal lung areas to assist in the diagnosis of lung images.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention provides a method for automatic segmentation and texture extraction of abnormal lung regions, characterized in that the method comprises the following steps:

[0007] S101, identifying an abnormal lung region from a lung CT image using a generator neural network; and generating a segmented image of the abnormal lung region based on the abnormal lung region;

[0008] S102, calculating the difference between the generated segmentation image of the abnormal lung region and the segmentation mask image of the actual abnormal lung region through the discriminator neural network to obtain a network loss value, and optimizing and updating the generator neural network and the discriminator neural network according to the loss value to obtain an accurate segmentation image of the abnormal lung region;

[0009] S103. At the same time, the generator neural network and the discriminator neural network work to enhance the texture details of the abnormal lung area and realize automatic texture extraction on the accurate segmented image of the abnormal lung area.

[0010] Furthermore, the specific implementation process of identifying abnormal lung areas by the generator neural network in step S101 includes:

[0011] The generator neural network is used to take the lung CT image as network input, and the generator neural network is used to automatically identify and generate abnormal lung areas.

[0012] Furthermore, the specific implementation process of generating the segmented image of the abnormal lung area in step S101 includes:

[0013] The abnormal lung region is automatically identified and generated as the output of the generator neural network, that is, a segmented image of the abnormal lung region is generated.

[0014] Furthermore, the specific implementation process of step S102 includes:

[0015] The segmented image of the generated lung abnormality area is used as the input image and sent to the discriminator neural network. At the same time, the segmented mask image of the real lung abnormality area is sent to the discriminator neural network to calculate the image difference to obtain the network loss, perform network optimization and update, and obtain an accurate segmented image of the lung abnormality area.

[0016] Furthermore, the specific implementation process of step S102 includes:

[0017] The loss D_loss of the discriminator neural network is calculated according to the following formula:

[0018]

[0019] in, represents the loss of the segmentation mask image x of the true lung abnormality area fed into the discriminator neural network, Represents the loss of the segmented image G(x) of the generated lung abnormality area sent to the discriminator neural network.

[0020] Furthermore, the specific implementation process of step S102 includes:

[0021] The loss G_loss of the generator neural network is calculated according to the following formula:

[0022]

[0023] in, represents the loss of the generated segmentation image G(x) of the abnormal lung region fed into the discriminator neural network. loss(G(x),x) represents the auxiliary loss between the generated segmentation image G(x) of the abnormal lung region and the segmentation mask image x of the actual abnormal lung region.

[0024] Furthermore, the specific implementation process of step S103 includes:

[0025] Based on the work of the generator neural network and the discriminator neural network, the texture information of the abnormal lung region is enhanced on the accurate segmented image of the abnormal lung region, and automatic texture extraction is achieved on the accurate segmented image of the abnormal lung region.

[0026] Compared with the prior art, the present invention has the following beneficial effects.

[0027] The present invention does not need to set an initial growth seed point for automatic segmentation of abnormal lung areas, thus avoiding the human-computer interaction defect in the region growing method.

[0028] The neural network used in the present invention is an automatic segmentation method based on the collaborative work of a generator neural network and a discriminator neural network. It can directly obtain the boundaries of abnormal lung areas on lung CT images. Compared with the bounding box extraction results of existing methods, the results of this method can more accurately reflect the contour structure of abnormal lung areas.

[0029] The texture extraction method for abnormal lung regions provided by this invention is a method that enhances and extracts texture details in abnormal lung regions while achieving automatic segmentation. Its advantage is that only a single network design is required to achieve both the segmentation and texture extraction goals of the current lung CT image. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Flowchart of the automatic segmentation and texture extraction method for abnormal lung areas.

[0031] Figure 2 Schematic diagram of ground-glass opacity type lung abnormality area segmentation and texture extraction.

[0032] Figure 3 Schematic diagram of the segmentation and texture extraction of abnormal lung areas adhered to blood vessels.

[0033] Figure 4 Schematic diagram of the segmentation and texture extraction of abnormal lung areas adhered to structures such as the pleura.

[0034] Figure 5 Schematic diagram of solid lung abnormality area segmentation and texture extraction.

[0035] Figure 6 Schematic diagram of texture extraction of the generated segmented image of the abnormal lung area and the texture of the abnormal lung area on the original CT image. DETAILED DESCRIPTION

[0036] The present invention is described in detail below with reference to specific embodiments.

[0037] The overall concept of the present invention is to generate a segmentation image of the abnormal lung area on the input lung CT image through a generator neural network, analyze the image difference between the generated segmentation image of the abnormal lung area and the real segmentation mask image of the abnormal lung area provided by the expert through a discriminator neural network as the network loss, and optimize and update the generator neural network and the discriminator neural network based on the network loss to accurately obtain the segmentation image and texture extraction results of the abnormal lung area.

[0038] Example 1. Figure 1 The automatic segmentation and texture extraction methods of abnormal lung areas are described in detail.

[0039] Reference Figure 1 In step S101, an abnormal lung region is identified from a lung CT image through a generator neural network; and a segmented image of the abnormal lung region is generated based on the abnormal lung region.

[0040] Here, lung CT images serve as input to the generator neural network.

[0041] The specific process of step S101 is as follows: The generator neural network is composed of a convolutional layer, an image normalization layer, and an activation layer, which are connected in series to form an image decoder. The deconvolutional layer, an image normalization layer, and an activation layer are connected in series to form an image decoder. The image decoder and image decoder together form the generator neural network.

[0042] The lung CT image is first input into the image decoder, and the abnormal lung area is identified through each operation unit of the decoder in sequence, and then the segmented image of the abnormal lung area is generated through each operation unit of the image decoder in sequence.

[0043] In step S102, the discriminator neural network is constructed by sequentially connecting operation units consisting of convolutional layers and activation layers.

[0044] In this step, the generated segmented image of the lung abnormality region is input into the discriminator neural network, and the real segmentation mask image of the lung abnormality region provided by the expert corresponding to each lung CT image is also input into the discriminator neural network. The discriminator neural network calculates the difference between the network output results of the above two input images, and the loss D_loss of the discriminator neural network is obtained by the following formula:

[0045]

[0046] in, represents the loss of the segmentation mask image x of the true lung abnormality area fed into the discriminator neural network, The loss of the discriminator neural network is represented by the generated segmented image G(x) of the abnormal lung region. The difference between the two is the loss of the discriminator neural network.

[0047] At the same time, the loss G_loss of the generator neural network is calculated according to the following formula:

[0048]

[0049] in, represents the loss of the generated segmentation image G(x) of the abnormal lung region fed into the discriminator neural network. loss(G(x),x) represents the auxiliary loss between the generated segmentation image G(x) of the abnormal lung region and the segmentation mask image x of the true abnormal lung region:

[0050] loss(G(x))=MSE(G(x),x)+VGG(G(x),x)

[0051] Here, MSE(G(x),x) is the mean square error of G(x) and x, and VGG(G(x),x) is the image loss of G(x) and x through a VGG network.

[0052] The generator neural network and the discriminator neural network are optimized and updated respectively according to the loss, so as to accurately obtain the segmentation image and image texture of the abnormal lung area.

[0053] In step S103, based on the accurate segmented image of the abnormal lung region, the segmented image only includes the abnormal lung region and excludes the non-abnormal lung region and background. The texture of the abnormal lung region in the accurate segmented image of the abnormal lung region is enhanced by the operation of the generator neural network and the discriminator neural network, and texture extraction is achieved on the accurate segmented image of the abnormal lung region. Therefore, this method obtains a segmentation result and a texture extraction result for the abnormal lung region.

[0054] Example 2. Examples of specific parameters of the method for automatic segmentation and texture extraction of abnormal lung areas.

[0055] The lung CT image is first input into the generator neural network's image decoder, where it passes through each of the decoder's operational units to identify abnormal lung areas. The generator neural network's decoder consists of four operational units, each consisting of an image convolution layer, an image normalization layer, and an activation layer, connected in series. The image convolution layer filters are 32×3×3, 64×3×3, 128×3×3, and 256×3×3, respectively. The activation layer uses the LeakyReLU function.

[0056] After passing through the image decoder, the lung CT image is sequentially passed through the various operating units of the image decoder to generate a segmented image of the abnormal lung area. The decoder part of the generator neural network is composed of four operating units consisting of an image convolution layer, an image normalization layer, and an activation layer, which are connected in series. The number of convolution layer filters is 128×3×3, 64×3×3, 32×3×3, and 1×3×3, respectively. The output results of the first three operating units of the encoder are summed pixel by pixel with the output results of the image convolution layer of the first three operating units of the corresponding decoder. The summed result is then sent to the image normalization layer of the decoder operating unit for subsequent operations.

[0057] Then, the generated segmentation image of the abnormal lung area is input into the discriminator neural network, and the segmentation mask image of the actual abnormal lung area provided by the expert corresponding to each lung CT image is input into the discriminator neural network. The discriminator neural network is composed of three operation units consisting of image convolution layers and activation layers connected in series. The number of filters in the convolution layer is 32×3×3, 64×3×3 and 128×3×3 respectively. The activation layer is composed of LeakyReLU function. Finally, a fully connected layer is used to obtain the loss value of the segmentation image of the abnormal lung area and the segmentation mask image of the actual abnormal lung area after passing through the discriminator neural network.

[0058] The network output results of the above two images are obtained according to the discriminator neural network. The loss D_loss of the discriminator neural network is obtained by the following formula:

[0059]

[0060] in, represents the loss of the segmentation mask image x of the true lung abnormality area fed into the discriminator neural network, The loss of the discriminator neural network is represented by the generated segmented image G(x) of the abnormal lung region. The difference between the two is the loss of the discriminator neural network.

[0061] At the same time, the loss G_loss of the generator neural network is calculated according to the following formula:

[0062]

[0063] in, represents the loss of the generated segmentation image G(x) of the abnormal lung region fed into the discriminator neural network. loss(G(x),x) represents the auxiliary loss between the generated segmentation image G(x) of the abnormal lung region and the segmentation mask image x of the true abnormal lung region:

[0064] loss(G(x))=MSE(G(x),x)+VGG(G(x),x)

[0065] Here, MSE(G(x),x) is the mean square error of G(x) and x, and VGG(G(x),x) is the image loss of G(x) and x through a VGG network.

[0066] The generator neural network and the discriminator neural network are optimized and updated respectively according to the loss, so as to accurately obtain the segmentation image and image texture of the abnormal lung area.

[0067] According to the accurate segmented image of the abnormal lung area, the segmented image only includes the abnormal lung area and does not include the non-abnormal lung area and the background.

[0068] Example 3. Schematic diagram of automatic segmentation and texture extraction of abnormal lung regions of three ground-glass opacity types using the parameters and process of Example 2 of the present invention.

[0069] Reference Figure 2 , (a), (b), and (c) are three ground-glass opacity-type abnormal lung areas, Figure 2 The first column is the original lung CT image; the second column is the segmentation contour of the actual lung abnormality area provided by the expert; the third column is the segmentation image of the lung abnormality area generated by this method; the fourth column is the texture extraction result of the segmentation image of the lung abnormality area obtained by this method.

[0070] Depend on Figure 2 It can be seen that for ground-glass opacity-type lung abnormalities, this method can achieve segmentation results close to the true segmentation contours provided by experts, and the extracted texture information can provide rich textures of the lung abnormalities.

[0071] Example 4. Schematic diagram of automatic segmentation and texture extraction of abnormal lung regions with vascular adhesions using the parameters and process analysis of Example 2 of the present invention.

[0072] Reference Figure 3 , (a), (b), and (c) are three abnormal lung areas adhered to blood vessels. Figure 3 The first column is the original lung CT image; the second column is the segmentation contour of the actual lung abnormality area provided by the expert; the third column is the segmentation image of the lung abnormality area generated by this method; the fourth column is the texture extraction result of the segmentation image of the lung abnormality area obtained by this method.

[0073] Depend on Figure 3 It can be seen that for abnormal lung areas adhered to blood vessels, this method can achieve segmentation results close to the true segmentation contours provided by experts, and the extracted texture information can provide rich textures of abnormal lung areas.

[0074] Example 5. Schematic diagram of automatic segmentation and texture extraction of abnormal lung regions adhered to tissues such as the pleura using the parameters and process of Example 2 of the present invention.

[0075] Reference Figure 4 , (a), (b), and (c) are three abnormal lung areas that are adhered to structures such as the pleura. Figure 4 The first column is the original lung CT image; the second column is the segmentation contour of the actual lung abnormality area provided by the expert; the third column is the segmentation image of the lung abnormality area generated by this method; and the fourth column is the texture extraction result of the lung abnormality area obtained by this method.

[0076] Depend on Figure 4 It can be seen that for abnormal lung areas that are adhered to structures such as the pleura, this method can achieve segmentation results close to the true segmentation contours provided by experts, and the extracted texture information can provide rich textures of abnormal lung areas.

[0077] Example 6. Schematic diagram of automatic segmentation and texture extraction of solid abnormal lung regions using the parameters and process of Example 2 of the present invention.

[0078] Reference Figure 5 , (a), (b), and (c) are three solid abnormal lung areas, Figure 4 The first column is the original lung CT image; the second column is the segmentation contour of the actual lung abnormality area provided by the expert; the third column is the segmentation image of the lung abnormality area generated by this method; and the fourth column is the texture extraction result of the lung abnormality area obtained by this method.

[0079] Depend on Figure 5 It can be seen that for solid abnormal lung areas, this method can achieve segmentation results close to the true segmentation contours provided by experts, and the extracted texture information can provide rich texture of abnormal lung areas.

[0080] Example 7. Schematic diagram of the segmented image of the abnormal lung area generated by analyzing the parameters and process of Example 2 of the present invention and the texture results of the abnormal lung area on the original CT image.

[0081] Reference Figure 6 ,The first to fourth columns are four different images of abnormal lung regions, (a) represents the texture of the abnormal lung region on the original CT image, and (b) represents the texture extracted from the segmented image of the abnormal lung region obtained by this method.

[0082] Depend on Figure 6It can be seen that the segmented image obtained by the proposed lung abnormal region segmentation method can extract richer image textures in the lung abnormal region than in the original CT image. Therefore, it can be seen that the method adopted by the present invention has high practical value.

[0083] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for automatic segmentation and texture extraction of abnormal lung regions, characterized in that: The method comprises the following steps: S101, identifying an abnormal lung region from a lung CT image using a generator neural network; and generating a segmented image of the abnormal lung region based on the abnormal lung region; The specific implementation process of identifying the abnormal lung region by the generator neural network in step S101 includes: using the lung CT image as a network input according to the generator neural network, and automatically identifying and generating the abnormal lung region according to the generator neural network; The specific implementation process of generating the segmented image of the abnormal lung region in step S101 includes: automatically identifying and generating the abnormal lung region as the output of the generator neural network, that is, generating the segmented image of the abnormal lung region; In step S101, the generator neural network is composed of an image encoder and an image decoder; the image encoder is composed of an operation unit consisting of a convolution layer, an image normalization layer, and an activation layer connected in series; the image decoder is composed of an operation unit consisting of a deconvolution layer, an image normalization layer, and an activation layer connected in series; In step S101, the generator neural network directly generates an image of the abnormal lung area; S102, calculating the difference between the generated segmentation image of the abnormal lung region and the segmentation mask image of the actual abnormal lung region through the discriminator neural network to obtain a network loss value, and optimizing and updating the generator neural network and the discriminator neural network according to the loss value to obtain an accurate segmentation image of the abnormal lung region; The segmented image of the abnormal lung area is generated as an input image and fed into the discriminator neural network. At the same time, the segmented mask image of the actual abnormal lung area is fed into the discriminator neural network to calculate the image difference to obtain the network loss, perform network optimization and update, and obtain an accurate segmented image of the abnormal lung area. The encoder of the generator neural network is composed of four operation units consisting of an image convolution layer, an image normalization layer, and an activation layer connected in series. The image convolution layer filters are 32×3×3, 64×3×3, 128×3×3, and 256×3×3, respectively. The activation layer is composed of the LeakyReLU function. The decoder part of the generator neural network consists of four operation units consisting of an image deconvolution layer, an image normalization layer, and an activation layer connected in series, where the number of convolution layer filters is 128×3×3, 64×3×3, 32×3×3, and 1×3×3, respectively; the output results of the first three operation units of the encoder are summed pixel by pixel with the output results of the image convolution layer of the first three operation units of the corresponding decoder, and the summed result is then sent to the image normalization layer of the decoder operation unit for subsequent operation; The discriminator neural network is composed of three operation units consisting of image convolution layers and activation layers connected in series: the number of filters in the convolution layer is 32×3×3, 64×3×3, and 128×3×3, respectively; the activation layer is composed of the LeakyReLU function, and finally a fully connected layer is used to obtain the loss value of the segmentation image of the abnormal lung area and the segmentation mask image of the actual abnormal lung area after passing through the discriminator neural network; The loss D_loss of the discriminator neural network is calculated according to the following formula: in, represents the loss of the segmentation mask image x of the true lung abnormality area fed into the discriminator neural network, represents the loss of the segmented image G(x) of the generated lung abnormality area being fed into the discriminator neural network; The loss G_loss of the generator neural network is calculated according to the following formula: in, represents the loss of the generated segmentation image G(x) of the abnormal lung area fed into the discriminator neural network; loss(G(x),x) represents the auxiliary loss of the generated segmentation image G(x) of the abnormal lung area and the segmentation mask image x of the real abnormal lung area; S103. Simultaneously, the generator neural network and the discriminator neural network work to enhance the texture details of the abnormal lung region and achieve automatic texture extraction on the accurate segmented image of the abnormal lung region. According to the accurate segmented image of the abnormal lung area, based on the work of the generator neural network and the discriminator neural network, the texture details of the abnormal lung area are enhanced, and automatic texture extraction is achieved on the accurate segmented image of the abnormal lung area; In step S103, the texture details of the abnormal lung area are enhanced, and the texture features of the abnormal lung area are generated based on the collaborative work of the generator neural network in S101 and the discriminator neural network in S102.

Images