A lung image segmentation method and system
By enhancing lung CT images and combining the DPA attention module and the skip connections of dense Swing Transformer blocks, the U-shaped symmetric network segmentation model 3D-SDUnet is optimized, solving the problem of poor performance of traditional segmentation algorithms and achieving high-precision segmentation of lung images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI DROIDSURG MEDICAL CO LTD
- Filing Date
- 2024-03-26
- Publication Date
- 2026-06-23
AI Technical Summary
Traditional lung image segmentation algorithms suffer from problems such as excessive noise interference, unclear segmentation results, and low segmentation accuracy in lung CT images. Furthermore, deep learning segmentation models have not yet been fully optimized for lung image segmentation.
The Lap-CLAHE algorithm is used to enhance lung CT images, and the images are segmented using the U-shaped symmetric network segmentation model 3D-SDUnet. By combining the DPA attention module and the skip connections of the dense Swing Transformer block, the information transmission between the encoder and decoder is enhanced.
It improves the precision and accuracy of lung image segmentation, accurately segmenting the lung contour and enhancing image detail and contrast.
Smart Images
Figure CN118115742B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of image segmentation, and more particularly to a method and system for lung image segmentation. Background Technology
[0002] Lung diseases are common and highly contagious, posing an immeasurable threat to human health. How to quickly and accurately treat lung diseases is a crucial area of ongoing research in the medical field. The structure of the human lung is extremely complex and irregular. Currently, lung diseases are diagnosed through lung CT scans. Determining the type of lung lesion by observing CT images requires careful interpretation by experienced physicians. Lung CT images typically contain numerous blood vessels, trachea, and other complex tissues, which complicates clinical diagnosis. Blood vessels near lung lesions are generally very small and not clearly visible on CT imaging, severely impacting the diagnosis of lung lesions and the accuracy of subsequent segmentation.
[0003] Lung segmentation can be divided into traditional algorithm segmentation and deep learning segmentation. Traditional algorithms, such as threshold-based methods, edge detection-based methods, or wavelet transform methods, often fail to perform well due to the complex shape, rich data details, and uneven brightness variations in lung CT images. Traditional lung image segmentation is mostly based on conventional methods, but the unique characteristics and uneven brightness of lung images impose many limitations on these algorithms, leaving significant room for improvement in segmentation accuracy. For example, traditional algorithms often suffer from noise interference, blurry and unclear lung CT images, resulting in poor segmentation quality and low accuracy.
[0004] Deep learning segmentation offers significant improvements over traditional algorithms and is widely used in medical image segmentation. With continuous updates and iterations in deep learning, more and more researchers are incorporating it into medical image segmentation, using it to achieve intelligent and refined segmentation of lung images. This helps doctors achieve rapid diagnosis and pathological analysis, and is becoming a trend in the future development of computer-assisted medical systems. Summary of the Invention
[0005] To address the aforementioned problems, the present invention aims to provide a lung image segmentation method and system. First, the lung image is enhanced to improve detail while maintaining good image contrast, providing high-quality input data for subsequent model segmentation. The segmentation model is then optimized by incorporating skip connections composed of dense Swing Transformer blocks (DATB) and a DPA attention module to strengthen the information connection between the encoder and decoder, improve segmentation accuracy, and obtain a complete lung model.
[0006] The above-mentioned objective of this invention is achieved through the following technical solutions:
[0007] A lung image segmentation method includes the following steps:
[0008] S1: Input the DICOM sequence of the lung CT scan, and convert the original DICOM sequence into a JPG format image as the original lung image;
[0009] S2: The original lung image is enhanced using the Lap-CLAHE image enhancement algorithm to obtain an enhanced image;
[0010] S3: Input the enhanced image into the U-shaped symmetric network segmentation model 3D-SDUnet, which consists of an encoder, a decoder, DPA dual-path attention, and skip connections, and output the final segmentation result of the lung image.
[0011] Further, in step S1, the DICOM sequence of the lung CT scan is input, and the original DICOM sequence is converted into a JPG format image as the original lung image, specifically:
[0012] Each layer in the DICOM sequence is processed one by one, and the DICOM sequence of each layer is converted into a JPG format image.
[0013] The converted original lung images corresponding to each layer of the DICOM sequence are the same size as the original DICOM sequence, and the number of original lung images is the same as the number of layers in the DICOM sequence.
[0014] Further, in step S2, the original lung image is enhanced using the Lap-CLAHE image enhancement algorithm to obtain the enhanced image, specifically as follows:
[0015] S21: Input the original lung image of each layer, decompose the original lung image through Gaussian pyramid and Laplacian pyramid to obtain a decomposed image, and reconstruct the image through Laplacian pyramid to obtain a reconstructed image.
[0016] S22: Perform image fusion on the reconstructed image obtained from each layer of reconstruction to obtain the final reconstructed image, and use the CLAHE algorithm to uniformly distribute the gray values of the final reconstructed image to obtain the enhanced image.
[0017] Further, in step S21, the original lung image of each layer is input, and the original lung image is decomposed using the Gaussian pyramid and the Laplacian pyramid to obtain the decomposed image. The decomposed image is then reconstructed using the Laplacian pyramid to obtain the reconstructed image. Specifically:
[0018] Assuming the original lung image is lct, and the bottom layer GO of the Gaussian pyramid is GO = lct, the first layer G1 of the Gaussian pyramid is obtained through low-pass filtering and downsampling. This process is repeated N times to obtain the Nth layer Gn of the Gaussian pyramid. The expression for each layer of the Gaussian pyramid is as follows:
[0019]
[0020] Among them, C l and R l Let m and n represent the number of rows and columns of the l-th layer image of the Gaussian pyramid, respectively, and ω(m, n) represent a low-pass filter of size 5*5.
[0021] By upsampling layer by layer the Gaussian pyramid, a pyramid image G' with the same size as the previous layer is obtained. l G l and G' l The difference is used to obtain the Laplacian pyramid L. l The expression is as follows:
[0022]
[0023] Among them, when and When it is an integer, when and When not an integer,
[0024] so: The decomposed image L is obtained n ;
[0025] Image reconstruction of the decomposed image can be specifically represented as follows:
[0026] G l =L l +G' l
[0027] Obtain the reconstructed image G of layer l. l .
[0028] Further, in step S22, image fusion is performed on the reconstructed images obtained from each layer of reconstruction to obtain the final reconstructed image. The enhanced image is then obtained by uniformly distributing the grayscale values of the final reconstructed image using the CLAHE algorithm. Specifically:
[0029] Reconstruct image G from each layer l The final reconstructed image G is obtained by fusing images after unifying their sizes. N Specifically, it can be expressed as:
[0030] G N =G1+G2+...+G l
[0031] The enhanced image is obtained by uniformly distributing the gray values of the image using the CLAHE algorithm on the final reconstructed image.
[0032] Further, in step S3, the enhanced image is input into the U-shaped symmetric network segmentation model 3D-SDUnet, which consists of the encoder, the decoder, the DPA dual-path attention module, and the skip connections, and the final segmentation result of the lung image is output, specifically as follows:
[0033] The depth of the U-shaped symmetric network segmentation model 3D-SDUnet was changed from 3 upsampling and downsampling operations to 4 upsampling operations to extract more image features;
[0034] The DPA dual-path attention module is added to the encoder in each layer to improve feature extraction capability;
[0035] A skip connection consisting of dense Swing Transformer blocks is added between the encoder and the decoder in each layer to enhance information transmission between the encoder and the decoder.
[0036] The DPA dual-path attention module specifically consists of a first path and a second path. The first path is composed of global average pooling (GAP), fully connected FC, and ReLU activation function to acquire information between channels. The second path is composed of fully connected FC and ReLU activation function to focus on the global spatial information of the lung image. The feature maps of the first path and the second path are fused by element-wise addition, and then weighted coefficients are obtained by sigmoid activation function. The weighted coefficients are then multiplied by element and added to the original feature map so that the network can simultaneously focus on the channel information and spatial feature information of the image.
[0037] The skip connections formed by the dense Swing Transformer blocks are specifically as follows: the dense Swing Transformer blocks are used between the encoder and the decoder in each layer. Each dense Swing Transformer block consists of multiple deep learning model Swing Transformers, connection concats, and convolutions. Each deep learning model Swing Transformer consists of a LayerNorm layer, a multi-head self-attention module, residual connections, and two MLP layers. Between every two Transformers, a window-based multi-head self-attention module W-MSA and a displacement-based window-based multi-head self-attention module SW-MSA are used.
[0038] A lung image segmentation system for performing the lung image segmentation method as described above, comprising:
[0039] The input module is used to input the DICOM sequence of lung CT and convert the original DICOM sequence into a JPG format image as the original lung image;
[0040] The enhancement module is used to enhance the original lung image using the Lap-CLAHE image enhancement algorithm to obtain an enhanced image;
[0041] The segmentation module is used to input the enhanced image into the U-shaped symmetric network segmentation model 3D-SDUnet, which consists of an encoder, a decoder, DPA dual-path attention, and skip connections, and output the final segmentation result of the lung image.
[0042] A computer device includes a memory and one or more processors, the memory storing computer code that, when executed by the one or more processors, causes the one or more processors to perform the method described above.
[0043] A computer-readable storage medium storing computer code that, when executed, performs the method described above.
[0044] Compared with the prior art, the beneficial effects of the present invention are:
[0045] This paper proposes a lung image segmentation method, comprising: S1: inputting a DICOM sequence from a lung CT scan and converting the original DICOM sequence into a JPG format image as the original lung image; S2: enhancing the original lung image using the Lap-CLAHE image enhancement algorithm to obtain an enhanced image; S3: inputting the enhanced image into a U-shaped symmetric network segmentation model 3D-SDUnet, composed of an encoder, decoder, DPA dual-path attention, and skip connections, and outputting the final segmentation result of the lung image. The above technical solution can maintain good contrast while improving image detail information during image enhancement, providing high-quality data for subsequent segmentation. A segmentation model based on 3D-SDUnet is proposed, which adds a DPA attention module to each layer of the network encoder, simultaneously focusing on channel information and global information, and adds skip connections composed of dense Swing Transformer blocks (DATB), which can strengthen the information connection between each layer of encoder and decoder, and can accurately segment the lung contour. Attached Figure Description
[0046] Figure 1 This is an overall flowchart of the lung image segmentation method of the present invention;
[0047] Figure 2 This is a flowchart of the Lap-CLAHE enhancement algorithm of the present invention;
[0048] Figure 3 This is a structural diagram of the 3D-SDUnet model of the present invention;
[0049] Figure 4 This is a structural diagram of the DPA attention module of the present invention;
[0050] Figure 5 This is a structural diagram of the Dense Swing Transformer Block (DATB) of the present invention;
[0051] Figure 6 This is a structural diagram of the Swin Transformer (ST) of the present invention;
[0052] Figure 7 This is an overall structural diagram of the lung image segmentation system of the present invention. Detailed Implementation
[0053] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0054] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.
[0055] First Embodiment
[0056] like Figure 1 As shown, this embodiment provides a lung image segmentation method, including the following steps:
[0057] S1: Input the DICOM sequence of the lung CT scan and convert the original DICOM sequence into a JPG format image as the original lung image.
[0058] Specifically, in this embodiment, each layer in the DICOM sequence needs to be processed one by one, and the DICOM sequence of each layer is converted into a JPG format image. The original lung image corresponding to each layer of the converted DICOM sequence is the same size as the original DICOM sequence, and the number of the original lung images is the same as the number of layers in the DICOM sequence.
[0059] Furthermore, in this embodiment, the DICOM sequence can also be replaced by other types of data, such as NII, MHD, and other formats.
[0060] S2: The original lung image is enhanced using the Lap-CLAHE image enhancement algorithm to obtain an enhanced image.
[0061] Laplacian Contrast Limited Adaptive Histogram Equalization (Lap-CLAHE) is an image enhancement algorithm that combines Laplacian pyramid and adaptive histogram equalization (CLAHE). Its main purpose is to improve image contrast and detail, and it is frequently used, particularly in medical image processing. The specific execution steps in this embodiment are as follows:
[0062] S21: Input the original lung image of each layer, decompose the original lung image through Gaussian pyramid and Laplacian pyramid to obtain a decomposed image, and reconstruct the image through Laplacian pyramid to obtain a reconstructed image.
[0063] Specifically, assuming the original lung image is lct, and the bottom layer GO of the Gaussian pyramid is GO = lct, the first layer G1 of the Gaussian pyramid is obtained through low-pass filtering and downsampling, and so on, repeating N times to obtain the Nth layer Gn of the Gaussian pyramid. The expression for each layer of the Gaussian pyramid is as follows:
[0064]
[0065] Among them, C l and R l Let m and n represent the number of rows and columns of the l-th layer image of the Gaussian pyramid, respectively, and ω(m, n) represent a low-pass filter of size 5*5.
[0066] The Laplacian pyramid and the Gaussian pyramid work together, and the construction of the Gaussian pyramid is the reverse process. Specifically, the Gaussian pyramid is upsampled layer by layer to obtain a pyramid image G' of the same size as the previous layer. l G l and G' l The difference is used to obtain the Laplacian pyramid L. l The expression is as follows:
[0067]
[0068] Among them, when and When it is an integer, when and When not an integer,
[0069] so: The decomposed image L is obtained n ;
[0070] Image reconstruction of the decomposed image can be specifically represented as follows:
[0071] G l =L l +G' l
[0072] Obtain the reconstructed image G of layer l. l .
[0073] S22: After unifying the size of the reconstructed images obtained from each layer, image fusion is performed to obtain the final reconstructed image. The CLAHE algorithm is then used to uniformly distribute the gray values of the final reconstructed image to obtain the enhanced image.
[0074] Reconstruct image G from each layer l The final reconstructed image G is obtained by fusing images after unifying their sizes.N Specifically, it can be expressed as:
[0075] G N =G1+G2+...+G l
[0076] The enhanced image is obtained by uniformly distributing the gray values of the image using the CLAHE algorithm on the final reconstructed image.
[0077] The reconstructed image has rich detail, but it lacks contrast and grayscale distribution. This paper proposes the Lap-CLAHE enhancement method by combining the Laplacian pyramid and the CLAHE algorithm. The complete Lap-CLAHE enhancement method flowchart is as follows: Figure 2 As shown.
[0078] S3: Input the enhanced image into the 3D-SDUnet U-shaped symmetric network segmentation model, which consists of an encoder, decoder, DPA dual-path attention, and skip connections, and output the final segmentation result of the lung image. This method enhances the information transmission between the encoder and decoder, reduces information loss, improves segmentation accuracy, and obtains a precise segmented image.
[0079] Specifically, in this embodiment, the depth of the U-shaped symmetric network segmentation model 3D-SDUnet is changed from 3 upsampling / downsampling operations to 4 upsampling operations to extract more image features; the DPA dual-path attention module is added to the encoder of each layer to improve feature extraction capability; and skip connections composed of dense Swing Transformer blocks (DATB) are added between the encoder and decoder of each layer to strengthen information transmission between the encoder and decoder. The model structure is as follows: Figure 3 As shown.
[0080] like Figure 4 As shown, the DPA dual-path attention module specifically consists of a first path and a second path. The first path is composed of global average pooling (GAP), fully connected FC, and ReLU activation functions to acquire information between channels. The second path is composed of fully connected FC and ReLU activation functions to focus on the global spatial information of the lung image. The feature maps of the first path and the second path are fused by element-wise addition, and then weighted coefficients are obtained by using the sigmoid activation function. The weighted coefficients are then multiplied element-wise and applied to the original feature map so that the network simultaneously focuses on the channel information and spatial feature information of the image.
[0081] like Figure 5As shown, the skip connections composed of the dense Swing Transformer blocks specifically involve using dense Swing Transformer blocks between the encoder and decoder in each layer. Each dense Swing Transformer block (DSTB) consists of multiple deep learning model Swing Transformers (ST), connections, concatenation, and convolutions, as shown below. Figure 6 As shown, each of the deep learning models, Swin Transfomer (ST), consists of a LayerNorm (LN) layer, a multi-head self-attention module, residual connections, and two MLP layers. Between every two Transfomers, a window-based multi-head self-attention module W-MSA and a displacement window-based multi-head self-attention module SW-MSA are used.
[0082] Second Embodiment
[0083] like Figure 7 As shown, this embodiment provides a lung image segmentation system for performing the lung image segmentation method as described in the first embodiment, characterized in that it includes:
[0084] Input module 1 is used to input the DICOM sequence of lung CT and convert the original DICOM sequence into a JPG format image as the original lung image;
[0085] Enhancement module 2 is used to enhance the original lung image using the Lap-CLAHE image enhancement algorithm to obtain an enhanced image;
[0086] The segmentation module 3 is used to input the enhanced image into the U-shaped symmetric network segmentation model 3D-SDUnet, which consists of an encoder, a decoder, DPA dual-path attention, and skip connections, and output the final segmentation result of the lung image.
[0087] A computer-readable storage medium stores computer code that, when executed, performs the methods described above. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.
[0088] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.
[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0090] It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A lung image segmentation method, characterized in that, Includes the following steps: S1: Input the DICOM sequence of the lung CT scan, and convert the original DICOM sequence into a JPG format image as the original lung image; S2: The original lung image is enhanced using the Lap-CLAHE image enhancement algorithm to obtain an enhanced image; S3: Input the enhanced image into the U-shaped symmetric network segmentation model 3D-SDUnet, which consists of an encoder, a decoder, DPA dual-path attention, and skip connections, and output the final segmentation result of the lung image; In step S3, the enhanced image is input into the U-shaped symmetric network segmentation model 3D-SDUnet, which consists of the encoder, the decoder, the DPA dual-path attention module, and the skip connections, and the final segmentation result of the lung image is output, specifically: The depth of the U-shaped symmetric network segmentation model 3D-SDUnet was changed from 3 upsampling and downsampling operations to 4 upsampling operations to extract more image features; The DPA dual-path attention module is added to the encoder in each layer to improve feature extraction capability; A skip connection consisting of dense Swing Transformer blocks is added between the encoder and the decoder in each layer to enhance the information transmission between the encoder and the decoder; The DPA dual-path attention module specifically consists of a first path and a second path. The first path is composed of global average pooling (GAP), fully connected FC, and ReLU activation function to acquire information between channels. The second path is composed of fully connected FC and ReLU activation function to focus on the global spatial information of the lung image. The feature maps of the first path and the second path are fused by element-wise addition, and then weighted coefficients are obtained by sigmoid activation function. The weighted coefficients are then multiplied by element and added to the original feature map so that the network can simultaneously focus on the channel information and spatial feature information of the image. The skip connections composed of the dense Swing Transformer blocks are specifically as follows: the dense Swing Transformer blocks are used between the encoder and the decoder in each layer. Each dense Swing Transformer block consists of multiple deep learning model Swing Transformers, connection concats, and convolutions. Each deep learning model Swing Transformer consists of a LayerNorm layer, a multi-head self-attention module, residual connections, and two MLP layers. Between every two Transformers, a window-based multi-head self-attention module W-MSA and a displacement-based window-based multi-head self-attention module SW-MSA are used.
2. The lung image segmentation method according to claim 1, characterized in that, In step S1, the DICOM sequence of the lung CT scan is input, and the original DICOM sequence is converted into a JPG format image as the original lung image, specifically: Each layer in the DICOM sequence is processed one by one, and the DICOM sequence of each layer is converted into a JPG format image. The converted original lung images corresponding to each layer of the DICOM sequence are the same size as the original DICOM sequence, and the number of original lung images is the same as the number of layers in the DICOM sequence.
3. The lung image segmentation method according to claim 1, characterized in that, In step S2, the original lung image is enhanced using the Lap-CLAHE image enhancement algorithm to obtain the enhanced image, specifically as follows: S21: Input the original lung image of each layer, decompose the original lung image through Gaussian pyramid and Laplacian pyramid to obtain a decomposed image, and reconstruct the image through Laplacian pyramid to obtain a reconstructed image. S22: Perform image fusion on the reconstructed image obtained from each layer of reconstruction to obtain the final reconstructed image, and use the CLAHE algorithm to uniformly distribute the gray values of the final reconstructed image to obtain the enhanced image.
4. The lung image segmentation method according to claim 3, characterized in that, In step S21, the original lung image of each layer is input, and the original lung image is decomposed using the Gaussian pyramid and the Laplacian pyramid to obtain the decomposed image. The decomposed image is then reconstructed using the Laplacian pyramid to obtain the reconstructed image. Specifically: Assuming the original lung image is Ict, and the bottom layer G0 of the Gaussian pyramid is Ict, the first layer of the Gaussian pyramid is obtained through low-pass filtering and downsampling. This process is repeated N times to obtain the Nth layer Gn of the Gaussian pyramid, where the expression for each layer of the Gaussian pyramid is as follows: in, and These represent the first and second halves of the Gaussian pyramid, respectively. The number of rows and columns in the layer image. This represents a low-pass filter with a size of 5*5; By upsampling layer by layer, a pyramid image of the same size as the previous layer is obtained. , and The difference is used to obtain the Laplacian pyramid. The expression is as follows: in, , ; so: The decomposed image is obtained. ; Image reconstruction of the decomposed image can be specifically represented as follows: Obtain the reconstructed image of layer l. .
5. The lung image segmentation method according to claim 4, characterized in that, In step S22, image fusion is performed on the reconstructed images obtained from each layer of reconstruction to obtain the final reconstructed image. The enhanced image is then obtained by uniformly distributing the grayscale values of the final reconstructed image using the CLAHE algorithm. Specifically: Reconstruct the image at each layer The final reconstructed image is obtained by fusing the images after unifying their sizes. Specifically, it can be expressed as: The enhanced image is obtained by uniformly distributing the gray values of the image using the CLAHE algorithm on the final reconstructed image.
6. A lung image segmentation system for performing the lung image segmentation method as described in any one of claims 1-5, characterized in that, include: The input module is used to input the DICOM sequence of lung CT and convert the original DICOM sequence into a JPG format image as the original lung image; The enhancement module is used to enhance the original lung image using the Lap-CLAHE image enhancement algorithm to obtain an enhanced image; The segmentation module is used to input the enhanced image into the U-shaped symmetric network segmentation model 3D-SDUnet, which consists of an encoder, a decoder, DPA dual-path attention, and skip connections, and output the final segmentation result of the lung image.
7. A computer device comprising a memory and one or more processors, the memory storing computer code that, when executed by the one or more processors, causes the one or more processors to perform the method as described in any one of claims 1 to 5.
8. A computer-readable storage medium storing computer code, wherein when the computer code is executed, the method of any one of claims 1 to 5 is performed.