Pulmonary nodule segmentation method based on DVMF-Unet

Abstract

The invention relates to a DVMF-Unet-based pulmonary nodule segmentation method. The method comprises the steps of S1, collecting a lung CT and performing preprocessing to generate input data; s2, carrying out feature extraction on the input data through a multi-domain hybrid convolution cascade encoder and a Vision Mama cascade encoder; s3, performing exchange fusion on the features extracted by the multi-frequency domain hybrid convolution cascade encoder and the Vision Mama cascade encoder through a multi-channel enhanced feature coupling module to obtain enhanced features; s4, inputting the reinforcement features into a decoder through two parallel paths: performing reinforcement processing on one path through a stacking parallel multi-scale module; the other path is directly transmitted to the decoder without other processing; after the two paths are fused, two prediction initial results are output; and S5, splicing and fusing the two prediction initial results to generate a final segmentation result. According to the invention, accurate segmentation of pulmonary nodules can be realized. The pulmonary nodule segmentation method based on the DVMF-Unet can be widely applied to the field of medical image segmentation.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and specifically to a lung nodule segmentation method based on DVMF-Unet. Background Technology

[0002] Accurate segmentation of pulmonary nodules is crucial for the diagnosis and treatment of early-stage lung cancer. With the widespread application of computed tomography (CT) technology, lung CT images have become an indispensable tool in clinical diagnosis. However, due to the complex morphology, varying sizes, and low contrast with surrounding tissues of pulmonary nodules, automated and precise segmentation remains a challenging task.

[0003] Existing methods for lung nodule segmentation mainly include algorithms based on traditional image processing and algorithms based on deep learning. Traditional methods rely on manual feature extraction and rules, which are difficult to handle the diversity and complexity of nodule morphology. In contrast, deep learning-based methods, especially convolutional neural networks (U-Net), have made significant progress in feature extraction and segmentation accuracy.

[0004] While these deep learning models have made significant progress in lung nodule segmentation, they still have limitations in handling blurred nodule edges and complex texture features, such as ground-glass nodules. On CT images, these nodules appear as faint, hazy shadows, like breath covering glass, lacking clear boundaries and exhibiting low contrast, making them one of the biggest challenges in lung nodule segmentation. This affects the accuracy of nodule edge definition, leading to segmentation results that are either too large (including too much surrounding tissue) or too small (missing parts of the nodule area), thus impacting subsequent diagnostic and treatment decisions. Therefore, how to effectively improve the model's ability to capture subtle texture and edge features of lung nodules while simultaneously considering their global characteristics remains a pressing technical challenge.

[0005] In summary, existing technologies are insufficient in efficiently utilizing the global and local features of lung nodule images, and there is an urgent need to develop new segmentation methods to improve the accuracy and reliability of segmentation. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a lung nodule segmentation method based on DVMF-Unet, which can provide users with a more efficient and accurate lung nodule segmentation solution.

[0007] Specifically, the method includes the following steps: S1: Acquire lung CT scans and preprocess them to generate input data; S2: Feature extraction from input data is performed using a multi-domain hybrid convolutional cascade encoder and a Vision Mamba cascade encoder; S3: Enhanced features are obtained by exchanging and fusing the features extracted by the multi-frequency domain hybrid convolutional cascade encoder and the Vision Mamba cascade encoder through a multi-channel enhanced feature coupling module. S4: The enhanced features are input to the decoder through two parallel paths: one path is enhanced by stacked parallel multi-scale modules; the other path is directly transmitted to the decoder without any other processing; the two paths are fused and two initial prediction results are output. S5: The two initial prediction results are spliced ​​and fused to generate the final segmentation result.

[0008] Preferably, S1 includes the following steps: S1.1: Acquire lung CT image data and obtain the coordinates and size of each nodule; S1.2: Extract and generate image data and nodule labels of the same size based on the coordinates and size of each nodule; S1.3: Divide the acquired lung nodule images and corresponding annotations into a dataset to obtain a training set, a validation set, and a test set; Preferably, S2 includes the following steps: S2.1: Construct a multi-domain fusion convolution cascade encoder (MDFCCE) and a Vision Mamba cascade encoder (VMCE); S2.2: Based on the MDFCCE encoder, feature extraction calculation is performed on the input image features to obtain multi-domain features. ; S2.3: Based on the VMCE encoder, feature extraction and calculation are performed on the input image features to obtain the Vision Mamba features. ; Preferably, S2.1 includes the following steps: By incorporating multiple convolutions through parallel convolution and introducing Multi-Domain Fusion Convolution (MDFC) to enhance the encoder's ability to extract edge features, the MDFCCE encoder is obtained.

[0009]

[0010]

[0011]

[0012]

[0013]

[0014]

[0015] In the above formula, These represent the four frequency domains after wavelet transform. The splicing characteristics represent the four frequency domains. , These represent wavelet transform and inverse wavelet transform, respectively. , These represent the Fast Fourier Transform and the Inverse Fast Fourier Transform, respectively. Represents a Gaussian high-pass filter. , Represents intermediate characteristics. This represents a convolution kernel of size n. This represents a splicing operation. The features representing the output of wavelet domain convolutional splicing are... The features representing the output of the Fast Fourier Transform convolution, Represents element-based multiplication. Represents matrix addition. Represents matrix multiplication. Represents the Sigmoid activation function; Preferably, S3 includes the following steps: S3.1: Construct a multi-channel enhanced feature coupling module; S3.2: Exchange and couple the features of the same layer of the two networks to obtain enhanced multi-domain features. and Vision Mamba features ; Preferably, S3.1 includes the following steps: Based on the multi-domain features output using the MDFCCE encoder Vision Mamba features output by the VMCE encoder By enriching each other's features to improve the accuracy of lung nodule segmentation, the FCUMCE structure is obtained:

[0016]

[0017]

[0018]

[0019]

[0020]

[0021] In the above formula, , , The representatives passed Split Features obtained by segmenting channels, where , , This represents a convolution kernel of size n. , Represents intermediate characteristics. Represents depthwise convolution. Represents Layer Norm normalization. This represents a splicing operation. Represents element-based multiplication. Represents matrix addition. , Characteristics representing the output of the MDFCCE encoder and VMCE encoder, This represents the enhanced Vision Mamba features; Preferably, S4 includes the following steps: S4.1: Enhanced multi-domain features from encoder output Features of Vision Mamba As input, multi-scale feature fusion and enhancement are performed through skip connections with stacked parallel multi-scale modules as the core, and the processed features are transmitted to the decoder in the same layer. S4.2: Apply the same enhanced multi-domain features Features of Vision Mamba Without going through intermediate modules, it is directly transmitted to the decoder as another input to ensure the integrity of the original feature information; S4.3: The decoder synchronously receives enhanced features from skip connection paths and raw features from direct paths, performs corresponding decoding operations based on these features, restores spatial resolution through layer-by-layer upsampling, and finally outputs the initial lung nodule segmentation results corresponding to the multi-domain features. Preliminary results of lung nodule segmentation corresponding to Vision Mamba features ; Preferably, S4.1 includes the following steps: For the three-layer pyramid structure in the encoder and decoder, a multi-channel multi-scale feature fusion module is used as a skip connection to combine shallow detailed features with deep semantic features, thereby improving the comprehensiveness and accuracy of segmentation.

[0022]

[0023]

[0024] In the above formula, These represent the features of the pyramid encoder MDFCCE. Represents intermediate characteristics. This represents a convolution kernel of size n. This represents a splicing operation. Represents matrix addition. Represents the Sigmoid activation function. This represents the features obtained after passing through the multi-channel, multi-scale feature fusion module; Preferably, S5 includes the following steps: S5.1: The initial segmentation result and After stitching, adjust the channels and output the lung nodule segmentation results; S5.2: Obtain the segmentation model of the lung nodule segmentation method based on DVMF-Unet; Preferably, S5.1 includes the following steps:

[0025] In the above formula, This is the final output prediction result.

[0026] The beneficial effects of this invention are as follows: This invention extracts lung nodules from CT lung images through a series of preprocessing steps and divides the acquired lung nodule images and corresponding annotations into training, validation, and test sets. Then, it uses a Multi-Frequency Domain Hybrid Convolutional Cascade Encoder (MDFCCE) and a Vision Mamba Cascade Encoder (VMCE) to extract global and local features from the lung nodule images, respectively. Next, it effectively fuses the two features using a Multi-Channel Enhanced Feature Coupling Module (FCUMCE) and employs a Stacked Parallel Multi-Scale Module (SPMSM) as skip connections to enhance edge texture features. Finally, the two features are input into their respective decoders, and feature fusion is performed to generate the lung nodule segmentation result, thereby achieving accurate segmentation of lung nodules with blurred edge textures. Meanwhile, this invention can effectively address the common challenges of existing neural network models in lung nodule segmentation. On the one hand, by adopting a dual-stream Unet model, it improves the model's ability to extract both global and local features of lung nodules. On the other hand, by using a multi-channel enhanced feature coupling module to effectively fuse global and local features, the model can segment lung nodules from a more comprehensive perspective. Furthermore, by stacking parallel multi-scale modules, it further improves the model's utilization of local features, ultimately enhancing the effectiveness and reliability of lung nodule segmentation. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following is a brief introduction to the drawings used in the prior art and embodiments. The following drawings are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 is a flowchart illustrating a lung nodule segmentation method based on DVMF-Unet according to the present invention; Figure 2 is a schematic diagram of the model architecture of a lung nodule segmentation method based on DVMF-Unet according to the present invention; Figure 3 This is a schematic diagram of the MDFCCE module structure of a lung nodule segmentation method based on DVMF-Unet according to the present invention; Figure 4 This is a schematic diagram of the FCUMCE module structure of a lung nodule segmentation method based on DVMF-Unet according to the present invention; Figure 5 This is a schematic diagram of the SPMSM module structure of a lung nodule segmentation method based on DVMF-Unet according to the present invention; Detailed Implementation Plan To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0029] This invention provides a lung nodule segmentation method and apparatus based on DVMF-Unet, which is used to efficiently and accurately segment nodules in lung CT images at the algorithm level, thereby improving the accuracy and efficiency of diagnosis.

[0030] A typical embodiment of the present invention, using the LUNA16 dataset as an example, refers to... Figure 1 The method includes the following steps: S1: Acquire lung CT scans and preprocess them to generate input data; S2: Feature extraction from input data is performed using a multi-domain hybrid convolutional cascade encoder and a Vision Mamba cascade encoder; S3: Enhanced features are obtained by exchanging and fusing the features extracted by the multi-frequency domain hybrid convolutional cascade encoder and the Vision Mamba cascade encoder through a multi-channel enhanced feature coupling module. S4: The enhanced features are input to the decoder through two parallel paths: one path is enhanced by stacked parallel multi-scale modules; the other path is directly transmitted to the decoder without any other processing; the two paths are fused and two initial prediction results are output. S5: The two initial prediction results are spliced ​​and fused to generate the final segmentation result.

[0031] Furthermore, S1 includes the following steps: S1.1: Acquire lung CT image data and obtain the coordinates and size of each nodule; Furthermore, S1.1 includes the following steps: S1.1.1: Download the lung CT images contained in the LUNA16 dataset; S1.1.2: Read the DICOM sequence and convert it to Hounsfield units (HU value); S1.1.3: Linearly map the selected HU value range to a grayscale range of 0-255, and truncate values ​​outside the range; S1.1.4: Convert the processed floating-point HU value data into an 8-bit unsigned integer (range 0-255). S1.2: Extract and generate image data and nodule labels of the same size based on the coordinates and size of each nodule; Furthermore, S1.2 includes the following steps: S1.2.1: Adjust the position of the cropping region according to the size of the nodule to generate an image block with a resolution of 64×64 pixels; S1.2.2: For the generated image patch, create a corresponding nodule annotation mask to mark the nodule region; S1.3: Divide the acquired lung nodule images and corresponding annotations into a dataset to obtain a training set, a validation set, and a test set; Furthermore, S1.3 includes the following steps: S1.3.1: Divide the prepared dataset into training set, validation set, and test set by random sampling; S1.3.2: Distribute the training set, validation set, and test set in a ratio of 7:1:2; Furthermore, refer to Figure 2 and Figure 3 S2 includes the following steps: S2.1: Construct a multi-domain fusion convolution cascade encoder (MDFCCE) and a Vision Mamba cascade encoder (VMCE); S2.2: Based on the MDFCCE encoder, feature extraction calculation is performed on the input image features to obtain multi-domain features. ; S2.3: Based on the VMCE encoder, feature extraction and calculation are performed on the input image features to obtain the Vision Mamba features. ; Furthermore, S2.1 includes the following steps: By incorporating multiple convolutions through parallel convolution and introducing Multi-Domain Fusion Convolution (MDFC) to enhance the encoder's ability to extract edge features, the MDFCCE encoder is obtained.

[0032]

[0033]

[0034]

[0035]

[0036]

[0037]

[0038] In the above formula, These represent the four frequency domains after wavelet transform. The splicing characteristics represent the four frequency domains. , These represent wavelet transform and inverse wavelet transform, respectively. , These represent the Fast Fourier Transform and the Inverse Fast Fourier Transform, respectively. Represents a Gaussian high-pass filter. , Represents intermediate characteristics. This represents a convolution kernel of size n. This represents a splicing operation. The features representing the output of wavelet domain convolutional splicing are... The features representing the output of the Fast Fourier Transform convolution, Represents element-based multiplication. Represents matrix addition. Represents matrix multiplication. Represents the Sigmoid activation function; Furthermore, refer to Figure 3 S3 includes the following steps: S3.1: Construct a multi-channel enhanced feature coupling module; S3.2: Exchange and couple the features of the same layer of the two networks to obtain enhanced multi-domain features. and Vision Mamba features ; Furthermore, S3.1 includes the following steps: Based on the multi-domain features output using the MDFCCE encoder Vision Mamba features output by the VMCE encoder By enriching each other's features to improve the accuracy of lung nodule segmentation, the FCUMCE structure is obtained:

[0039]

[0040]

[0041]

[0042]

[0043]

[0044] In the above formula, , , The representatives passed Split Features obtained by segmenting channels, where , , This represents a convolution kernel of size n. , Represents intermediate characteristics. Represents depthwise convolution. Represents Layer Norm normalization. This represents a splicing operation. Represents element-based multiplication. Represents matrix addition. , Characteristics representing the output of the MDFCCE encoder and VMCE encoder, This represents the enhanced Vision Mamba features; Furthermore, refer to Figure 5 S4 includes the following steps: S4.1: Enhanced multi-domain features from encoder output Features of Vision Mamba As input, multi-scale feature fusion and enhancement are performed through skip connections with stacked parallel multi-scale modules as the core, and the processed features are transmitted to the decoder in the same layer. S4.2: Apply the same enhanced multi-domain features Features of Vision Mamba Without going through intermediate modules, it is directly transmitted to the decoder as another input to ensure the integrity of the original feature information; S4.3: The decoder synchronously receives enhanced features from skip connection paths and raw features from direct paths, performs corresponding decoding operations based on these features, restores spatial resolution through layer-by-layer upsampling, and finally outputs the initial lung nodule segmentation results corresponding to the multi-domain features. Preliminary results of lung nodule segmentation corresponding to Vision Mamba features ; Furthermore, S4.1 includes the following steps: For the pyramid structure in the encoder and decoder, a multi-channel multi-scale feature fusion module is used as a skip connection to combine shallow detailed features with deep semantic features, thereby improving the comprehensiveness and accuracy of segmentation.

[0045]

[0046]

[0047] In the above formula, These represent the features of the pyramid encoder MDFCCE. Represents intermediate characteristics. This represents a convolution kernel of size n. This represents a splicing operation. Represents matrix addition. Represents the Sigmoid activation function. This represents the features after passing through the multi-channel, multi-scale feature fusion module.

[0048] Furthermore, S5 includes the following steps: S5.1: The initial segmentation result and After stitching, adjust the channels and output the lung nodule segmentation results; S5.2: Obtain the segmentation model of the lung nodule segmentation method based on DVMF-Unet; Furthermore, S5.1 includes the following steps:

[0049] In the above formula, This is the final output prediction result.

[0050] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make other equivalent modifications or substitutions without departing from the spirit of the invention. These equivalent modifications or substitutions are included within the scope defined by the claims.

Claims

1. A lung nodule segmentation method based on DVMF-Unet, characterized in that, The method includes the following steps: S1: Acquire lung CT scans and preprocess them to generate input data; S1 includes the following steps: S1.1: Acquire lung CT image data and obtain the coordinates and size of each nodule; S1.2: Extract and generate image data and nodule labels of the same size based on the coordinates and size of each nodule; S1.3: Divide the acquired lung nodule images and corresponding annotations into a dataset to obtain a training set, a validation set, and a test set; S2: Feature extraction from input data is performed using a multi-domain hybrid convolutional cascade encoder and a Vision Mamba cascade encoder; S2 includes the following steps: S2.1: Construct a multi-domain fusion convolution cascade encoder (MDFCCE) and a Vision Mamba cascade encoder (VMCE); S2.2: Based on the MDFCCE encoder, feature extraction calculation is performed on the input image features to obtain multi-domain features. ; S2.3: Based on the VMCE encoder, feature extraction and calculation are performed on the input image features to obtain the Vision Mamba features. ; S3: Enhanced features are obtained by exchanging and fusing the features extracted by the multi-frequency domain hybrid convolutional cascade encoder and the Vision Mamba cascade encoder through a multi-channel enhanced feature coupling module. S3 includes the following steps: S3.1: Construct a multi-channel enhanced feature coupling module; S3.2: Exchange and couple the features of the same layer of the two networks to obtain enhanced multi-domain features. and VisionMamba features ; S4: The enhanced features are input to the decoder through two parallel paths: one path is enhanced by stacked parallel multi-scale modules; the other path is directly transmitted to the decoder without any other processing; the two paths are fused and two initial prediction results are output. S4 includes the following steps: S4.1: Enhanced multi-domain features from encoder output Features of Vision Mamba As input, multi-scale feature fusion and enhancement are performed through skip connections with stacked parallel multi-scale modules as the core, and the processed features are transmitted to the decoder in the same layer. S4.2: Apply the same enhanced multi-domain features Features of Vision Mamba Without going through intermediate modules, it is directly transmitted to the decoder as another input to ensure the integrity of the original feature information; S4.3: The decoder synchronously receives enhanced features from skip connection paths and raw features from direct paths, performs corresponding decoding operations based on these features, restores spatial resolution through layer-by-layer upsampling, and finally outputs the initial lung nodule segmentation results corresponding to the multi-domain features. Preliminary results of lung nodule segmentation corresponding to Vision Mamba features ; S5: The two initial prediction results are concatenated and merged to generate the final segmentation result.

2. The lung nodule segmentation method based on DVMF-Unet according to claim 1, characterized in that, S2 includes the following steps: S2.1: Construct a multi-domain fusion convolution cascade encoder (MDFCCE) and a Vision Mamba cascade encoder (VMCE); S2.2: Based on the MDFCCE encoder, feature extraction calculation is performed on the input image features to obtain multi-domain features. ; S2.3: Based on the VMCE encoder, feature extraction and calculation are performed on the input image features to obtain the Vision Mamba features. ; S2.1 includes the following steps: By incorporating multiple convolutions through parallel convolution and introducing Multi-Domain Fusion Convolution (MDFC) to enhance the encoder's ability to extract edge features, the MDFCCE encoder is obtained. In the above formula, These represent the four frequency domains after wavelet transform. The splicing characteristics represent the four frequency domains. , These represent wavelet transform and inverse wavelet transform, respectively. , These represent the Fast Fourier Transform and the Inverse Fast Fourier Transform, respectively. Represents a Gaussian high-pass filter. , Represents intermediate characteristics. This represents a convolution kernel of size n. This represents a splicing operation. The features representing the output of wavelet domain convolutional splicing are... The features representing the output of the Fast Fourier Transform convolution, Represents element-based multiplication. Represents matrix addition. Represents matrix multiplication. This represents the Sigmoid activation function.

3. The lung nodule segmentation method based on DVMF-Unet according to claim 1, characterized in that, S3 includes the following steps: S3.1: Construct a multi-channel enhanced feature coupling module; S3.2: Exchange and couple the features of the same layer of the two networks to obtain enhanced multi-domain features. and VisionMamba features ; S3.1 includes the following steps: Based on the multi-domain features output using the MDFCCE encoder Vision Mamba features output by the VMCE encoder By enriching each other's features to improve the accuracy of lung nodule segmentation, the FCUMCE structure is obtained: In the above formula, , , The representatives passed Split Features obtained by segmenting channels, where , , This represents a convolution kernel of size n. , Represents intermediate characteristics. Represents depthwise convolution. Represents Layer Norm normalization. This represents a splicing operation. Represents element-based multiplication. Represents matrix addition. , Characteristics representing the output of the MDFCCE encoder and VMCE encoder, This represents the enhanced Vision Mamba features.

4. The lung nodule segmentation method based on DVMF-Unet according to claim 1, characterized in that, S4 includes the following steps: S4.1: Enhanced multi-domain features from encoder output Features of Vision Mamba As input, multi-scale feature fusion and enhancement are performed through skip connections with stacked parallel multi-scale modules as the core, and the processed features are transmitted to the decoder in the same layer. S4.2: Apply the same enhanced multi-domain features Features of Vision Mamba Without going through intermediate modules, it is directly transmitted to the decoder as another input to ensure the integrity of the original feature information; S4.3: The decoder synchronously receives enhanced features from skip connection paths and raw features from direct paths, performs corresponding decoding operations based on these features, restores spatial resolution through layer-by-layer upsampling, and finally outputs the initial lung nodule segmentation results corresponding to the multi-domain features. Preliminary results of lung nodule segmentation corresponding to Vision Mamba features ; S4.1 includes the following steps: For the pyramid structure in the encoder and decoder, stacked parallel multi-scale modules are used as skip connections to combine shallow detailed features with deep semantic features, thereby improving the comprehensiveness and accuracy of segmentation. In the above formula, These represent the features of the pyramid encoder MDFCCE. Represents intermediate characteristics. This represents a convolution kernel of size n. This represents a splicing operation. Represents matrix addition. Represents the Sigmoid activation function. This represents the features after passing through the multi-channel, multi-scale feature fusion module.

5. The lung nodule segmentation method based on DVMF-Unet according to claim 1, characterized in that, S5 includes the following steps: S5.1: The initial segmentation result and After stitching, adjust the channels and output the lung nodule segmentation results; S5.2: Obtain the segmentation model of the lung nodule segmentation method based on DVMF-Unet; S5.1 includes the following steps: In the above formula, This is the final output prediction result.

Images