A lung nodule segmentation method and device based on ABVM-UNet

Abstract

The application provides a lung nodule segmentation method and device based on an ABVM-UNet, and the method comprises the following steps: S1, collecting a lung CT image dataset, and performing enhancement processing on a lung nodule region in the image to highlight the texture and edge features of the nodule; S2, constructing a main branch and a target enhancement auxiliary branch based on a Gabor convolutional neural network to realize enhancement response of the lung nodule texture and edge features; S3, using a branch information interaction fusion module BIIF to realize fusion and transmission of the auxiliary branch enhancement features; S4, constructing an ABVM-UNet based on a Vision Mamba U-shaped architecture network to realize effective fusion of local features and global context information of the image; S5, using deep supervision to realize multi-level segmentation loss and training and optimization of the ABVM-UNet model; and S6, using the ABVM-UNet model to perform lung nodule segmentation on the lung CT image. By introducing the ABVM-UNet model, the edge and texture features of the lung nodule are effectively extracted, so that accurate lung nodule segmentation is realized.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging, and in particular to a pulmonary nodule segmentation method and device based on ABVM-UNet. Background Art

[0002] According to statistics from the World Health Organization's International Agency for Research on Cancer, lung cancer ranks first in cancer incidence and has the highest mortality rate worldwide. Early detection of lung nodules is crucial for lung cancer prevention and treatment. During screening, doctors must accurately annotate lesions in CT images and utilize lung nodule segmentation algorithms to define nodule margins, monitor growth and morphological changes, and measure key data such as nodule size and diameter.

[0003] In recent years, deep learning technology has become a hot topic in the field of lung nodule segmentation, and many deep learning-based methods have been proposed and applied to this task. These models can automatically learn and extract features from CT slices to achieve lung nodule segmentation. Although these deep learning models have made significant progress in lung nodule segmentation, they still have certain limitations when dealing with fuzzy nodule edges and complex texture features. For example, when the boundary between the nodule and the surrounding lung tissue is unclear or the internal structure of the nodule is similar to that of normal lung tissue, the model often finds it difficult to accurately distinguish the nodule from its surrounding tissue. This not only affects the accuracy of the nodule edge definition, but may also lead to false positive or false negative results, thereby affecting subsequent diagnosis and treatment decisions. Therefore, how to effectively capture and utilize the subtle features of the texture and edges of lung nodules to improve the segmentation accuracy of the model has become a key issue that needs to be urgently addressed in current research. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a lung nodule segmentation method and device based on ABVM-UNet, which can realize accurate segmentation of lung nodules in lung CT images for users.

[0005] Specifically, the method comprises the following steps:

[0006] S1: Collect lung CT image datasets and enhance the lung nodule areas in the images to highlight the texture and edge features of the nodules;

[0007] S2: Constructing a main branch and a target enhancement auxiliary branch based on a Gabor convolutional neural network to achieve enhanced response to the texture and edge features of lung nodules;

[0008] S3: Utilize the branch information interaction fusion module BIIF to achieve the fusion and transmission of auxiliary branch enhancement features;

[0009] S4: Construct a U-shaped architecture network ABVM-UNet based on Vision Mamba to effectively fuse local features and global context information of images;

[0010] S5: Use deep supervision to realize multi-level segmentation loss and train and optimize the ABVM-UNet model;

[0011] S6: Use the ABVM-UNet model to segment lung nodules in the lung CT image.

[0012] Preferably, S1 comprises the following steps:

[0013] S1.1: Collect lung CT image data sets, generate lung parenchymal images, and unify the pixel intervals between images;

[0014] Preferably, S1.1 comprises the following steps:

[0015] S1.1.1: Collect lung CT image data sets, multiply the images and lung mask images element by element, remove irrelevant areas, and generate lung parenchymal images;

[0016] S1.1.2: Resample the lung parenchymal images to unify the pixel intervals.

[0017] S1.2: Crop lung nodule image blocks and perform image enhancement operations on the lung nodule tissue parts in the image blocks to obtain enhanced image blocks;

[0018] Preferably, S1.2 comprises the following steps:

[0019] S1.2.1: Centered on the lung nodule center coordinate point, crop image blocks of the same size from the lung parenchymal image and the lung nodule mask image, respectively denoted as Image raw and GroundTruth;

[0020] S1.2.2: Multiply the original image block and the lung nodule mask image block element by element, only keep the information of the lung nodule tissue, perform SG double filtering at the same time, and superimpose the filtered image block into the original image block as the enhanced image block Image aug .

[0021] S1.3: Perform data preprocessing operations on the original image block and the enhanced image block, and simultaneously perform image data augmentation operations.

[0022] Wherein, S1.3 comprises the following steps:

[0023] S1.3.1: Perform data normalization and mean removal processing on the original image block and the enhanced image block;

[0024] S1.3.2: Use image data augmentation techniques to expand the data.

[0025] Preferably, S2 includes the following steps:

[0026] S2.1: Construct a main branch and a target enhancement auxiliary branch based on Gabor convolutional neural network. The main branch consists of multiple Conv modules, and the target enhancement auxiliary branch consists of multiple GCN modules.

[0027] S2.2: The original image block Image raw Input the main branch Conv module to perform feature extraction calculation and obtain the feature map f main ;

[0028] Preferably, S2.2 includes the following steps:

[0029] Using convolutional neural network, convolution calculation is performed to obtain the Conv structure:

[0030] f maim =Swish(BN(conv(x)))

[0031] In the above formula, Swish(.) represents the Swish activation function, BN(.) represents Batch Normalization, conv(.) represents the convolution kernel, and x represents the input data.

[0032] S2.3: Enhance the image block Image aug Input the target enhancement auxiliary branch GCN module to perform feature extraction calculation and obtain the enhanced feature map f aug .

[0033] Preferably, S2.3 includes the following steps:

[0034] Using Gabor convolutional neural network, convolution calculation is performed to obtain the GCN structure:

[0035] f aug =Swish(BN(Gabor(x)))

[0036] In the above formula, Swish(.) represents the Swish activation function, BN(.) represents Batch Normalization, Gabor(.) represents Gabor filtering, and x represents the input data.

[0037] Preferably, S3 includes the following steps:

[0038] S3.1: Construct the branch information interaction fusion module BIIF (Branch Information Interaction Fusion);

[0039] S3.2: The feature f extracted by the main branch main and the feature f extracted by the target enhancement auxiliary branch aug Input the BIIF module to perform information interaction and fusion to obtain the fused feature f m ′ ain With f a ′ ug .

[0040] Preferably, S3.2 includes the following steps:

[0041] Different calculation methods are used for the features extracted by the main branch and the target enhanced auxiliary branch to obtain the BIIF structure:

[0042] f′ main =f main

[0043] +Res(channelshuffle(concatenate(VSS(f main )

[0044] +VSS(f aug ))))

[0045] f′ aug =f aug

[0046] +Res(channelshuffle(concatenate(VSS(f main )

[0047] +VSS(f aug ))))

[0048] In the above formula, f main represents the features extracted by the main branch, f aug Represents the features extracted by the target enhancement auxiliary branch, Res(.) represents the residual module, channel shuffle(.) represents the channel shuffle operation, that is, rearranging the channel order of the original feature map to enhance the interactive fusion of information between channels, concatenate(.) represents the channel dimension splicing operation, and VSS(.) represents the VSS module based on Vision Mamba.

[0049] Preferably, S4 includes the following steps:

[0050] S4.1: Design a U-shaped architecture network ABVM-UNet based on Vision Mamba;

[0051] S4.2: Effectively integrate the Conv module, GCN module, BIIF module and VSS module into ABVM-UNet to achieve effective fusion of local image features and global context information.

[0052] Preferably, S4.2 includes the following steps:

[0053] S4.2.1: The encoder of ABVM-UNet consists of a main branch and a target-enhanced auxiliary branch, which is composed of multiple Conv modules, GCN modules, and BIIF modules alternating with each other;

[0054] S4.2.2: The bottleneck of ABVM-UNet is the encoder-decoder conversion process, which consists of two VSS modules.

[0055] S4.2.3: The decoder of ABVM-UNet consists of multiple Conv modules and VSS modules alternating with each other, and a 1×1 convolutional layer is used in the last layer of the decoder to predict the segmentation result;

[0056] S4.2.4: The downsampling part between each encoder layer of ABVM-UNet uses the Max Pooling operation, and the upsampling part between each decoder layer uses the deconvolution operation;

[0057] S4.2.5: ABVM-UNet uses skip connections to concatenate the output of the BIIF module in the encoder and the output of the VSS module in the decoder in the channel dimension, and perform channel shuffling operations at the same time.

[0058] Preferably, S5 includes the following steps:

[0059] S5.1: Calculate the segmentation loss for the segmentation prediction results of each decoder layer of the model through deep supervision. The segmentation loss consists of Dice loss and Binary Cross-Entropy loss.

[0060] Preferably, S5.1 includes the following steps:

[0061] S5.1.1: Apply a 1x1 convolution operation to each decoder layer to obtain multi-scale segmentation prediction results. From the shallowest layer to the deepest layer of the decoder, the segmentation prediction results are recorded as: Pred, Pred1, Pred2, Pred3, Pred4;

[0062] S5.1.2: Calculate the segmentation loss based on the lung nodule mask label and the segmentation prediction result Pred;

[0063] S5.1.3: Perform Bilinear Interpolation upsampling on Pred1 and Pred2 to obtain a segmentation prediction result of the same size as the lung nodule mask label and calculate its segmentation loss;

[0064] S5.1.4: Perform Max Pooling downsampling on the lung nodule mask labels to obtain lung nodule mask labels of the same size as Pred3 and Pred4, and calculate the segmentation loss of the segmentation prediction results Pred3 and Pred4.

[0065] S5.2: Assign the segmentation loss of each layer of decoder to a weight parameter and form a multi-level segmentation loss;

[0066] Preferably, S5.2 comprises the following steps:

[0067] Using Dice loss and cross entropy loss, we calculate the loss of the segmentation prediction results of each layer of decoder and obtain the multi-level segmentation loss:

[0068]

[0069] In the above formula, L seg Represents the segmentation loss of the segmentation prediction result, L Dice (.) represents Dice loss, L BCE (.) represents Binary Cross-Entropy cross entropy loss, y represents the lung nodule mask label, Represents the final segmentation prediction result, represents the segmentation prediction result of the i-th layer decoder, L i Represents the segmentation loss of the i-th layer decoder segmentation prediction result, BI(.) represents Bilinear Interpolation upsampling, MP(.) represents Max Pooling downsampling, L total represents the multi-level segmentation loss, α i Represents the weight parameter of the segmentation loss of the i-th layer decoder.

[0070] S5.3: Use multi-level segmentation loss to train and optimize the ABVM-UNet model. During training, use the backpropagation algorithm to optimize the model parameters to reduce the multi-level segmentation loss and thus improve the segmentation performance of the model.

[0071] Preferably, S6 includes the following steps:

[0072] S6.1: Import the pre-trained weights of the ABVM-UNet model and use the model to perform lung nodule segmentation on lung CT images.

[0073] S6.2: superimpose the lung CT image with the segmentation prediction result, and mark the predicted lung nodule tissue part to obtain a lung nodule segmentation result of the CT image of the patient.

[0074] The second technical solution adopted by the present application is a lung nodule segmentation device based on ABVM-UNet, comprising:

[0075] The preprocessing module is used for collecting data, performing a series of preprocessing and SG double filtering operations on the data, and converting the data into an input format of the model.

[0076] The GCN module is used for the target enhancement auxiliary branch to realize enhanced response of the model to the texture and edge features of the lung nodule.

[0077] The Conv module is used for the main branch to realize effective extraction of the lung nodule features by the model.

[0078] The VSS module is used for modeling the remote interaction of the data to realize effective extraction of the global context information of the image.

[0079] The BIIF module is used for fusing the features extracted by the main branch and the target enhancement auxiliary branch to realize fusion and transmission of the enhanced features of the auxiliary branch.

[0080] The multi-level segmentation loss module is used for training and optimizing the model to realize minimization of the prediction error and improvement of the performance of the model.

[0081] The method and device of the present application have the following beneficial effects: the present application first obtains an enhanced image of the lung nodule target by performing a series of preprocessing and SG double filtering operations on the lung CT image. Then, the main branch and the target enhancement auxiliary branch are designed to realize effective extraction of the image features by the model and enhanced response of the model to the texture and edge features of the lung nodule. Then, the branch information interaction fusion module is used to realize effective fusion of the features extracted by the main branch and the target enhancement auxiliary branch. Finally, the ABVM-UNet based on Vision Mamba is used to integrate the modules to realize effective fusion of the local features and the global context information of the image, so as to achieve the purpose of accurately segmenting the lung nodule tissue in the lung CT image. At the same time, the present application can overcome the challenges generally faced by the existing deep learning model in the field of lung nodule segmentation. On the one hand, by using the double-branch network structure, the extraction ability of the model for the lung nodule features in the CT image is effectively enhanced, and the feature representation is enriched. On the other hand, with the help of the branch information interaction fusion module, the efficient integration ability of the model for the features at each stage is strengthened. At the same time, by using the lung nodule segmentation model based on ABVM-UNet, the precision of the lung nodule segmentation in the lung CT image is improved. BRIEF DESCRIPTION OF DRAWINGS

[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the prior art and the drawings required for use in the embodiments. The following drawings are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0083] Figure 1 Schematic diagram of a process of pulmonary nodule segmentation method based on ABVM-UNet of the present invention;

[0084] Figure 2 This is a model architecture diagram of a lung nodule segmentation method based on ABVM-UNet in the present invention;

[0085] Figure 3 This is a schematic diagram of the BIIF module structure of the ABVM-UNet-based lung nodule segmentation method of the present invention. Specific implementation plan

[0086] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making creative work are within the scope of protection of the present invention.

[0087] The embodiments of the present application provide a lung nodule segmentation method and device based on ABVM-UNet, which is used to effectively extract lung nodule feature information from lung CT images, thereby achieving accurate segmentation of lung nodules.

[0088] As a typical embodiment of the present invention, the LUNA16 lung nodule dataset is used as an example. Figure 1 As shown, the method includes the following steps:

[0089] S1: Collect lung CT image datasets and enhance the lung nodule areas in the images to highlight the texture and edge features of the nodules;

[0090] S2: Constructing a main branch and a target enhancement auxiliary branch based on a Gabor convolutional neural network to achieve enhanced response to the texture and edge features of lung nodules;

[0091] S3: Utilize the branch information interaction fusion module BIIF to achieve the fusion and transmission of auxiliary branch enhancement features;

[0092] S4: Build a U-shaped architecture network ABVM-UNet based on Vision Mamba to effectively integrate local image features with global context information;

[0093] S5: Implementing multi-level segmentation loss using deep supervision and training and optimizing the ABVM-UNet model;

[0094] S6: Use the ABVM-UNet model to segment lung nodules in lung CT images.

[0095] Furthermore, S1 includes the following steps:

[0096] S1.1: Acquire lung CT image datasets, generate lung parenchyma images, and unify the pixel intervals between images;

[0097] Furthermore, S1.1 includes the following steps:

[0098] S1.1.1: Acquire a lung CT image dataset, multiply the image by the lung mask image element-wise, remove irrelevant regions, and generate a lung parenchymal image.

[0099] S1.1.2: Resample the lung parenchymal images to uniform pixel spacing.

[0100] S1.2: intercepting a lung nodule image block and performing an image enhancement operation on the lung nodule tissue portion in the image block to obtain an enhanced image block;

[0101] Furthermore, S1.2 includes the following steps:

[0102] S1.2.1: With the central coordinate point of the lung nodule as the center, cut out image blocks of the same size from the lung parenchyma image and the lung nodule mask image, respectively denoted as Image raw and GroundTruth;

[0103] S1.2.2: Multiply the original image block and the lung nodule mask image block element by element, retaining only the information of the lung nodule tissue, and use the Sobel filter and Gabor filter to perform SG double filtering on the image, and superimpose the image block after the filtering operation into the original image block as the enhanced image block Image aug .

[0104] S1.3: Perform data preprocessing operations on the original image block and the enhanced image block, and perform image data augmentation operations at the same time.

[0105] Furthermore, S1.3 includes the following steps:

[0106] S1.3.1: Perform data normalization and de-averaging on the original image block and the enhanced image block;

[0107] S1.3.2: Image data augmentation is performed by image rotation, image scaling, up-down symmetric flipping, left-right symmetric flipping, and color jittering.

[0108] Further, referring to Figure 2 , S2 includes the following steps:

[0109] S2.1: A main branch and a target enhancement auxiliary branch based on a Gabor convolutional neural network are constructed, the main branch is composed of multiple Conv modules, and the target enhancement auxiliary branch is composed of multiple GCN modules;

[0110] S2.2: The original image block Image raw is input into the main branch Conv module for feature extraction calculation to obtain a feature map f main .

[0111] Further, S2.2 includes the following steps:

[0112] A convolutional neural network is used to perform convolution calculation to obtain a Conv structure:

[0113] f main = Swish(BN(conv(x)))

[0114] In the above formula, Swish(.) represents the Swish activation function, BN(.) represents Batch Normalization, conv(.) represents the convolution kernel, and x represents the input data.

[0115] S2.3: The enhanced image block Image aug is input into the target enhancement auxiliary branch GCN module for feature extraction calculation to obtain an enhanced feature map f aug .

[0116] Further, S2.3 includes the following steps:

[0117] A Gabor convolutional neural network is used to perform convolution calculation to obtain a GCN structure:

[0118] f aug = Swish(BN(Gabor(x)))

[0119] In the above formula, Swish(.) represents the Swish activation function, BN(.) represents Batch Normalization, Gabor(.) represents Gabor filtering, and x represents the input data.

[0120] Further, referring to Figure 3 , S3 includes the following steps:

[0121] S3.1: Construct the branch information interaction fusion module BIIF (Branch Information Interaction Fusion);

[0122] S3.2: The feature f extracted by the main branch main and the feature f extracted by the target enhancement auxiliary branch aug Input the BIIF module to perform information interaction and fusion to obtain the fused feature f′ main and f′ aug .

[0123] Furthermore, S3.2 includes the following steps:

[0124] Different calculation methods are used for the features extracted by the main branch and the target enhanced auxiliary branch to obtain the BIIF structure:

[0125] f′ main =f main

[0126] +Res(channelshuffle(concatenate(VSS(f main )

[0127] +VSS(f aug ))))

[0128] f′ aug =f aug

[0129] +Res(channelshuffle(concatenate(VSS(f main )

[0130] +VSS(f aug ))))

[0131] In the above formula, f main represents the features extracted by the main branch, f aug Represents the features extracted by the target enhancement auxiliary branch, Res(.) represents the residual module, channelshuffle(.) represents the channel shuffle operation, that is, rearranging the channel order of the original feature map to enhance the interactive fusion of information between channels, concatenate(.) represents the channel dimension splicing operation, and VSS(.) represents the VSS module based on Vision Mamba.

[0132] Further, refer to Figure 2 , S4 includes the following steps:

[0133] S4.1: Design a U-shaped architecture network ABVM-UNet based on Vision Mamba;

[0134] S4.2: Effectively integrate the Conv module, GCN module, BIIF module and VSS module into ABVM-UNet to achieve effective fusion of local image features and global context information.

[0135] Furthermore, S4.2 includes the following steps:

[0136] S4.2.1: The encoder of ABVM-UNet consists of a main branch and a target-enhanced auxiliary branch, which is composed of multiple Conv modules, GCN modules, and BIIF modules alternating with each other;

[0137] S4.2.2: The bottleneck of ABVM-UNet is the encoder-decoder conversion process, which consists of two VSS modules.

[0138] S4.2.3: The decoder of ABVM-UNet consists of multiple Conv modules and VSS modules alternating with each other, and a 1×1 convolutional layer is used in the last layer of the decoder to predict the segmentation result;

[0139] S4.2.4: The downsampling part between each encoder layer of ABVM-UNet uses the Max Pooling operation, and the upsampling part between each decoder layer uses the deconvolution operation;

[0140] S4.2.5: ABVM-UNet uses skip connections to concatenate the output of the BIIF module in the encoder and the output of the VSS module in the decoder in the channel dimension, and perform channel shuffling operations at the same time.

[0141] Further, refer to Figure 2 , S5 includes the following steps:

[0142] S5.1: Calculate the segmentation loss for the segmentation prediction results of each decoder layer of the model through deep supervision. The segmentation loss consists of Dice loss and Binary Cross-Entropy loss.

[0143] Furthermore, S5.1 includes the following steps:

[0144] S5.1.1: Apply a 1x1 convolution operation to each decoder layer to obtain multi-scale segmentation prediction results. From the shallowest layer to the deepest layer of the decoder, the segmentation prediction results are recorded as: Pred, Pred1, Pred2, Pred3, Pred4;

[0145] S5.1.2: Calculate the segmentation loss based on the lung nodule mask label and the segmentation prediction result Pred;

[0146] S5.1.3: Perform Bilinear Interpolation upsampling on Pred1 and Pred2 to obtain a segmentation prediction result of the same size as the lung nodule mask label and calculate its segmentation loss;

[0147] S5.1.4: Perform Max Pooling downsampling on the lung nodule mask labels to obtain lung nodule mask labels of the same size as Pred3 and Pred4, and calculate the segmentation loss of the segmentation prediction results Pred3 and Pred4.

[0148] S5.2: Assign the segmentation loss of each layer of decoder to a weight parameter and form a multi-level segmentation loss;

[0149] Furthermore, S5.2 includes the following steps:

[0150] Using Dice loss and cross entropy loss, we calculate the loss of the segmentation prediction results of each layer of decoder and obtain the multi-level segmentation loss:

[0151]

[0152] In the above formula, L seg Represents the segmentation loss of the segmentation prediction result, L Dice (.) represents Dice loss, L BCE (.) represents Binary Cross-Entropy cross entropy loss, y represents the lung nodule mask label, Represents the final segmentation prediction result, represents the segmentation prediction result of the i-th layer decoder, L i Represents the segmentation loss of the i-th layer decoder segmentation prediction result, BI(.) represents Bilinear Interpolation upsampling, MP(.) represents Max Pooling downsampling, L total represents the multi-level segmentation loss, α i Represents the weight parameter of the segmentation loss of the i-th layer decoder.

[0153] S5.3: Use multi-level segmentation loss to train and optimize the ABVM-UNet model. During training, use the backpropagation algorithm to optimize the model parameters to reduce the multi-level segmentation loss and thus improve the segmentation performance of the model.

[0154] Furthermore, S6 includes the following steps:

[0155] S6.1: Import the pre-trained weights of the ABVM-UNet model and use the model to perform lung nodule segmentation on lung CT images.

[0156] S6.2: Superimpose the segmentation prediction result on the lung CT image and mark the predicted lung nodule tissue location to obtain the lung nodule segmentation result of the patient's CT image.

[0157] The above is a specific description of the implementation of the present invention, but the invention is not limited to the embodiments. Technical personnel should know that although the present invention has been described according to the above specific implementation methods, the inventive concept of the present invention is not limited to this invention. Any modification using the inventive concept will be included in the scope of protection of this patent.

Claims

1. A pulmonary nodule segmentation method based on ABVM-UNet, characterized in that: The method comprises the following steps: S1: Collect lung CT image datasets and enhance the lung nodule areas in the images to highlight the texture and edge features of the nodules; S2: Constructing a main branch and a target enhancement auxiliary branch based on a Gabor convolutional neural network to achieve enhanced response to the texture and edge features of lung nodules; Among them, S2 includes the following steps: S2.1: Construct a main branch and a target enhancement auxiliary branch based on Gabor convolutional neural network. The main branch consists of multiple Conv modules, and the target enhancement auxiliary branch consists of multiple GCN modules. S2.2: The original image block Image raw Input the main branch Conv module to perform feature extraction calculation and obtain the feature map f main ; S2.3: Enhance the image block Image aug Input the target enhancement auxiliary branch GCN module to perform feature extraction calculation and obtain the enhanced feature map f aug ; S3: Utilize the branch information interaction fusion module BIIF to achieve the fusion and transmission of auxiliary branch enhancement features; Among them, S3 includes the following steps: S3.1: Construct the branch information interaction fusion module BIIF (Branch Information Interaction Fusion); S3.2: The feature f extracted by the main branch main and the feature f extracted by the target enhancement auxiliary branch aug Input the BIIF module to perform information interaction and fusion to obtain the fused feature f′ main and f′ aug ; S4: Build a U-shaped architecture network ABVM-UNet based on Vision Mamba to effectively integrate local image features with global context information; Among them, S4 includes the following steps: S4.1: Design a U-shaped architecture network ABVM-UNet based on Vision Mamba; S4.2: Effectively integrate the Conv module, GCN module, BIIF module, and VSS module into ABVM-UNet to achieve effective fusion of local image features and global context information; S5: Implementing multi-level segmentation loss using deep supervision and training and optimizing the ABVM-UNet model; S6: Use the ABVM-UNet model to segment lung nodules in lung CT images.

2. The pulmonary nodule segmentation method based on ABVM-UNet according to claim 1, characterized in that: S1 includes the following steps: S1.1: Acquire lung CT image datasets, generate lung parenchyma images, and unify the pixel intervals between images; Among them, S1.1 includes the following steps: S1.1.1: Acquire a lung CT image dataset, multiply the image by the lung mask image element-wise, remove irrelevant regions, and generate a lung parenchymal image. S1.1.2: Resample the lung parenchymal images to uniform pixel spacing. S1.2: intercepting a lung nodule image block and performing an image enhancement operation on the lung nodule tissue portion in the image block to obtain an enhanced image block; Among them, S1.2 includes the following steps: S1.2.1: With the central coordinate point of the lung nodule as the center, cut out image blocks of the same size from the lung parenchyma image and the lung nodule mask image, respectively denoted as Image raw and GroundTruth; S1.2.2: Multiply the original image block and the lung nodule mask image block element by element, retaining only the information of the lung nodule tissue, and perform SG double filtering at the same time, and superimpose the image block after the filtering operation into the original image block as the enhanced image block Image aug ; S1.3: Perform data preprocessing on the original image block and the enhanced image block, and perform image data augmentation at the same time; Among them, S1.3 includes the following steps: S1.3.1: Perform data normalization and de-averaging on the original image block and the enhanced image block; S1.3.2: Use image data augmentation techniques to augment data.

3. The pulmonary nodule segmentation method based on ABVM-UNet according to claim 1, characterized in that: S2.2 includes the following steps: Using convolutional neural network, convolution calculation is performed to obtain the Conv structure: f main =Swish(BN(conv(x))) In the above formula, Swish(.) represents the Swish activation function, BN(.) represents Batch Normalization, conv(.) represents the convolution kernel, and x represents the input data; S2.3 includes the following steps: Using Gabor convolutional neural network, convolution calculation is performed to obtain the GCN structure: f aug =Swish(BN(Gabor(x))) In the above formula, Swish(.) represents the Swish activation function, BN(.) represents Batch Normalization, Gabor(.) represents Gabor filtering, and x represents the input data.

4. The pulmonary nodule segmentation method based on ABVM-UNet according to claim 1, characterized in that: S3.2 includes the following steps: Different calculation methods are used for the features extracted by the main branch and the target enhanced auxiliary branch to obtain the BIIF structure: f′ main =f main +Res(channel shuffle(concatenate(VSS(f main )+VSS(f aug )))) f′ aug =f aug +Res(channel shuffle(concatenate(VSS(f main )+VSS(f aug )))) In the above formula, f main represents the features extracted by the main branch, f aug Represents the features extracted by the target enhancement auxiliary branch, Res(.) represents the residual module, channel shuffle(.) represents the channel shuffle operation, that is, rearranging the channel order of the original feature map to enhance the interactive fusion of information between channels, concatenate(.) represents the channel dimension splicing operation, and VSS(.) represents the VSS module based on Vision Mamba.

5. The pulmonary nodule segmentation method based on ABVM-UNet according to claim 1, characterized in that: S4.2 includes the following steps: S4.2.1: The encoder of ABVM-UNet consists of a main branch and a target-enhanced auxiliary branch, which is composed of multiple Conv modules, GCN modules, and BIIF modules alternating with each other; S4.2.2: The bottleneck of ABVM-UNet is the encoder-decoder conversion process, which consists of two VSS modules. S4.2.3: The decoder of ABVM-UNet consists of multiple Conv modules and VSS modules alternating with each other, and a 1×1 convolutional layer is used in the last layer of the decoder to predict the segmentation result; S4.2.4: The downsampling part between each encoder layer of ABVM-UNet uses the Max Pooling operation, and the upsampling part between each decoder layer uses the deconvolution operation; S4.2.5: ABVM-UNet uses skip connections to concatenate the output of the BIIF module in the encoder and the output of the VSS module in the decoder in the channel dimension, and perform channel shuffling operations at the same time.

6. The pulmonary nodule segmentation method based on ABVM-UNet according to claim 1, characterized in that: S5 includes the following steps: S5.1: Calculate the segmentation loss for the segmentation prediction results of each decoder layer of the model through deep supervision. The segmentation loss consists of Dice loss and Binary Cross-Entropy loss. S5.1 includes the following steps: S5.1.1: Apply a 1x1 convolution operation to each decoder layer to obtain multi-scale segmentation prediction results. From the shallowest layer to the deepest layer of the decoder, the segmentation prediction results are recorded as: Pred, Pred1, Pred2, Pred3, Pred4; S5.1.2: Calculate the segmentation loss based on the lung nodule mask label and the segmentation prediction result Pred; S5.1.3: Perform Bilinear Interpolation upsampling on Pred1 and Pred2 to obtain a segmentation prediction result of the same size as the lung nodule mask label and calculate its segmentation loss; S5.1.4: Perform Max Pooling downsampling on the lung nodule mask labels to obtain lung nodule mask labels of the same size as Pred3 and Pred4, and calculate the segmentation loss of the segmentation prediction results Pred3 and Pred4; S5.2: Assign the segmentation loss of each layer of decoder to a weight parameter and form a multi-level segmentation loss; Among them, S5.2 includes the following steps: Using Dice loss and cross entropy loss, we calculate the loss of the segmentation prediction results of each layer of decoder and obtain the multi-level segmentation loss: In the above formula, L seg Represents the segmentation loss of the segmentation prediction result, L Dice (.) represents Dice loss, L BCE (.) represents BinaryCross-Entropy cross entropy loss, y represents the lung nodule mask label, Represents the final segmentation prediction result, represents the segmentation prediction result of the i-th layer decoder, L i Represents the segmentation loss of the i-th layer decoder segmentation prediction result, BI(.) represents Bilinear Interpolation upsampling, MP(.) represents Max Pooling downsampling, L total represents the multi-level segmentation loss, α i Represents the weight parameter of the segmentation loss of the i-th layer decoder; S5.3: Use multi-level segmentation loss to train and optimize the ABVM-UNet model. During training, use the backpropagation algorithm to optimize the model parameters to reduce the multi-level segmentation loss and thus improve the segmentation performance of the model.

7. The pulmonary nodule segmentation method based on ABVM-UNet according to claim 1, characterized in that: S6 includes the following steps: S6.1: Import the pre-trained weights of the ABVM-UNet model and use the model to perform lung nodule segmentation on lung CT images. S6.2: Superimpose the segmentation prediction results on the lung CT image and mark the predicted lung nodule tissue locations to obtain the lung nodule segmentation results of the patient's CT image.

8. A pulmonary nodule segmentation device based on ABVM-UNet, characterized in that: include: Preprocessing module: used to collect data, perform a series of preprocessing and SG double filtering operations on the data, and convert it into the input format of the model; GCN module: used to build a target enhancement auxiliary branch based on Gabor convolutional neural network to achieve the model's enhanced response to the texture and edge features of lung nodules; the target enhancement auxiliary branch is composed of multiple GCN modules, which enhances the image block Image aug Input the target enhancement auxiliary branch GCN module to perform feature extraction calculation and obtain the enhanced feature map f aug ; Conv module: used to construct the main branch to realize the effective extraction of lung nodule features by the model; the main branch is composed of multiple Conv modules, which convert the original image block Image raw Input the main branch Conv module to perform feature extraction calculation and obtain the feature map f main ; VSS module: used to model remote data interactions and effectively extract global context information from images; BIIF module: used to fuse the features extracted by the main branch and the target enhanced auxiliary branch, and realize the fusion and transmission of the enhanced features of the auxiliary branch; it is also used to build the branch information interaction fusion module BIIF (Branch Information Interaction Fusion), which combines the features extracted by the main branch f main and the feature f extracted by the target enhancement auxiliary branch aug Input the BIIF module to perform information interaction and fusion to obtain the fused feature f′ main and f′ aug ; Multi-level segmentation loss module: used to train and optimize the model to minimize prediction error and improve model performance; segmentation loss is calculated for the segmentation prediction results of each decoder layer of the model through deep supervision. The segmentation loss consists of Dice loss and Binary Cross-Entropy loss; at the same time, the segmentation loss of each decoder layer is assigned a weight parameter to form a multi-level segmentation loss, which is used to train and optimize the ABVM-UNet model; during the training process, the backpropagation algorithm is used to optimize the model parameters to reduce the multi-level segmentation loss, thereby improving the segmentation performance of the model.

Images