Pulmonary nodule segmentation method based on multi-size multi-level feature fusion

Abstract

The invention relates to a pulmonary nodule segmentation method based on multi-size and multi-level feature fusion, and the method comprises the following steps: taking a lung CT image data set, and carrying out the preprocessing of the lung CT image data set; constructing a pulmonary nodule segmentation model based on multi-size and multi-level feature fusion, and performing training and parameter adjustment on the segmentation model by using the training set; taking an ASSLA-Unet network as a segmentation model, and substituting the optimal parameters into the segmentation model to obtain an optimal pulmonary nodule segmentation model; and testing and evaluating the model precision of the optimal pulmonary nodule segmentation model by adopting an evaluation index. The problem that the segmentation accuracy and reliability are poor due to the fact that a pulmonary nodule segmentation method cannot effectively capture multi-scale and multi-level features is solved, ASSLA-Unet adopts a multi-scale feature fusion SLA module, an encoder and an adaptive level connection ASCF module in a decoder, and the effect of extracting pulmonary nodule features more completely is achieved; and an ExpandDwUp module is arranged in the decoder, so that a relatively good detail recovery capability is maintained, and a just segmentation effect is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of pulmonary nodule segmentation, and particularly to a pulmonary nodule segmentation method based on multi-size and multi-level feature fusion. Background Art

[0002] Lung cancer, as the malignant tumor with the highest mortality rate globally, early diagnosis is the key to improving the survival rate of patients. Early-stage lung cancer appears as pulmonary nodules in CT images, and doctors need to judge their benignity or malignancy based on this. However, manual segmentation of pulmonary nodules places a heavy burden on radiologists. Especially in hospitals with limited manpower and resources, it not only consumes energy but also easily delays radiotherapy and reduces the survival rate of patients. Moreover, the accuracy of manual segmentation depends on the experience of doctors, and even with the same strategy, the results often vary. In contrast, the automatic segmentation technology has significant advantages and can quickly and accurately segment lung tumors.

[0003] Traditional pulmonary nodule segmentation methods are based on features such as nodule morphology and gray scale, and operate after the nodule position is known. There are threshold and region growing, clustering, active contour models, and mathematical model optimization methods. For example, threshold and region growing rely on gray scale differences to segment adherent nodules, and use morphological opening operations to remove adherent structures, but it is easy to over-segment or under-segment; the clustering method clusters according to the similarity of pixel features, and introduces the Gaussian Mixture Model (GMM) to improve the accuracy; the active contour model uses curve boundary evolution to obtain the target contour, and the parameters are uncertain; the mathematical model optimization method has achieved results in the segmentation of solid nodules but is sensitive to tubular structures. In addition, deep learning is widely used in pulmonary nodule segmentation, and the network structures are divided into three categories: segmentation methods based on convolutional neural networks, fully convolutional neural networks, and encoder-decoder structure networks. Convolutional neural networks process features using convolutional, pooling, and fully connected layers, and the fully connected layer is prone to losing information; the fully convolutional neural network has fewer parameters and fast calculations, but the deconvolution for restoring the feature map is not fine enough; the encoder-decoder structure network is represented by U-Net, where the encoder extracts features and the decoder restores the resolution to achieve fine segmentation.

[0004] However, in current lung nodule segmentation methods, the receptive field scales used for image feature extraction are often relatively single. The sizes of lung nodules vary greatly. Some small nodules have a diameter of only a few millimeters, while large nodules can have a diameter of several centimeters. A single receptive field scale is difficult to effectively capture the diverse features of nodules of different sizes. Although semantic segmentation models with an encoder-decoder structure improve the resolution and alleviate the gradient vanishing problem through skip connections, at the decoder end, they cannot fully encode multi-scale semantic information at different levels. Lung nodules have complex edge features. Some nodules have smooth edges, while some have irregular and spiky edges. These edge features, which are crucial for accurately identifying nodules, exist at various scales. Existing architectures cannot comprehensively capture edge features at different scales, which greatly affects the segmentation accuracy. In addition, common upsampling methods usually use a fixed ratio. When dealing with complex or high-resolution lung images containing nodules, these methods are insufficient to restore high-quality details. Nodules may be adjacent to normal lung tissue with similar density. Without sufficient detail restoration, it is difficult to accurately distinguish nodules from their surrounding tissues. Therefore, although deep learning has become increasingly mature in the field of image segmentation, there is still much room for improvement, especially in improving the accuracy and reliability of lung nodule segmentation. Summary of the Invention

[0005] Aiming at the problem that the lung nodule segmentation method cannot effectively capture multi-scale and multi-level features, resulting in poor segmentation accuracy and reliability, a lung nodule segmentation method based on multi-size and multi-level feature fusion is proposed.

[0006] The technical solution of the present invention is as follows:

[0007] A lung nodule segmentation method based on multi-size and multi-level feature fusion, comprising the following steps:

[0008] S1: Obtain a lung CT image dataset, preprocess the lung CT image dataset, and divide the preprocessed images into a training set, a test set, a validation set, and prepare the experimental environment;

[0009] S2: Build a lung nodule segmentation model based on multi-size and multi-level feature fusion, and use the training set to train the segmentation model; after each round of training is completed, save the parameters of the segmentation model for this round, calculate the loss by comparing the detection results obtained by the segmentation model with the actual results, and adjust the parameters of the segmentation model according to the loss.

[0010] In step S2, the ASSLA-Unet network is used as the segmentation model. The ASSLA-Unet network adopts an encoder-decoder architecture. The encoder is based on the hierarchical window attention of the SSLA Transformer, and the decoder gradually restores the spatial resolution through the upsampling module ExpandDwUp module. An adaptive skip connection module ASCF module is added between the encoding path and the decoding path; the SSLA Transformer is the Swin transformer with the SLA module added;

[0011] S3: Read the optimal parameters saved in the training phase, and substitute the optimal parameters into the segmentation model to obtain the optimal pulmonary nodule segmentation model;

[0012] S4: Use the test set to test the optimal pulmonary nodule segmentation model, and evaluate the model accuracy of the optimal pulmonary nodule segmentation model using evaluation metrics.

[0013] Furthermore, the specific steps of S2 are as follows:

[0014] S2.1: Input the images in the training set into the encoder. First, each input image is divided into non-overlapping small blocks through the PatchPartition layer, and each divided small block will be linearly projected into a vector of a fixed length by the Linear Embedding layer;

[0015] S2.2: In the encoder, add the SLA module, that is, the SSLA transformer, after normalization and before attention calculation in the Swin transformer. The vector obtained in step S2.1 passes through the SSLA transformer and undergoes downsampling to extract the shallow feature map;

[0016] S2.3: Between the encoder and the decoder, through the adaptive hierarchical connection layer ASCF, the shallow features extracted by the encoder and the deep features of the decoder are fused layer by layer;

[0017] S2.4: In the decoder, after the SSLA transformer, add the upsampling module ExpandDwUp, and upsample the fused features layer by layer, and finally output the pulmonary nodule segmentation image;

[0018] S2.5: Calculate the loss between the pulmonary nodule segmentation image output in step S2.4 and the actual pulmonary nodule segmentation image through the hybrid loss function, adjust the parameters in the ASSLA-Unet network, update the ASSLA-Unet network model, and return to step S2.1 for a new round of training until all the pictures in the training set are trained.

[0019] Further, the specific implementation steps of the encoder in S2 are as follows:

[0020] S2.2.1: Construct multiple SSLA transformer blocks according to the encoder depth. In each block, perform window attention calculation, normalization, and multi-layer perceptron (MLP) processing operations. In the SLA module, alternately use different window shift sizes for window multi-head self-attention mechanism and shifted window multi-head self-attention mechanism operations, and at the same time perform stochastic depth processing;

[0021] S2.2.2: Process the input features through the SSLA transformer blocks in sequence;

[0022] S2.2.3: Perform downsampling operations on the features, halving the resolution and doubling the feature dimension to achieve multi-scale feature learning; After passing through the Patch Merging layer, the divided small blocks are merged again.

[0023] Further, the specific operation of S2.2.1 is as follows:

[0024] S2.2.1.1: Normalization: First, perform layer normalization on the input features;

[0025] S2.2.1.2: SLA module: Select the corresponding ChannelAttention and SpatialAttention modules according to different numbers of channels respectively; Divide the input feature map x into two parts on the height and width directions respectively, apply the corresponding channel attention mechanism to the generated four parts, and finally complete the splicing, combining the results of local processing and global processing to output the final feature map;

[0026] S2.2.1.3: Window partitioning and attention calculation: Use the window multi-head self-attention mechanism (W-MSA) and the shifted window multi-head self-attention mechanism (SW-MSA) for feature learning; W-MSA divides the feature map into multiple non-overlapping windows and calculates the multi-head self-attention within each window to capture local feature information; SW-MSA performs a shift operation on the windows so that the windows can cover different local regions to learn the relationship between features at different positions;

[0027] S2.2.1.4: Before and after the attention calculation, perform normalization operations on the features using layer normalization, and at the same time use a multi-layer perceptron to perform non-linear transformation on the features to further enrich the feature representation.

[0028] Further, the specific implementation steps of the adaptive connection layer ASCF are as follows:

[0029] S2.3.1: Take the shallow feature x and the deep feature skip as inputs, convert their dimensions and splice them;

[0030] S2.3.2: Rearrange and reshape the splicing features, use average pooling, then perform 1x1 convolution and Sigmoid activation function. The 1x1 convolution is used to fuse channel information, and the Sigmoid function maps the output value to the interval [0,1] to generate attention weights, which are multiplied by the original features to obtain the attention-weighted features;

[0031] S2.3.3: Process the features through a 1x1 convolution layer to further fuse channel information. At the same time, transform x and skip through 1x1 convolutions respectively, add them up, and generate another attention weight through the Sigmoid function, which is multiplied by the features obtained in S2.3.2 to update the features;

[0032] S2.3.1: Reshape and transform the updated features to ensure that the final output has the same shape as the input x, realizing feature fusion and information interaction, and enhancing the feature representation ability.

[0033] Furthermore, the specific steps of the decoder are as follows:

[0034] S2.4.1: Before passing through three layers of SSLA transformers, the features input to each layer are the fusion of features at different levels through the ASCF layer;

[0035] S2.4.2: The specific steps of the upsampling module ExpandDwUp are as follows:

[0036] S2.4.2.1: Expand the dimensions of the features output by the SSLA transformer layer in the decoder stage, expanding their dimensions from C to 2*C;

[0037] S2.4.2.2: Use upsampling to double the resolution of the feature map, then perform feature extraction through depthwise separable convolution, and then use batch normalization and activation functions for processing to obtain a feature map with expanded resolution and after feature extraction and nonlinear transformation;

[0038] S2.4.2.3: Use the channel_shuffle function to rearrange the channels to enhance the information interaction between channels;

[0039] S2.4.2.4: Use 1x1 convolution to adjust the feature dimensions, perform shape transformation and normalization operations on the feature map, and finally obtain the upsampled and dimension-adjusted feature map.

[0040] Furthermore, in step S2.5, when calculating the loss between the detection result and the actual result, a hybrid loss function is used for calculation. The hybrid loss function model is L total = αL BCE +(1 - α)L Dice; where α is the weight parameter of the hybrid loss function, and L BCE is the binary cross-entropy loss function, and L Dice is the Dice loss function;

[0041] The functional expression of L Dice is as follows:

[0042]

[0043] where X represents the ground truth value and Y represents the segmentation result;

[0044] The functional expression of L BCE is as follows:

[0045] L BCE =-∑ (r，c) [G (r，c) log(S (r，c) )+(1 - G (r，c) )log(1 - S (r，c) )]

[0046] where G(r, c) is the ground truth value of the pixel at coordinate (r, c), and S(r, c) is the predicted value of the pixel at coordinate (r, c);

[0047] The Dice coefficient formula is as follows:

[0048]

[0049] where TP: the number of pixels correctly predicted as the foreground class; FN: the number of pixels that are actually the foreground class but are mispredicted as the background class; FP: the number of pixels that are actually the background class but are mispredicted as the foreground class.

[0050] Furthermore, the specific steps of S1 are as follows:

[0051] S1.1: Select the Luna16 dataset;

[0052] S1.2: According to the annotation information, generate a mask for image and annotation processing, perform format conversion, image slicing, resize, and scale to a size of 224×224;

[0053] S1.3: Perform data augmentation, randomly shuffle the lung CT image dataset after the data augmentation operation, and divide it into a training set, a test set, and a validation set according to a ratio of 8:1:1.

[0054] Furthermore, Stochastic Gradient Descent (SGD) is used as the optimization algorithm for the model. By updating the model parameters along the opposite direction of the gradient, the loss function is minimized, and the model parameters are optimized. The initial learning rate is set to 0.001, the total number of training rounds is set to 300, and the batch size is 24.

[0055] The beneficial effects of the present invention are as follows:

[0056] Applying the ASSLA-Unet network (Adaptive Stratum-Scale-Location Attention Unet) as the segmentation model can effectively improve the segmentation accuracy of pulmonary nodules. The ASSLA-Unet adopts a multi-scale feature fusion SLA module (Scale-Local-Attention) and an adaptive hierarchical connection ASCF module (Adaptive Spatial-Channel-Feature) in the encoder and decoder, thus better connecting the shallow feature map and the deep feature map, and playing a role in more completely extracting the features of pulmonary nodules. At the same time, an efficient upsampling module, the ExpandDwUp module (Expand Depthwise Up-sampling), is designed in the decoder, maintaining good detail recovery ability, and thus achieving a better segmentation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic structural diagram of the pulmonary nodule segmentation algorithm based on multi-size and multi-level feature fusion in the embodiment of the present invention;

[0058] Figure 2 It is a network structure diagram of the multi-scale feature fusion SLA module in the present invention;

[0059] Figure 3 It is a network structure diagram of the adaptive hierarchical connection ASCF module in the present invention;

[0060] Figure 4 It is a network structure diagram of the efficient upsampling model ExpandDwUp module in the present invention;

[0061] Figure 5 It is a comparison chart of the segmentation results of the ASSLA-Unet model and other open-source models for pulmonary nodules for samples with four different nodule sizes. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] The present invention will be described in detail below with reference to the drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and the detailed implementation manners and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments.

[0063] A lung nodule image segmentation method based on multi - size and multi - level feature fusion provided by the present invention includes the following steps:

[0064] S1: Obtain a lung CT image dataset, preprocess the lung CT image dataset, divide the preprocessed images into a training set, a test set, a validation set according to the ratio of 8:1:1, and prepare the experimental environment;

[0065] The specific steps of S1 include:

[0066] S1.1: Select the Luna16 dataset;

[0067] S1.2: According to the annotation information, generate a mask for image and annotation processing, perform format conversion, image slicing, resize, and scale to a size of 224×224;

[0068] S1.3: Perform data augmentation, randomly shuffle the lung CT image dataset after the data augmentation operation, and divide it into a training set, a test set, and a validation set according to the ratio of 8:1:1;

[0069] S2: Build a segmentation model based on multi - size and multi - level feature fusion, and use the training set to train the segmentation model; after each round of training is completed, save the parameters of the segmentation model in this round, calculate the loss by comparing the detection results and the actual results obtained by the segmentation model, and adjust the parameters of the segmentation model according to the loss;

[0070] In step S2, use the ASSLA - Unet network as the segmentation model, and the network architecture is as Figure 1 shown. The ASSLA - Unet network adopts an encoder - decoder architecture. The encoder is based on the hierarchical window attention of the SSLA Transformer, and the decoder gradually restores the spatial resolution through the efficient upsampling module ExpandDwUp module. An adaptive skip connection module ASCF module is added between the encoding path and the decoding path. The specific process of using the ASSLA - Unet network for segmentation model training is as follows:

[0071] S2.1: Input the images in the training set into the encoder. First, divide each input image into non - overlapping small blocks through the PatchPartition layer (refer to Figure 1 ), and each divided small block will be linearly projected into a vector with a fixed length by the Linear Embedding layer.

[0072] S2.2: In the encoder, in the Swin transformer [1]Add the SLA module (Swin transformer includes normalization, window partitioning and attention calculation, residual connection and feed-forward network, and the SLA module is added after normalization and before attention calculation), named SSLA transformer. The vector obtained in step S2.1 passes through the SSLA transformer and is downsampled to extract the shallow feature map. The SLA module enhances the capture of local region features important for pulmonary nodule segmentation by combining channel attention and spatial attention mechanisms.

[0073] [1].Liu Z,Lin Y,Cao Y,et al.Swin transformer:Hierarchical vision transformer using shifted windows[C] / / Proceedings of the IEEE / CVF International conference on computer vision.2021:10012-10022.

[0074] The specific implementation steps of the encoder are as follows:

[0075] S2.2.1: Construct multiple SSLA transformer blocks according to the encoder depth. In each block, perform window attention calculation, normalization, and multilayer perceptron (MLP) processing operations. The SLA module alternately uses different window shift sizes for window multi-head self-attention mechanism and shifted window multi-head self-attention mechanism operations, and at the same time performs stochastic depth processing.

[0076] Specific operations:

[0077] S2.2.1.1: Normalization: First, perform layer normalization on the input features;

[0078] S2.2.1.2: SLA module: According to different numbers of channels (96, 192, 384, 768), select the corresponding (select the corresponding input dimension according to the number of channels) ChannelAttention (channel attention mechanism, denoted as CA in the Figure 2 architecture diagram) and SpatialAttention (spatial attention mechanism, in Figure 2The (denoted as SA in the architecture diagram) module. The input feature map x is evenly divided into two parts in the height and width directions respectively. The corresponding channel attention mechanism is applied to the four generated parts, and finally, they are concatenated. The results of local (the four split parts) processing and global (the complete unsplit feature map) processing are combined to output the final feature map. This operation allows the model to extract local features using the SLA module before performing attention calculation, thereby enhancing the model's perception ability of local information.

[0079] S2.2.1.3: Window Partitioning and Attention Calculation: Use the window multi-head self-attention mechanism W-MSA and the shifted window multi-head self-attention mechanism SW-MSA for feature learning. W-MSA divides the feature map into multiple non-overlapping windows and calculates the multi-head self-attention within each window to capture local feature information. SW-MSA performs a shifting operation on the windows (controlled by the window size) so that the windows can cover different local regions to learn the relationships between features at different positions, avoid the isolation of information between windows, and enhance the ability to capture global information.

[0080] S2.2.1.4: Before and after attention calculation, use layer normalization to normalize the features, and at the same time use a multi-layer perceptron (MLP) to perform a non-linear transformation on the features to further enrich the feature representation.

[0081] S2.2.2: Process the input features through the SSLA transformer block in sequence;

[0082] S2.2.3: Perform downsampling operations on the features, halving the resolution and doubling the feature dimension to achieve multi-scale feature learning. After passing through the Patch Merging layer, the divided small blocks are recombined.

[0083] S2.3: Between the encoder and the decoder, through the adaptive hierarchical connection layer ASCF (the architecture diagram is as Figure 3 shown), the shallow features extracted by the encoder and the deep features of the decoder are fused layer by layer. The ASCF module is added. The module dynamically adjusts the fusion weights according to the specific content of the input feature map, obtains global information using adaptive average pooling, and combines the local feature map.

[0084] The specific implementation steps of the adaptive connection layer ASCF are as follows:

[0085] S2.3.1: Take the shallow features x and the deep features skip as inputs, convert their dimensions and concatenate them.

[0086] S2.3.2: Rearrange and reshape the splicing features, use average pooling, then pass through a 1x1 convolution and a Sigmoid activation function. The 1x1 convolution is used to fuse channel information, and the Sigmoid function maps the output value to the interval [0,1] to generate attention weights, which are multiplied by the original features to obtain the attention-weighted features.

[0087] S2.3.3: Process the features through a 1x1 convolution layer to further fuse channel information. At the same time, convert x and skip through 1x1 convolutions respectively, add them together, and then generate another attention weight through the Sigmoid function, which is multiplied by the features obtained in S2.3.2 to update the features.

[0088] S2.3.1: Reshape and transform the updated features to ensure that the final output has the same shape as the input x, realizing feature fusion and information interaction, and enhancing the feature representation ability.

[0089] S2.4: In the decoder, after the SSLA transformer, add an efficient upsampling module ExpandDwUp (Expand Depthwise Up-sampling) (the architecture diagram is as Figure 4 shown), upsample the fused features layer by layer, and finally output the lung nodule segmentation image; add the efficient upsampling module ExpandDwUp to restore high-quality details.

[0090] The specific steps of the decoder are as follows:

[0091] S2.4.1: Before passing through three layers of SSLA transformers, the features input to each layer are the fusion of features at different levels through the ASCF layer.

[0092] S2.4.2: The specific steps of the upsampling module ExpandDwUp are as follows:

[0093] S2.4.2.1: Expand the dimensions of the features output by the SSLA transformer layer in the decoder stage, expanding its dimensions from C to 2*C;

[0094] S2.4.2.2: Use upsampling to double the resolution of the feature map, then perform feature extraction through depthwise separable convolution, and then use batch normalization and activation functions for processing to obtain a feature map with increased resolution and after feature extraction and non-linear transformation.

[0095] S2.4.2.3: Use the channel_shuffle function to rearrange the channels, enhancing the information interaction between channels, which is helpful for feature diversity and information fusion.

[0096] S2.4.2.4: Adjust the feature dimension using 1x1 convolution, transform the shape of the feature map, and perform a normalization operation to finally obtain an upsampled and dimension-adjusted feature map.

[0097] S2.5: Calculate the loss between the lung nodule segmentation image output in step S2.4 and the actual lung nodule segmentation image through a hybrid loss function, adjust the parameters in the ASSLA-Unet network, update the ASSLA-Unet network model, return to step S2.1 for a new round of training until all images in the training set are trained.

[0098] In step S2.5, when calculating the loss between the detection result and the actual result, a hybrid loss function is used for calculation. The hybrid loss function model is L total = αL BCE + (1 - α)L Dice . Among them, α is the weight parameter of the hybrid loss function, L BCE is the binary cross-entropy loss function, and L Dice is the Dice loss function.

[0099] The functional expression of L Dice is as follows:

[0100]

[0101] Among them, X represents the true value, and Y represents the segmentation result.

[0102] The functional expression of L BCE is as follows:

[0103] L BCE = -∑ (r，c) [G (r，c) log(S (r，c) ) + (1 - G (r，c) )log(1 - S (r，c) )]

[0104] Among them, G(r, c) is the true value of the pixel at coordinate (r, c), and S(r, c) is the predicted value of the pixel at coordinate (r, c);

[0105] Dice coefficient: As a measurement method, it is mainly used to measure the similarity between the segmentation result and the true label. The formula is as follows:

[0106]

[0107] HD95 is a metric used to evaluate the boundaries of image segmentation. It measures the 95th percentile of the maximum distance between the predicted segmentation boundary and the true label boundary. Compared with the standard Hausdorff Distance (HD), HD95 reduces the influence of a very small number of outliers, making the evaluation more stable and representative. The lower the HD95 value, the better the segmentation effect.

[0108] Among them, TP (True Positives): The number of pixels correctly predicted as the foreground class. FN (False Negatives): The number of pixels that are actually the foreground class but are mispredicted as the background class. FP (False Positives): The number of pixels that are actually the background class but are mispredicted as the foreground class. TN (True Negatives): The number of pixels correctly predicted as the background class.

[0109] S3: Read the optimal parameters saved in the training stage, and substitute the optimal parameters into the segmentation model to obtain the optimal pulmonary nodule segmentation model;

[0110] S4: Use the test set to test the optimal pulmonary nodule segmentation model, and evaluate the model accuracy of the optimal pulmonary nodule segmentation model using evaluation metrics;

[0111] When conducting the experiment specifically, this work uses SGD (Stochastic Gradient Descent) as the optimization algorithm for the model. This is a classic optimization algorithm that updates the model parameters along the opposite direction of the gradient to minimize the loss function and optimize the model parameters; set the initial learning rate to 0.001, the total number of training rounds to 300, and the batch size to 24.

[0112] In this experiment, the Pytorch framework is selected to implement the proposed model. The Pytorch framework has the significant advantages of intuitive and easy-to-understand source code and simple and efficient design. These characteristics make it an ideal model for construction. The specific software parameters involved in the experiment are shown in Table 1.

[0113] Table 1 Software Parameters

[0114]

[0115] Furthermore, in order to prove that the SSLA module, ASCF module, and ExpandDwUp in ASSLA-Unet have a positive impact on the accuracy of pulmonary nodule segmentation, this embodiment sets corresponding ablation experiments to complete the effectiveness test of each module; the test results are shown in Table 2.

[0116] Table 2 Ablation experiments of ASSLA-Unet

[0117]

[0118] Table 2 shows the lung nodule segmentation accuracy after the model is processed by adding different modules at different positions on the basis of the SwinUnet model. These indicators are evaluated on the Luna16 dataset; the basic model 1 is the SwinUnet model. Compared with the basic model 1, when SSLA, SACF, and ExpandDwUp are gradually integrated into the benchmark model, each indicator is improved to varying degrees. After integrating all three models into the benchmark model, the improvement of each indicator reaches the optimal, the Dice coefficient increases by 9%, and the HD95 decreases by 0.9.

[0119] To determine the best value of the α weight in the hybrid loss function designed in the present invention, different α values are set in the experiment, and ASSLA-Unet is trained and evaluated. By observing the performance of ASSLA-Unet on the test set, the best-performing value is selected as the weight parameter α in the loss function; the influence of different α values on the model segmentation effect is shown in Table 3.

[0120] Table 3 Influence of different α values on the segmentation effect

[0121]

[0122] As can be seen from Table 3, the value of α will have some influence on the segmentation accuracy of the model; when the value of α is 0.4, the evaluation indicators of each category reach the best. Therefore, this work selects 0.4 as the value of α.

[0123] The above-described embodiments only represent one implementation manner of the present invention, and its description is relatively specific and detailed, but it cannot be understood as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the appended claims.

Claims

1. A lung nodule segmentation method based on multi - size and multi - level feature fusion, characterized in that, It includes the following steps: S1: Obtain the lung CT image dataset, preprocess the lung CT image dataset, divide the preprocessed images into a training set, a test set, a validation set, and prepare the experimental environment; S2: Construct a lung nodule segmentation model based on multi-size multi-level feature fusion, and use the training set to train the segmentation model; after each round of training is completed, save the parameters of the segmentation model for this round, calculate the loss by comparing the detection results obtained by the segmentation model with the actual results, and adjust the parameters of the segmentation model according to the loss; In step S2, the ASSLA-Unet network is used as the segmentation model. The ASSLA-Unet network adopts an encoder-decoder architecture. The encoder is based on the hierarchical window attention of the SSLA Transformer, and the decoder gradually restores the spatial resolution through the upsampling module ExpandDwUp module. An adaptive skip connection module ASCF module is added between the encoding path and the decoding path; the SSLA Transformer is the Swin transformer with the SLA module added; S3: Read the optimal parameters saved in the training stage, and substitute the optimal parameters into the segmentation model to obtain the optimal lung nodule segmentation model; S4: Use the test set to test the optimal lung nodule segmentation model, and use evaluation metrics to evaluate the model accuracy of the optimal lung nodule segmentation model.

2. The lung nodule segmentation method based on multi-size multi-level feature fusion according to claim 1, wherein The specific steps of S2 are as follows: S2.1: Input the images in the training set into the encoder. First, each input image is divided into non-overlapping small blocks through the Patch Partition layer, and each divided small block will be linearly projected into a vector with a fixed length by the Linear Embedding layer; S2.2: In the encoder, add the SLA module, that is, the SSLA transformer, after normalization and before attention calculation in the Swin transformer. The vector obtained in step S2.1 passes through the SSLA transformer and undergoes downsampling to extract the shallow feature map; S2.3: Between the encoder and the decoder, through the adaptive hierarchical connection layer ASCF, fuse the shallow features extracted by the encoder and the deep features of the decoder layer by layer; S2.4: In the decoder, after the SSLA transformer, add the upsampling module ExpandDwUp, and upsample the fused features layer by layer, and finally output the lung nodule segmentation image; S2.5: Calculate the loss between the lung nodule segmentation image output in step S2.4 and the actual lung nodule segmentation image through the hybrid loss function, adjust the parameters in the ASSLA-Unet network, update the ASSLA-Unet network model, and return to step S2.1 for a new round of training until all the pictures in the training set are trained.

3. The lung nodule segmentation method based on multi-size multi-level feature fusion according to claim 1, wherein The specific implementation steps of the encoder in S2 are as follows: S2.2.1: Construct multiple SSLA transformer blocks according to the encoder depth. In each block, perform window attention calculation, normalization, and multi-layer perceptron (MLP) processing operations. In the SLA module, alternately use different window shift sizes for window multi-head self-attention mechanism and shifted window multi-head self-attention mechanism operations, and at the same time perform stochastic depth processing; S2.2.2: Process the input features through the SSLA transformer blocks in sequence; S2.2.3: Perform downsampling on the features, halving the resolution and doubling the feature dimension to achieve multi-scale feature learning; After passing through the Patch Merging layer, the divided small blocks are merged again.

4. The lung nodule segmentation method based on multi-size multi-level feature fusion according to claim 3, characterized in that, The specific operation of S2.2.1 is as follows: S2.2.1.1: Normalization: First, perform layer normalization on the input features; S2.2.1.2: SLA module: Select the corresponding ChannelAttention and SpatialAttention modules according to different numbers of channels respectively; Divide the input feature map x into two equal parts in the height and width directions respectively, apply the corresponding channel attention mechanism to the four generated parts, and finally complete the splicing, combining the results of local processing and global processing to output the final feature map; S2.2.1.3: Window partitioning and attention calculation: Use the window multi-head self-attention mechanism (W-MSA) and the shifted window multi-head self-attention mechanism (SW-MSA) for feature learning; W-MSA divides the feature map into multiple non-overlapping windows and calculates multi-head self-attention within each window to capture local feature information; SW-MSA performs a shift operation on the windows so that the windows can cover different local regions to learn the relationship between features at different positions; S2.2.1.4: Before and after the attention calculation, use layer normalization to normalize the features, and at the same time use a multi-layer perceptron to perform a non-linear transformation on the features to further enrich the feature representation.

5. The lung nodule segmentation method based on multi-size multi-level feature fusion according to claim 1, wherein The specific implementation steps of the adaptive connection layer ASCF are as follows: S2.3.1: Take the shallow feature x and the deep feature skip as inputs, convert their dimensions and splice them; S2.3.2: Perform dimension rearrangement and reshaping on the spliced features, use average pooling, then pass through a 1x1 convolution and a Sigmoid activation function. The 1x1 convolution is used to fuse channel information, and the Sigmoid function maps the output value to the interval [0,1] to generate an attention weight, which is multiplied by the original feature to obtain the attention-weighted feature; S2.3.3: Process the features through a 1x1 convolution layer to further fuse channel information. At the same time, x and skip are respectively converted by 1x1 convolutions, added and then passed through the Sigmoid function to generate another attention weight, which is multiplied by the feature obtained in S2.3.2 to update the feature; S2.3.1: Reshape and transform the updated features to ensure that the final output has the same shape as the input x, realizing feature fusion and information interaction, and enhancing the feature representation ability.

6. The lung nodule segmentation method based on multi-size multi-level feature fusion according to claim 1, characterized in that, The specific steps of the decoder are as follows: S2.4.1: Before passing through the three-layer SSLA transformer, the input features of each layer are fused by the features of different levels through the ASCF layer; S2.4.2: The specific steps of the upsampling module ExpandDwUp are as follows: S2.4.2.1: In the decoder stage, the features output by the SSLA transformer layer are dimensionally expanded from C to 2*C. S2.4.2.2: Use upsampling to increase the resolution of the feature map by a factor of 2, then perform feature extraction using depthwise separable convolution, followed by batch normalization and activation functions to obtain a feature map with increased resolution, feature extraction, and nonlinear transformations. S2.4.2.3: Use the channel_shuffle function to rearrange channels to enhance information interaction between channels; S2.4.2.4: Use 1x1 convolution to adjust the feature dimension, perform shape transformation and normalization on the feature map, and finally obtain the upsampled and dimension-adjusted feature map.

7. The lung nodule segmentation method based on multi - size multi - level feature fusion according to claim 2, wherein In step S2.5, when calculating the loss between the detection result and the actual result, a hybrid loss function is used for calculation. The hybrid loss function model is: L total = αL BCE + (1 - α)L Dice ; where α is the weight parameter of the hybrid loss function, L BCE is the binary cross-entropy loss function, and L Dice is the Dice loss function; L Dice The functional expression is as follows: Among them, X represents the true value and Y represents the segmentation result; L BCE The function expression is as follows: L BCE = -∑ (r，c) [G (r，c) log(S (r，c) ) + (1 - G (r，c) ) log(1 - S (r，c) )] Among them, G(r,c) is the true value of the pixel at coordinate (r,c), and S(r,c) is the predicted value of the pixel at coordinate (r,c); The Dice coefficient formula is as follows: Among them, TP: the number of pixels correctly predicted as foreground categories; FN: the number of pixels that are actually foreground categories but incorrectly predicted as background categories; FP: the number of pixels that are actually background categories but incorrectly predicted as foreground categories.

8. The lung nodule segmentation method based on multi-size multi-level feature fusion according to claim 1, wherein The specific steps of S1 are: S1.1: Use Luna16 dataset; S1.2: Based on the annotation information, the image and annotation are processed to generate a mask, the format is converted, the image is sliced, the size is adjusted, and the image is scaled to 224×224 size; S1.3: Perform data augmentation and randomly shuffle the lung CT image dataset after the data augmentation operation, dividing it into training set, test set, and validation set in a ratio of 8:1:

1.

9. The lung nodule segmentation method based on multi-size multi-level feature fusion according to claim 1, characterized in that Use SGD stochastic gradient descent as the model optimization algorithm, and update the model parameters in the opposite direction of the gradient to minimize the loss function and optimize the model parameters; The initial learning rate is set to 0.001, the total number of training rounds is set to 300, and the batch size is set to 24.

Images