Context feature aggregation pulmonary nodule segmentation method combining edge perception and multi-scale semantic guidance

Abstract

The invention provides a context feature aggregation pulmonary nodule segmentation method combining edge perception and multi-scale semantic guidance, and the method comprises the steps: obtaining a to-be-segmented pulmonary nodule CT image, inputting the to-be-segmented pulmonary nodule CT image into a trained EGP-Net network model, and obtaining a segmentation result; wherein in the EGP-Net network model, multilayer features of an encoder are sent to a global pyramid guide module to reconstruct and selectively introduce deep global semantics back to a shallow layer, and low-layer features are merged into an edge guide network to extract and reinforce fine-grained boundary information; the attention feature fusion module performs weighted attention fusion on global semantics and local edge features, and the multi-scale context decoder fuses multi-scale information through lightweight reconstruction such as sub-pixel convolution to generate a fine segmentation map. Experimental results show that the EGP-Net network model is superior to an existing advanced method in segmentation precision and boundary consistency, and has good clinical auxiliary diagnosis and precision medical application potential.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method for lung nodule segmentation that combines edge perception and multi-scale semantic guidance with contextual feature aggregation. Background Technology

[0002] Automatic and accurate segmentation of lung nodules in CT images is helpful for the early diagnosis and treatment of lung cancer. Currently, medical image analysis technology is widely used in the field of lung nodule segmentation, mainly including traditional segmentation methods and deep learning-based segmentation methods.

[0003] Traditional segmentation methods face challenges such as missing margins and false positives when handling different types of pulmonary nodule lesions. This is because the low gray-scale and texture contrast around the lesion and the blurred edge transitions make it difficult to accurately delineate the boundary, leading to diagnostic difficulties. Furthermore, traditional pulmonary nodule segmentation algorithms are often semi-automated, requiring significant human intervention to guide the segmentation process. This approach has limited versatility and is not well-suited for a wider range of applications.

[0004] Deep learning methods, through encoder-decoder structures and backpropagation algorithms, can automatically learn image features without human intervention, offering higher segmentation accuracy, robustness, and generalization ability, thus becoming the mainstream approach. While deep learning methods have achieved some success in automatic lung nodule segmentation in recent years, existing methods still face several challenges. First, traditional skip connections often weaken deep global semantic information when transferring it to shallower layers, and direct concatenation can easily introduce irrelevant features, creating semantic gaps. Therefore, effectively guiding global semantics in skip connections and enhancing information interaction between the encoder and decoder is a worthwhile area of ​​research. Second, lung nodule edges are often blurred and connected to structures such as blood vessels and bronchi, and low-level features retain rich boundary information. Therefore, effectively extracting and strengthening these edge features while reducing background interference is a significant challenge for improving segmentation accuracy. Third, simply fusing features from different semantic levels through simple linear operations is insufficient to fully model the complex complementary relationship between global context and local edge features. Calculating feature weights during feature fusion to achieve weighted fusion of semantic information at different levels is crucial for achieving fine-grained segmentation. Therefore, it is crucial to optimize the effect of transfer learning and to accurately segment lung nodules by introducing a specially designed segmentation network. Summary of the Invention

[0005] In view of the above, it is necessary to provide a lung nodule segmentation method that combines edge perception and multi-scale semantic guidance with contextual feature aggregation. This method can extract more efficient lesion features from lung nodule CT images and achieve more accurate segmentation performance.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A lung nodule segmentation method combining edge awareness and multi-scale semantic guidance with contextual feature aggregation includes the following steps:

[0008] The CT image of the lung nodule to be segmented is obtained, and the CT image of the lung nodule to be segmented is input into a trained lung nodule segmentation network model that combines edge perception and multi-scale semantic guidance to aggregate contextual features, and the segmentation result is obtained.

[0009] The context feature aggregation lung nodule segmentation network model is pre-built and trained, and the specific method is as follows:

[0010] S1, collect a dataset of historical lung nodule CT images and preprocess the lung nodule CT images in the dataset;

[0011] S2, randomly divide the processed dataset into training set, validation set and test set according to the proportion;

[0012] S3, Construct the context feature aggregation lung nodule segmentation network model:

[0013] S3.1, the input lung nodule CT image is preliminarily processed by a 1*1 convolutional block to obtain preliminary features, which are denoted as E. 1×1 This initial feature is then passed to the encoder network and the edge guidance network;

[0014] S3.2, perform multi-layer feature extraction in the encoder network to obtain multi-scale features S i , i=0,1,2,3,...,n-1, where n is the total number of stages in the encoder network for multi-layer feature extraction;

[0015] S3.3, edge feature extraction is performed in the edge-guided network to obtain local edge features. The specific method is as follows:

[0016] First, the preliminary feature E obtained in step S3.1 is... 1×1 After being processed by 3×3 convolutional blocks, the low-level features S0 extracted by the encoder network in step S3.2 are then fused with them after max pooling. Subsequently, edge information is gradually extracted and restored to the original spatial resolution through two sets of identical 3×3 convolutional blocks.

[0017] S3.4, extract the multi-scale features S0 and other features S0 extracted by the encoder network in step S3.2, excluding the low-level features S0. i Integrating features into the GPG module, where features from each stage are combined with features from all higher stages to reconstruct skip connections, thereby facilitating the effective capture of global semantic information;

[0018] S3.5, the multi-scale context decoder network uses ESPCN for super-resolution reconstruction to obtain global semantic features;

[0019] S3.6, The local edge features captured by the edge-guided network in step S3.3 and the global semantic features extracted by the multi-scale context decoder in step S3.5 are weighted and fused through the attention feature fusion module to obtain the final segmentation result;

[0020] S4. The context feature aggregation lung nodule segmentation network model constructed in step S3 is trained using the training set divided in step S2, and the context feature aggregation lung nodule segmentation network model is validated using the validation set. Finally, the trained context feature aggregation lung nodule segmentation network model is obtained.

[0021] S5. Introduce the lung nodule CT images from the test set divided in step S2 into the trained context feature aggregation lung nodule segmentation network model to obtain the segmentation results.

[0022] Preferably, the preprocessing step in step S1 is as follows:

[0023] S1.1, Use the Labelme tool to annotate CT images of lung nodules;

[0024] S1.2, adjust the resolution of all lung nodule CT images to 128×128 and process them into grayscale images.

[0025] 3. The lung nodule segmentation method combining edge perception and multi-scale semantic guidance with contextual feature aggregation as described in claim 1, characterized in that, during feature extraction by the encoder network in step S3.2, the operation of Stage 0 in the low-level feature extraction stage skips the initial pooling layer, and the feature resolution of the obtained low-level feature S0 is... Features S obtained from feature extraction in other stages i The feature resolution is , i>0, H and W correspond to the height and width of the feature, respectively.

[0026] Preferably, in step S3.3, the specific steps for the edge feature extraction by the edge-guided network are as follows:

[0027] E 1×1 First, the feature map resolution is compressed to half of its original value using convolutional blocks with a stride of 2, 64 channels, and a 3×3 kernel. Then, batch normalization, ReLU activation, and 2×2 max pooling are performed sequentially, resulting in a final image of size [size missing]. The feature map; simultaneously, max pooling is performed on S0 to obtain The feature maps are obtained; the two feature maps are concatenated along the channel dimension to obtain the fused feature; then, they are processed through two identical sets of convolutional blocks, the outputs of which are denoted as follows: and Each convolutional block contains a 3×3 convolutional layer, batch normalization, ReLU activation, and twice the bilinear interpolation, which improves resolution while halving the number of channels; ultimately, A single-channel boundary feature map F is generated through a 1×1 convolutional layer. out ∈R H×W×1 .

[0028] Preferably, in step S3.4, the specific method by which the GPG module reconstructs the skip connection is as follows:

[0029] First, a feature map F is generated by mapping the features of all stages above stage i to the same channel space as stage i using 3×3 convolutional blocks. j j = i+1,...,n-1;

[0030] Next, the generated feature map F j Upsampled to the feature map F of stage i i Same size and connected;

[0031] Then, multiple separable convolutional blocks with different dilation rates are used in parallel to extract global context information. The separable convolutional blocks are used to reduce model parameters. The number of separable convolutional blocks is n-i+1, and the dilation rates of the multiple separable convolutional blocks are progressive.

[0032] Finally, standard convolution is used to obtain the final feature map.

[0033] Preferably, the specific method for weighted fusion in step S3.6 of the attention feature fusion module is as follows:

[0034] First, perform initial feature fusion on the two input features X and Y, denoted as . ;

[0035] Next, the local branch reduces the number of feature channels in the fused configuration to the original number using a 1x1 point convolution. Then, batch normalization and ReLU activation function are executed sequentially. Then, the number of channels is restored to the same as the original input number of channels by 1*1 point convolution block and batch normalization. This local branch is the local edge feature captured by the edge-guided network. r refers to the channel compression ratio, which is used to control the number of channels.

[0036] Then, the global branch first performs a global average pooling operation on the fused features, and then processes them in the same way as the local branch operation described above; this global branch is the global semantic feature extracted by the multi-scale context decoder;

[0037] After the local and global branches have performed the above operations, they are fused again by adding elements one by one. After passing through the Sigmoid activation function, the calculated weight values ​​are used to perform attention operations on the input features X and Y respectively, and multiplied by their corresponding elements to obtain the outputs X' and Y' respectively.

[0038] Will After performing element-wise addition of X' and Y', the final output Z is obtained.

[0039] In addition, the present invention also provides a method for EGP-Net uncertainty evaluation and training set selection based on model segmentation results, the method comprising the following steps:

[0040] (1) Acquire new lung nodule CT images, input the new lung nodule CT images into the trained EGP-Net model, and make the random term in the step of fusing into the final segmentation probability map masked or set to zero with probability in each inference, and repeat this step T times; the above EGP-Net network model is a lung nodule segmentation network model that combines edge perception and multi-scale semantic guidance of any one of claims 1-6; the above final segmentation probability map step is the step in which the EGP-Net network model performs inference prediction on the newly acquired CT images to obtain the final segmentation result, that is, the stage in which the model generates the output probability map after multi-branch fusion;

[0041] (2) Obtain T final segmentation probability maps, analyze them, and obtain the probability value p of each pixel in T inferences. t,n ;

[0042] (3) Calculate the uncertainty index U of EGP-Net based on the analysis results and the following formula: Let the average pixel probability be The uncertainty is then represented by the average pixel entropy:

[0043]

[0044] Where U is the uncertainty index, T is the number of final segmented images, N is the total number of pixels, and p t,n The probability value of the nth pixel in the t-th final segmentation image, where ln is the natural logarithm to the base e;

[0045] (4) Determine whether the uncertainty index U is greater than the preset threshold U. th , if U>U thIf U ≤ U th If the CT image is deemed suitable as a training set, it will be added to the training set to be trained.

[0046] Preferably, in step (1), setting the random term in the step of fusing into the final segmentation probability map to 0 means that in each inference, any branch of the local branch or global branch of the attention feature fusion module is randomly selected and temporarily set to 0, or Dropout is enabled during inference to introduce randomness.

[0047] Preferably, in step (4), the threshold U th The validation set is determined by the distribution quantiles or mean ± kσ.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. This invention provides a lung nodule segmentation method combining edge perception and multi-scale semantic guidance with contextual feature aggregation. Utilizing lung nodule CT images, after preprocessing, a contextual feature aggregation lung nodule segmentation network model (hereinafter referred to as the EGP-Net network model) is constructed, consisting of an encoder, an edge guidance network, a global pyramid guidance module, an attention feature fusion module, and a multi-scale context decoder. The encoder's multi-layer features are fed into the global pyramid guidance module for reconstruction and selectively guide deep global semantics back to shallow layers; low-level features are incorporated into the edge guidance network to extract and enhance fine-grained boundary information; the attention feature fusion module performs weighted attention fusion of global semantics and local edge features; the multi-scale context decoder generates a fine segmentation map by lightweight reconstruction and fusion of multi-scale information through sub-pixel convolution. In other words, the EGP-Net network model fuses local edge and global multi-scale contextual features. Therefore, the segmentation method of this invention can extract more efficient lesion features from lung nodule CT images, achieving more accurate segmentation performance. It solves the defects of existing lung nodule segmentation methods, such as weakened and redundant semantics of skip connections, blurred edges, and difficulty in modeling complex complementary relationships between global and local features.

[0050] 2. Through experimental comparison and analysis, it can be seen that the EGP-Net of the present invention has better performance indicators and better and more accurate segmentation results compared with existing segmentation models. It can provide more information support for medical experts and help them formulate more effective treatment plans. Attached Figure Description

[0051] Figure 1 This is a flowchart of the process of the present invention.

[0052] Figure 2 This is a diagram of the overall architecture of the present invention.

[0053] Figure 3 This is a schematic diagram of the global pyramid guidance module of the present invention.

[0054] Figure 4 This is a schematic diagram of the attention feature fusion module of the present invention.

[0055] Figure 5 This is a schematic diagram of the basic unit structure of the multi-scale context decoder network of the present invention.

[0056] Figure 6 This is a visualization of the experimental results of the present invention and a schematic diagram comparing its effects with other advanced technologies.

[0057] Figure 7 This table presents a comparison of the lung nodule segmentation performance of the EGP-Net network model of this invention with other advanced network models.

[0058] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0059] Example 1

[0060] Please see Figures 1 to 5 In a preferred embodiment of the present invention, a lung nodule segmentation method combining edge perception and multi-scale semantic guidance with contextual feature aggregation includes the following steps: acquiring a CT image of the lung nodule to be segmented; inputting the CT image of the lung nodule to be segmented into a trained lung nodule segmentation network model combining edge perception and multi-scale semantic guidance with contextual feature aggregation; and obtaining the segmentation result.

[0061] The context feature aggregation lung nodule segmentation network model is pre-built and trained, and the specific method is as follows:

[0062] S1. Collect a dataset of historical lung nodule CT images and preprocess the lung nodule CT images in the dataset. The preprocessing steps are as follows:

[0063] S1.1, Use the Labelme tool to annotate CT images of lung nodules;

[0064] S1.2, adjust the resolution of all lung nodule CT images to 128×128 and process them into grayscale images.

[0065] Specifically, the aforementioned dataset of historical lung nodule CT images is the LIDC dataset collected by the National Cancer Institute, which includes 1,018 lung CT scans, each independently labeled and diagnosed by four senior radiologists, totaling 15,096 images.

[0066] S2 involves randomly dividing the processed dataset into training, validation, and test sets proportionally. For example, a 6:2:2 ratio could be used: 60% training, 20% validation, and 20% test. The 60% training set is used to train the constructed EGP-Net network model. The optimal model parameters are saved based on the results from the 20% validation set. Finally, the 20% test set is used for testing. The tested EGP-Net network model can then be used for segmenting lung nodules. This segmentation method effectively improves segmentation accuracy, aiding in early cancer identification and accurate diagnosis, thereby increasing patient survival rates. In other words, the training and validation sets are used to train the EGP-Net network model obtained in step S3, while the test set is used to test and evaluate the accuracy of the EGP-Net network model.

[0067] S3, Construct the context feature aggregation lung nodule segmentation network model, hereinafter referred to as the EGP-Net network model:

[0068] S3.1, the input lung nodule CT image is preliminarily processed by a 1*1 convolutional block to obtain preliminary features, which are denoted as E. 1×1 This initial feature is then passed to the encoder network and the edge guidance network.

[0069] S3.2, perform multi-layer feature extraction in the encoder network to obtain multi-scale features S i Let i = 0, 1, 2, 3, ..., n-1, where n is the total number of stages in the encoder network for feature extraction. For example, ... Figure 2 In the embodiment shown, the total number of stages is 5, n=5. The 5 stages are Stage0, Stage1, Stage2, Stage3, and Stage4, and the resulting features from the lower to the higher level are S0, S1, S2, S3, and S4.

[0070] Preferably, in order to better preserve spatial structure information and coordinate changes in different sizes, the encoder network in this embodiment skips the initial pooling layer in the Stage 0 operation of the low-level feature extraction stage when extracting features, and the feature resolution of the resulting low-level feature S0 is: Features S obtained from feature extraction in other stages i The feature resolution is , i>0. For example, after Stage0, a primary feature map x with a shape of 64×64×64 is obtained; then x is sequentially input into Stage1, Stage2, Stage3, and Stage4 to generate features x1, x2, x3, and x4 of Stage1~Stage4, with shapes of 256×64×64, 512×32×32, 1024×16×16, and 2048×8×8, respectively. This layered design mainly involves downsampling in several layers and simultaneously increasing the number of channels.

[0071] S3.3, edge feature extraction is performed in the edge-guided network to obtain local edge features. The specific method is as follows:

[0072] First, the preliminary feature E obtained in step S3.1 is... 1×1 The feature map is processed by 3×3 convolutional blocks, and then fused with the low-level features S0 extracted by the encoder network in step S3.2 after max pooling. Subsequently, edge information is extracted stepwise using two identical sets of 3×3 convolutional blocks and restored to its original spatial resolution. That is, in the edge guidance network, the feature map resolution is halved by convolutions with a kernel size of 3, 64 channels, and a stride of 2, and downsampling is performed through max pooling. Figure 2 As shown.

[0073] For this edge-guided network, this invention considers the problem that lung nodules, due to their small size, often present blurred boundaries in CT images and frequently overlap with surrounding structures such as blood vessels and bronchi, which can easily affect the segmentation results. Effectively extracting edge information is crucial for improving segmentation accuracy. Therefore, an edge-guided network is designed to extract and enhance local edge features, helping the model accurately recover the nodule contour at multiple scales and suppress interference from surrounding structures, thereby improving the boundary accuracy and robustness of the segmentation. Furthermore, in the network design, lower-level features retain richer boundary information; therefore, the model fully utilizes the information from the encoder's lower-level feature layers.

[0074] In this embodiment, the more specific feature extraction method of the edge-guided network is: E 1×1 First, the feature map resolution is compressed to half of its original value using convolutional blocks with a stride of 2, 64 channels, and a 3×3 kernel. Then, batch normalization, ReLU activation, and 2×2 max pooling are performed sequentially, resulting in a final image of size [size missing]. The feature map; simultaneously, max pooling is performed on S0 to obtain The feature maps are obtained; the two feature maps are concatenated along the channel dimension to obtain the fused feature; then, they are processed through two identical sets of convolutional blocks, the outputs of which are denoted as follows: and Each convolutional block contains a 3×3 convolutional layer, batch normalization, ReLU activation, and twice the bilinear interpolation, which improves resolution while halving the number of channels; ultimately, A single-channel boundary feature map F is generated through a 1×1 convolutional layer. out ∈R H×W×1 .

[0075] In other words, to enhance the extraction and utilization of boundary information, this invention designs an edge-guided network specifically for extracting edge features. In the edge-guided network of step S3.3 above, the spatial resolution of the feature map is 32×32 before splicing. Then, through two sets of identical 3×3 convolutional layers, batch normalization, and ReLU activation, the resolution is gradually restored while retaining fine-grained edge information. Finally, a 1*1 convolution is used to restore the feature map to its original resolution, thereby realizing the extraction and utilization of boundary information.

[0076] S3.4, extract the multi-scale features S0 and other features S0 extracted by the encoder network in step S3.2, excluding the low-level features S0. i Integrating features into the GPG module, where each stage's features are combined with features from all higher stages to reconstruct skip connections, thereby facilitating the effective capture of global semantic information. Specifically, the GPG module reconstructs skip connections in the following way:

[0077] First, a feature map F is generated by mapping the features of all stages above stage i to the same channel space as stage i using 3×3 convolutional blocks. j j = i+1,...,n-1;

[0078] Next, the generated feature map F j Upsampled to the feature map F of stage i i Same size and connected;

[0079] Then, multiple separable convolutional blocks with different dilation rates are used in parallel to extract global context information. The separable convolutional blocks are used to reduce model parameters. The number of separable convolutional blocks is n-i+1, and the dilation rates of the multiple separable convolutional blocks are progressive. That is, the number of parallel paths and the dilation rate vary with the number of fusion stages.

[0080] Finally, standard convolution is used to obtain the final feature map.

[0081] Taking Stage 1 as an example, such as Figure 3As shown, this describes the GPG module on Stage 1. First, features from all stages are mapped to the same channel space as Stage 1 using 3×3 convolutional blocks. Next, the generated feature maps F2 (the feature map corresponding to Stage 2), F3 (the feature map corresponding to Stage 3), and F4 (the feature map corresponding to Stage 4) are upsampled to the same size as F1 (the feature map corresponding to Stage 1) and concatenated. Then, to extract global contextual information from the feature maps at different levels, four separable convolutions (DG, D ... sconv @1,D sconv @2,D sconv @4,D sconv @8). Finally, standard convolution is used to obtain the final feature map.

[0082] For the GPG modules shown above, each GPG module at different stages can be summarized by formula (1) (regular convolution is ignored to simplify the formula):

[0083] (1)

[0084] Among them, G k F represents the output of the GPG module inserted at level k. k This represents the feature map of the k-th level in the encoder. 2 i-k C represents an upsampling operation with a rate of 2i-k, and D represents a concatenation operation. sconv @2 i-k This represents a separable dilated convolution with a dilation rate of 2i-k.

[0085] It should be noted that the purpose of employing GPG modules in this invention is to address the issue that, while the encoder can learn global contextual information from the input image, including the object's surrounding environment and category features, this information may gradually weaken as it is transmitted to shallower layers. Furthermore, simply using skip connections introduces irrelevant clutter and creates semantic gaps due to receptive field mismatches. Therefore, the invention avoids these shortcomings by using GPG modules to reconstruct skip connections by combining the feature maps of this stage with those of all higher-level stages. In other words, by introducing multiple GPG modules between the encoder and the multi-scale context decoder, global semantic information from higher-level stages can be progressively guided to different stages.

[0086] In addition, in order to reduce computational costs, only three GPG modules are used in the EGP-Net network model with a five-stage encoder network in this embodiment.

[0087] S3.5, the multi-scale context decoder network uses ESPCN for super-resolution reconstruction to obtain global semantic features; specifically, the three branches guided by the GPG module each use ESPCN (Efficient Subpixel Convolutional Neural Network) for feature reconstruction, and the reconstructed structure is as follows. Figure 5 The structure of this basic unit is shown as follows: First, it uses a sub-pixel convolution layer for upsampling to generate a feature map of size (C / r², r*H, r*W), where r is the upsampling magnification factor, which is the magnification ratio during super-resolution reconstruction. In this implementation, the PixelShuffle upsampling operation layer is used, with r set to 2, which reduces the number of channels to 4 times the original and the size to twice the original, thus improving resolution. Next, a 3×3 convolution block is used to halve the number of channels in the feature map to extract more discriminative features. Finally, batch normalization and ReLU nonlinear activation are used to normalize and activate the features to optimize feature performance and improve model stability, thereby achieving high-quality output.

[0088] In step S3.6, the local edge features captured by the edge-guided network in step S3.3 and the global semantic features extracted by the multi-scale context decoder in step S3.5 are weighted and fused through the attention feature fusion module to obtain the final segmentation result. That is, this invention considers that fusing features at different semantic levels through simple linear operations (e.g., summation or concatenation) is often not the optimal choice. Therefore, the network model of this invention introduces an attention feature fusion module (AFFM) to capture the complex complementary relationships and semantic differences between global semantics and local edge features, fully preserving fine-grained edge information and effectively enhancing the global context, thereby improving the overall segmentation performance of the model.

[0089] In this embodiment, the specific method for weighted fusion in the attention feature fusion module is as follows:

[0090] First, perform initial feature fusion on the two input features X and Y, denoted as . ;

[0091] Next, the local branch reduces the number of feature channels in the fused configuration to the original number using a 1x1 point convolution. Then, batch normalization and ReLU activation function are executed sequentially. Then, the number of channels is restored to the same as the original input number of channels by 1*1 point convolution block and batch normalization. This local branch is the local edge feature captured by the edge-guided network. r refers to the channel compression ratio, which is used to control the number of channels.

[0092] Then, the global branch first performs a global average pooling operation on the fused features, and then processes them in the same way as the local branch; that is, the processing method of the global branch is the same as that of the local branch, the only difference being that a global average pooling (GAP) operation is performed on the fused features first. This global branch is the global semantic feature extracted by the multi-scale context decoder.

[0093] After the local and global branches have performed the above operations, they are fused again by adding elements one by one. After passing through the Sigmoid activation function, the calculated weight values ​​are used to perform attention operations on the input features X and Y respectively, and multiplied by their corresponding elements to obtain the outputs X' and Y' respectively.

[0094] Will After performing element-wise addition of X' and Y', the final output Z is obtained.

[0095] The above operations are as follows Figure 4 As shown. For the above operations, the calculation formulas for channel attention of local features L(X), channel attention of global features L(Y), outputs X' and Y' obtained after X and Y attention, and the calculation formula Z of AFFM are shown in formulas (2), (3), (4), (5), and (6).

[0096] (2)

[0097] (3)

[0098] (4)

[0099] (5)

[0100] (6)

[0101] In this context, PWConv1 and PWConv2 both represent 1*1 point convolutions, BN represents a BatchNorm layer, Re represents the ReLU activation function, and GAP represents global average pooling. This indicates element-wise addition. This represents element-wise multiplication, where X and Y are weighted averages. Indicates the weight.

[0102] S4. The context feature aggregation lung nodule segmentation network model constructed in step S3 is trained using the training set divided in step S2, and the context feature aggregation lung nodule segmentation network model is validated using the validation set. Finally, the trained context feature aggregation lung nodule segmentation network model is obtained.

[0103] S5. Introduce the lung nodule CT images from the test set divided in step S2 into the trained context feature aggregation lung nodule segmentation network model to obtain the segmentation results.

[0104] verify

[0105] To demonstrate the performance of the EGP-Net network model of this invention, during the training phase on the LIDC public dataset, the EGP-Net network model of this invention was trained for 100 epochs with a batch size of 8 and an initial learning rate of 5×10⁻⁶. −5 A cosine annealing learning rate decay strategy is used, with the learning rate gradually decreasing to 5×10 per epoch. −6 Select the optimizer Adam, with weights decayed to 1×10. −4 The loss function used is a fusion loss function of Dice coefficient loss and binary cross-entropy loss. , where λ is the equilibrium and The coefficient, which is set to 0.5 in this invention). During testing, it was used with multiple related networks (specifically...). Figure 6 and Figure 7 This study compares the network with other networks besides "EGP-Net" of this invention, and uses multiple evaluation metrics (IoU, Dice coefficient, and Fβ-score) for comparative analysis, as follows: Figure 7 As shown.

[0106] Comprehensive analysis Figure 7 It is evident that this proposed method outperforms existing segmentation models in various parameters. The lung nodule segmentation method proposed in this invention, which combines edge awareness and multi-scale semantic guidance with contextual feature aggregation, can provide medical experts with more information to support their development of more effective treatment plans. Compared to existing advanced segmentation models, the EGP-Net network model offers higher performance while also having advantages in computational cost.

[0107] In addition, from Figures 6-7 As can be seen from the research results, when using the EGP-Net network model of this invention to segment CT images of lung nodules, it can extract more effective features from the CT images of lung nodules, thereby achieving more accurate segmentation of lesion points. In summary, the EGP-Net network model proposed in this invention has advantages over existing segmentation methods.

[0108] Example 2

[0109] In addition, this invention also provides a method for EGP-Net uncertainty evaluation and training set selection based on model segmentation results, the method comprising the following steps:

[0110] (1) Acquire new lung nodule CT images, input the new lung nodule CT images into the trained EGP-Net model, and make the random term in the step of fusing into the final segmentation probability map masked or set to zero with probability in each inference, and repeat this step T times; the above EGP-Net network model is a lung nodule segmentation network model that combines edge perception and multi-scale semantic guidance of any one of claims 1-6; the above final segmentation probability map step is the step in which the EGP-Net network model performs inference prediction on the newly acquired CT images to obtain the final segmentation result, that is, the stage in which the model generates the output probability map after multi-branch fusion.

[0111] Specifically, setting the random term in the step of fusing into the final segmentation probability map to 0 means that in each inference, any branch of the attention feature fusion module, either a local branch or a global branch, is randomly selected and temporarily set to 0 to mask the covered branch, or Dropout is enabled during inference to introduce randomness. The inference refers to the inference testing phase of the model, which uses the best model parameters saved during model validation.

[0112] In addition, it should be noted that the number of repetitions is set by those skilled in the art according to actual needs. The more repetitions, the more accurate the estimation of uncertainty, but the longer the calculation time required.

[0113] (2) Obtain T final segmentation probability maps, analyze them, and obtain the probability value p of each pixel in T inferences. t,n The above analysis specifically involves statistically analyzing the probability distribution of each pixel under different inferences at the pixel level in the segmentation probability map obtained from T inferences, and calculating its average probability to provide basic data for subsequent uncertainty entropy calculation.

[0114] (3) Calculate the uncertainty index U of EGP-Net based on the analysis results and the following formula: Let the average pixel probability be The uncertainty is then represented by the average pixel entropy:

[0115]

[0116] Where U is the uncertainty index, T is the number of final segmented images, N is the total number of pixels, and p t,n The probability value of the nth pixel in the t-th final segmentation image, where ln is the natural logarithm to the base e;

[0117] (4) Determine whether the uncertainty index U is greater than the preset threshold U.th , if U>U th If U ≤ U th If the CT image is deemed suitable for use as a training set, it will be added to the training set. Specifically, the threshold U... th The threshold U is determined on the validation set using either the quantiles or the mean ± kσ of the distribution; that is, the threshold U... th The uncertainty index is determined by statistically analyzing the distribution of each sample on the validation set. Its value can be determined by the upper quantile of the distribution or by the mean-weighted standard deviation.

[0118] This invention provides an EGP-Net uncertainty assessment and training set selection method based on model segmentation results. By repeatedly introducing inference randomness and calculating the uncertainty index of average pixel entropy, it is beneficial to judge the stability of the EGP-Net network model's segmentation results for new images. Specifically, using the calculated average pixel entropy as a reference index, the smaller the average pixel entropy, the more consistent the model's output is in multiple random inferences, indicating that its segmentation results are more stable and have higher confidence; conversely, the larger the average pixel entropy, the more uncertain and unstable the model's segmentation of the image. Furthermore, selecting training samples based on uncertainty can improve the reliability of the training set and promote the robustness of the model.

[0119] The above description is a detailed description of the preferred embodiments of the present invention. However, the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modifications made under the technical spirit of the present invention should fall within the patent scope covered by the present invention.

Claims

1. A method for lung nodule segmentation with context feature aggregation combining edge-awareness and multi-scale semantic guidance, characterized in that, The method comprises the following steps: Obtaining a lung nodule CT image to be segmented, inputting the lung nodule CT image to be segmented into a trained lung nodule segmentation network model combining edge perception and multi-scale semantic guidance context feature aggregation, and obtaining a segmentation result; The lung nodule segmentation network model combining edge perception and multi-scale semantic guidance context feature aggregation is pre-constructed and trained, and the specific method is as follows: S1, collecting a data set of historical lung nodule CT images, and pre-processing the lung nodule CT images in the data set; S2, dividing the processed data set into a training set, a validation set and a test set at a certain proportion; S3, constructing the lung nodule segmentation network model combining edge perception and multi-scale semantic guidance context feature aggregation: S3.1, the input lung nodule CT image is preliminarily processed by a 1*1 convolution block to obtain a preliminary feature, and the preliminary feature is denoted as E 1×1 The preliminary feature is transmitted to an encoder network and an edge guidance network; S3.2, multi-layer feature extraction is performed in the encoder network to obtain multi-scale features S i i = 0, 1, 2, 3, …, n-1, n is the total number of stages of multi-layer feature extraction of the encoder network; S3.3, performing edge feature extraction in the edge guidance network to obtain local edge features, and the specific method is as follows: The preliminary features E obtained in step S3.1 are first processed by a 3x3 convolutional block and then fused with the low-level features S0 extracted by the encoder network of step S3.2 after a max-pooling operation. Subsequently, edge information is progressively extracted and restored to the original spatial resolution by two groups of identical 3x3 convolutional blocks. 1×1 The preliminary features E obtained in step S3.1 are first processed by a 3x3 convolutional block and then fused with the low-level features S0 extracted by the encoder network of step S3.2 after a max-pooling operation. Subsequently, edge information is progressively extracted and restored to the original spatial resolution by two groups of identical 3x3 convolutional blocks. S3.4, the multi-scale features except the low-level features S0 extracted by the encoder network in step S3.2 i  fused into the GPG module, in which the features of each stage are combined with all the features of higher stages than it to reconstruct the skip connections, thereby facilitating the effective capture of global semantic information; S3.5, the multi-scale context decoder network adopts ESPCN to perform super-resolution reconstruction to obtain global semantic features; S3.6, the local edge features captured by the edge guidance network in step S3.3 and the global semantic features extracted by the multi-scale context decoder in step S3.5 are weighted and fused through an attention feature fusion module to obtain a final segmentation result; S4, training the lung nodule segmentation network model combining edge perception and multi-scale semantic guidance context feature aggregation constructed in step S3 by using the training set divided in step S2, verifying the lung nodule segmentation network model combining edge perception and multi-scale semantic guidance context feature aggregation by using the validation set, and finally obtaining the trained lung nodule segmentation network model combining edge perception and multi-scale semantic guidance context feature aggregation; S5, introducing the lung nodule CT images on the test set divided in step S2 into the trained lung nodule segmentation network model combining edge perception and multi-scale semantic guidance context feature aggregation to obtain a segmentation result.

2. The method of claim 1, wherein the context-aware feature aggregation for lung nodule segmentation with edge-awareness and multi-scale semantic guidance is characterized by, The pre-processing step in step S1 is as follows: S1.1, using a Labelme tool to label the lung nodule CT images; S1.2, adjusting the resolution of all lung nodule CT images to 128x128 and uniformly processing them into gray images.

3. The method of claim 1, wherein, The encoder network of step S3.2 extracts features, and the operation of the low-level feature extraction stage Stage0 skips the initial pooling layer, and the feature resolution of the obtained low-level feature S0 is ; the feature resolution of the features S i obtained by feature extraction of other stages is , i>0, and H and W respectively correspond to the height and width of the features.

4. The method of claim 1, wherein, In step S3.3, the specific steps of the specific method of edge feature extraction by the edge guidance network are as follows: E 1×1 Firstly, the feature map is processed by a convolution block with a stride of 2, a channel number of 64, and a convolution kernel of 3x3 to compress the feature map resolution to half of the original; then, batch normalization, a ReLU activation function, and a 2x2 max-pooling operation are sequentially performed, and finally a feature map with a size of is obtained; at the same time, S0 is max-pooled to obtain a feature map with a size of The two obtained feature maps are spliced in the channel dimension to obtain fused features; then, the fused features pass through two groups of the same convolution blocks, and the outputs of the two groups of convolution blocks are respectively denoted as and , wherein each group of convolution blocks contains a 3x3 convolution layer, batch normalization, ReLU activation, and bilinear interpolation by two, which increases the resolution while reducing the channel number by half; finally, a single-channel boundary feature map F out ∈R H×W×1 is generated by a 1x1 convolution layer.

5. The method of claim 1, wherein, In step S3.4, the specific method of the GPG module to reconstruct the skip connection is as follows: First, map the features of all stages higher than the i-th stage into the same channel space as the i-th stage through a 3x3 convolution block to generate a feature map F j , j = i + 1,..., n - 1; Next, the generated feature map F j is upsampled to the same size as the feature map F i of the i-th stage and concatenated; Then, a plurality of separable convolution blocks with different expansion rates are used in parallel to extract global context information, wherein the separable convolution block is used to reduce the model parameters, the number of separable convolution blocks is n-i+1, and the expansion rates of the plurality of separable convolution blocks are in a progressive relationship; Finally, a standard convolution is used to obtain the final feature map.

6. The method of claim 1, wherein, In step S3.6, the specific method of the attention feature fusion module for weighted fusion is as follows: First, the input two features X and Y are initially fused, denoted as ; Then, the local branch reduces the number of fused feature channels to the original number by 1*1 point convolution , and then performs batch normalization and ReLU activation function in turn, and then restores the number of channels to the same as the original input channel number by 1*1 point convolution block and batch normalization; the local branch is the local edge feature captured by the edge guidance network, and r refers to the channel compression rate for controlling the number of channels; Then, the global branch performs a global average pooling operation on the fused features, and then performs the operation and processing in the above-mentioned manner for the local branch; the global branch is the global semantic features extracted by the multi-scale context decoder; After the local branch and the global branch perform the above-mentioned operations, the local branch and the global branch are fused again through an element-by-element addition operation, and the weight values calculated after the Sigmoid activation function are used to perform attention operations on the input features X and Y, respectively, and are multiplied by the respective corresponding elements to obtain output X' and Y', respectively; The , X' and Y' are added element by element to obtain the final output Z.

7. A model segmentation result-based EGP-Net uncertainty evaluation and training set screening method, characterized in that, The method comprises the following steps: (1) obtaining a new lung nodule CT image, inputting the new lung nodule CT image into the trained EGP-Net model, and letting a random item in the step of fusing into a final segmentation probability map be shielded or zeroed with a probability at each inference time, repeating the step T times; the EGP-Net network model is a context feature aggregation lung nodule segmentation network model combining edge perception and multi-scale semantic guidance according to any one of claims 1-6; the step of the final segmentation probability map is the step of the EGP-Net network model performing inference prediction on the newly obtained CT image to obtain a final segmentation result, that is, the stage of generating an output probability map after the model is fused by multiple branches; (2) T final segmentation probability maps are obtained, and analysis is performed thereon to obtain a probability value p of each pixel in T inferences t,n ; (3) According to the analysis result and the following formula, the uncertainty index U of the EGP-Net is calculated: Let the pixel average probability be Then the uncertainty is expressed by the pixel average entropy: wherein U is an uncertainty index, T is the number of final segmentation maps, N is the total number of pixels, p t,n the probability value of the nthpixel of the tthfinal segmentation map, ln is the natural logarithm with base e; (4) judging whether the uncertainty index U is greater than a preset threshold U th , if U>U th , it is considered that the CT image is not suitable as a training set; if U≤U th , it is considered that the CT image is suitable as a training set, and the image is added to the training set to be trained.

8. The EGP-Net uncertainty evaluation and training set screening method based on model segmentation results according to claim 7, wherein, In step (1), the random item in the step of fusing into a final segmentation probability map is set to 0, which means that in each inference, any branch of the local branch or global branch of the attention feature fusion module is temporarily set to 0, or Dropout is enabled to introduce randomness during inference.

9. The EGP-Net uncertainty evaluation and training set screening method based on model segmentation results according to claim 8, wherein, In step (4), threshold U th On the validation set in a distribution-wise quantile or mean ± kσ fashion.

Images