Lung nodule CT image processing method and system, computer device and storage medium

Abstract

The application provides a lung nodule CT image processing method and system, computer equipment and a storage medium, and belongs to the field of image processing. The method comprises the following steps: collecting a lung nodule CT image; segmenting the lung nodule CT image to obtain a nodule segmentation mask; extracting nodule structure semantic features from the nodule segmentation mask; extracting lung nodule texture features from the nodule segmentation mask based on a gray level co-occurrence matrix; extracting lung nodule shape features from the nodule segmentation mask by a Fourier descriptor; splicing the lung nodule texture features and the lung nodule shape features to obtain hand-crafted features; fusing the nodule structure semantic features and the hand-crafted features based on kernel canonical analysis to obtain fused features; and determining the nodule structure complexity corresponding to the lung nodule CT image according to the fused features. The method retains the powerful pattern recognition capability of the nodule structure semantic features, and also integrates the stable discrimination information of the hand-crafted features in a small sample scene, thereby improving the accuracy and stability of classification.

Description

Technical Field

[0001] This invention belongs to the field of image processing, specifically relating to methods, systems, computer equipment, and storage media for processing CT images of lung nodules. Background Technology

[0002] Lung cancer is one of the leading causes of cancer-related morbidity and mortality worldwide, and its prognosis is closely related to the stage at diagnosis. Studies have shown that early-stage lung cancer patients confined to the pulmonary nodule stage can have a significantly improved five-year survival rate if they receive timely diagnosis and treatment. Therefore, early screening and accurate diagnosis of lung cancer have become crucial for improving patient outcomes. Low-dose chest computed tomography (CT) scans are currently the most effective method for early lung cancer screening, clearly revealing the presence of small pulmonary nodules. However, with the widespread use of screening, the number of pulmonary nodule CT images that radiologists need to review has increased dramatically. This not only brings a huge workload but also increases the risk of misdiagnosis or missed diagnosis due to visual fatigue. In addition, the high heterogeneity of pulmonary nodules in terms of morphology, size, texture, and edge characteristics, as well as the low contrast and blurred boundaries of some nodules with surrounding tissues, make it difficult for even experienced physicians to make subjective judgment differences.

[0003] Based on the above requirements, existing technologies divide lung nodule CT image processing and analysis into two main tasks: image segmentation and classification of the segmented images as benign or malignant. The classification of lung nodules as benign or malignant is primarily based on the heterogeneity and irregularity of the nodules in terms of morphology and texture. While deep learning methods such as ResNet, Med-Former, and MV-KBC can achieve high accuracy in identifying lung nodules under certain conditions, the performance of the trained models is unstable when the sample size is too small, resulting in poor robustness in lung nodule CT image processing and analysis. Summary of the Invention

[0004] To address the problem of poor robustness in existing lung nodule CT image processing due to limited sample size, this invention provides a lung nodule CT image processing method, system, computer equipment, and medium.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] Methods for processing CT images of lung nodules include:

[0007] Acquire CT images of lung nodules; segment the lung nodules from the CT images to obtain a nodule segmentation mask;

[0008] Semantic features of nodule structure are extracted from the nodule segmentation mask; texture features of lung nodules are extracted from the nodule segmentation mask based on the gray-level co-occurrence matrix; shape features of lung nodules are extracted from the nodule segmentation mask using Fourier descriptors; and handcrafted features are obtained by concatenating the texture features and shape features of lung nodules.

[0009] Based on nuclear canonical analysis, the semantic features and manual features of nodule structure are fused to obtain fused features, and the structural complexity of nodule corresponding to CT image of lung nodule is determined according to the fused features.

[0010] Optionally, in the lung nodule CT image processing method provided by the present invention, the lung nodule CT image is processed by a pre-trained U2-Net segmentation network to obtain a nodule segmentation mask; the U2-Net segmentation network includes an encoder, a decoder, a skip connection layer that is connected layer by layer to the encoder and decoder, and an output layer;

[0011] Semantic features are obtained by downsampling the CT images of lung nodules layer by layer using an encoder;

[0012] The downsampled semantic features are passed to the corresponding decoder layer by layer through skip connection layers;

[0013] The semantic features are upsampled layer by layer by the decoder and fused with the semantic features of the corresponding layer to obtain the upsampled feature map.

[0014] The lung nodule category probability of pixels in the upsampled feature map is determined by the output layer, and the nodule segmentation probability map is obtained. The nodule segmentation probability map is then converted into a nodule segmentation mask.

[0015] Optionally, the lung nodule CT image processing method provided by the present invention further includes:

[0016] Acquire CT training images of lung nodules and their corresponding real-world masks;

[0017] The CT training images of lung nodules are input into the untrained U2-Net segmentation network to obtain the nodule segmentation training mask;

[0018] The loss function is calculated based on the training mask and the real mask for nodule segmentation. The untrained U2-Net segmentation network is then adjusted based on the loss function to obtain the pre-trained U2-Net segmentation network. The loss function includes a Gaussian distribution clustering loss function, which is determined by weighting the difference between the corresponding pixels in the training mask and the real mask. The weight of the corresponding pixel is determined by assigning the Euclidean distance from the pixel to the center of the image using a Gaussian distribution.

[0019] Optionally, the loss function may also include an edge loss function. The lung nodule CT image processing method provided by the present invention further includes:

[0020] The gradients of the nodule segmentation training mask in the horizontal and vertical directions are calculated based on the Sobel operator, and the edge strength of each element in the nodule segmentation training mask is determined by the gradients of the nodule segmentation training mask in the horizontal and vertical directions.

[0021] The gradients of the real mask in the horizontal and vertical directions are calculated based on the Sobel operator, and the edge intensity of each element in the real mask is determined by the gradients of the real mask in the horizontal and vertical directions.

[0022] The edge loss function is calculated based on the edge strength of each element in the training mask and the edge strength of each element in the real mask.

[0023] Optionally, in the lung nodule CT image processing method provided by the present invention, the nodule segmentation mask is processed by a ResNet residual network to obtain the nodule structural complexity; wherein, the ResNet residual network includes a feature extraction module, a kernel correlation feature fusion module and a classification output module connected in sequence, the feature extraction module includes a parallel depth feature extraction module and a manual feature extraction module, the manual feature extraction module includes a parallel texture extraction module and a shape extraction module, and a stitching module;

[0024] The semantic features of nodule structure are extracted from the nodule segmentation mask using a deep feature extraction module;

[0025] The texture extraction module enhances the nodule segmentation mask through contrast-restricted adaptive histogram equalization to obtain the Clahe-enhanced image; the gray-level co-occurrence matrix of the Clahe-enhanced image is calculated to obtain the lung nodule texture features;

[0026] The shape extraction module enhances the nodule segmentation mask through an anti-sharp mask to obtain an Unsharp Mask enhanced image; the nodule boundary contour is extracted from the Unsharp Mask enhanced image based on the Canny edge detection algorithm, and the nodule boundary contour is subjected to discrete Fourier transform to construct lung nodule shape features;

[0027] The stitching module stitches together the texture and shape features of lung nodules in a dimensional manner to obtain handcrafted features.

[0028] Optionally, the lung nodule CT image processing method provided by the present invention further includes:

[0029] The semantic features and handcrafted features of the nodule structure are mapped into the feature space using kernel functions, respectively.

[0030] The weighting coefficients for the semantic features and handcrafted features of nodule structure are determined with the goal of maximizing the correlation coefficient after projection of the two feature spaces.

[0031] The projection representations of the deep features and the handcrafted features are determined based on the weight coefficients corresponding to the semantic features and handcrafted features of the nodule structure, respectively.

[0032] The semantic features of nodule structure and the projected representations of handcrafted features are fused to obtain fused features.

[0033] Optionally, the lung nodule CT image processing method provided by the present invention further includes:

[0034] The training mask for nodule segmentation is processed by an untrained ResNet residual network to obtain the nodule structure complexity for training.

[0035] When the structural complexity of the training nodes exceeds a preset threshold, increase the weight of the Gaussian clustering loss function in the loss function.

[0036] When the structural complexity of the training nodes is lower than a preset threshold, the weight of the edge loss function in the loss function is increased.

[0037] This invention also provides a CT image processing system for lung nodules, comprising:

[0038] The nodule image segmentation module is used to acquire CT images of lung nodules; and to segment the CT images of lung nodules to obtain a nodule segmentation mask.

[0039] The nodule feature extraction module is used to extract semantic features of nodule structure from the nodule segmentation mask; extract lung nodule texture features from the nodule segmentation mask based on the gray-level co-occurrence matrix; extract lung nodule shape features from the nodule segmentation mask through Fourier descriptors; and concatenate the lung nodule texture features and lung nodule shape features to obtain handcrafted features.

[0040] The structural complexity output module is used to fuse semantic features and manual features of nodule structure based on kernel canonical analysis to obtain fused features, and to determine the structural complexity of nodule CT images corresponding to lung nodules based on the fused features.

[0041] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps of the lung nodule CT image processing method.

[0042] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute any of the steps in the lung nodule CT image processing method.

[0043] The lung nodule CT image processing method provided by this invention has the following beneficial effects:

[0044] The lung nodule CT image processing method provided by this invention combines depth features with hand-designed explicit physical features: texture features extracted through the gray-level co-occurrence matrix can stably describe the density distribution pattern inside the nodule, while shape features extracted through the Fourier descriptor can quantify the geometric characteristics of the nodule edge, capturing the inherent patterns of the nodule. This results in hand-designed features that can effectively capture the inherent physical patterns of lung nodules without relying on a large amount of training data, providing reliable priors for few-sample learning. Subsequently, kernel canonical correlation analysis is used to nonlinearly fuse the depth features and hand-designed features. This retains the powerful pattern recognition capability of the depth features while incorporating the stable discriminative information of the hand-designed features in few-sample scenarios, enhancing the adaptability to limited data. This allows for accurate identification of the structural complexity of the segmented lung nodule segmentation mask. In summary, the lung nodule CT image processing method provided by this invention overcomes the problem of excessive reliance on data volume in existing methods, maintains high robustness under few-sample conditions, and enhances the applicability and reliability of lung nodule structural complexity analysis in real-world scenarios. Attached Figure Description

[0045] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of a lung nodule CT image processing method provided in an embodiment of the present invention;

[0047] Figure 2 This is an example of a Gaussian weight graph provided in an embodiment of the present invention;

[0048] Figure 3 An example of edge loss intensity provided in an embodiment of the present invention;

[0049] Figure 4 This is an example of a CT image processing workflow for lung nodules provided in an embodiment of the present invention;

[0050] Figure 5 An example of the original grayscale image provided in an embodiment of the present invention;

[0051] Figure 6 This is an example of a preprocessed image provided in an embodiment of the present invention;

[0052] Figure 7 This is an example of a nodule segmentation mask provided in an embodiment of the present invention;

[0053] Figure 8 This is an example of the manual feature extraction module process provided in an embodiment of the present invention;

[0054] Figure 9 This is an example of kernel transform feature fusion provided in an embodiment of the present invention;

[0055] Figure 10 This is an example of CT image analysis provided in an embodiment of the present invention. Detailed Implementation

[0056] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0057] Existing CT image analysis of lung nodules includes two main tasks: image segmentation and benign / malignant classification. In image segmentation, existing CT image segmentation methods include thresholding and region growing. While these methods are computationally simple, they are sensitive to noise and their actual segmentation results are not ideal. Active contour models (ACM) and graph cut methods, although capable of handling complex structures, are prone to getting trapped in local optima due to their sensitivity to initialization and parameters. Among deep learning methods, convolutional networks such as U-Net, Attention U-Net, ResU-Net, and Dense U-Net, with their encoder-decoder structures, have shown excellent performance in medical segmentation. However, these methods tend to lose feature details when dealing with small nodules or regions with blurred boundaries, leading to insufficient segmentation accuracy. While Transformer-based methods like Swin-UNet and Mask2Former offer stronger global modeling capabilities, they are computationally expensive and heavily reliant on labeled data.

[0058] In the classification of benign and malignant nodules, methods relying solely on handcrafted features such as GLCM texture features and Fourier shape descriptors (FD) struggle to handle high-dimensional and complex data. While deep learning methods like ResNet, Med-Former, and MV-KBC achieve high accuracy, they generally suffer from poor interpretability and performance instability with small sample sizes. In summary, existing lung nodule CT image analysis generally suffers from drawbacks such as low segmentation accuracy in small nodule regions or regions with blurred nodule edges, insufficient exploitation of correlations between multimodal features, and weak interpretability due to the "black box" nature of deep models.

[0059] The lung nodule CT image processing method provided by this invention utilizes a geometrically enhanced multi-feature fusion optimization model (GMFKC) to achieve collaborative learning for early lung nodule segmentation and classification. In the image segmentation stage, based on U-Net, the encoding depth is simplified, and a composite loss function fusing Gaussian distribution and Sobel operator is introduced to enhance the response to nodule center and edge features. In the classification stage, based on image enhancement processing and combined with gray-level co-occurrence matrix and Fourier descriptors, a GLCM-FD feature encoding module is designed to fuse shape and texture information. Simultaneously, a feature fusion module is constructed to perform kernel correlation analysis and nonlinear alignment on highly abstract deep features such as nodule structural semantic features and handcrafted features, enhancing feature resolution and improving the interpretability of nodule analysis. Finally, a dynamic weight adjustment strategy and collaborative optimization mechanism linking classification and segmentation losses are introduced to improve model convergence efficiency and stability.

[0060] Example 1

[0061] This application provides a method for processing CT images of lung nodules, specifically as follows: Figure 1 As shown, it includes the following steps:

[0062] Step 11: Obtain lung nodule CT training images and corresponding ground truth masks; input the lung nodule CT training images into the untrained U2-Net segmentation network to obtain the nodule segmentation training mask;

[0063] Step 12: Calculate the loss function based on the training mask and the real mask for nodule segmentation, and adjust the untrained U2-Net segmentation network based on the loss function to obtain the pre-trained U2-Net segmentation network. The loss function includes a Gaussian distribution clustering loss function, which is determined by weighting the difference and weight of the corresponding pixels between the training mask and the real mask for nodule segmentation. The weight of the corresponding pixel is determined by assigning the Euclidean distance from the pixel to the center of the image using a Gaussian distribution.

[0064] The loss function also includes an edge loss function, and step 12 further includes:

[0065] Step 121: Calculate the gradients of the nodule segmentation training mask in the horizontal and vertical directions based on the Sobel operator, and determine the edge intensity of each element in the nodule segmentation training mask based on the gradients of the nodule segmentation training mask in the horizontal and vertical directions.

[0066] Step 122: Calculate the gradient of the real mask in the horizontal and vertical directions based on the Sobel operator, and determine the edge intensity of each element in the real mask by the gradient of the real mask in the horizontal and vertical directions.

[0067] Step 123: Calculate the edge loss function based on the edge strength of each element in the training mask for nodule segmentation and the edge strength of each element in the real mask.

[0068] Specifically, during the training process of the U2-Net segmentation network, the model is optimized using a loss function composed of a clustering loss function based on Gaussian distribution and an edge loss function based on Sobel operator.

[0069] In this process, after inputting CT training images of lung nodules into the model to obtain a nodule segmentation training mask, a Gaussian distribution-based clustering loss function assigns a weight related to the image center position to each pixel, thereby strengthening the model's focus on the central region of the image. This weight is then incorporated into the traditional cross-entropy loss, weighting the loss of each pixel to make the model pay more attention to the region where the nodules are located during training. For example, calculating the loss of each pixel in the nodule segmentation training mask... Image center of the training mask for nodule segmentation The Euclidean distance between them is then used to assign weights to the pixels using a Gaussian distribution, as shown in formula (1):

[0070] (1)

[0071] in, and These are the height and width of the image, respectively. The parameters that control the spatial range of the weight distribution determine the "degree of attention" in the central region of the image; smaller parameters result in a smaller "degree of attention." Values ​​that cause weights to concentrate in the center of the image will result in larger values. This value will cause the pixel weights to be more dispersed. For pixels Weights, for example Figure 2 As shown. The clustering loss function based on the Gauss distribution is shown in Equation (2):

[0072] (2)

[0073] in, The loss value is the Gaussian distribution clustering loss function. For pixels The corresponding nodule segmentation training mask, For pixels The corresponding real mask.

[0074] Furthermore, to effectively guide the model to better capture image edge information, an edge loss based on the Sobel operator is introduced into the loss function. For example, this is achieved through convolutional kernels... and Calculate the gradient of the image in the horizontal direction. and convolution kernel and As shown in formula (3):

[0075] (3)

[0076] The edge strength of the image As shown in formula (4):

[0077] (4)

[0078] Calculate the edge strength of the training mask and the real mask for nodule segmentation respectively. The edge loss function is obtained as shown in formula (5):

[0079] (5)

[0080] Where N is the total number of pixels in the image. The index of a pixel in the image. For pixels The edge strength of the training mask for nodule segmentation at the corresponding location. For pixels The edge strength of the corresponding real mask at that location. The edge loss strength of the edge loss function, such as Figure 3 As shown.

[0081] Then, the edge loss function and the Gaussian distribution clustering loss function are weighted to obtain the combined loss function of geometrically enhanced Gauss-Sobel loss, as shown in Equation (6):

[0082] (6)

[0083] in, For the comprehensive loss function, and These are the weights for the two loss functions.

[0084] Step 13: Process the nodal segmentation training mask using an untrained ResNet residual network to obtain the nodal structure complexity for training; when the nodal structure complexity for training is higher than a preset threshold, increase the weight of the Gaussian distribution clustering loss function in the loss function; when the nodal structure complexity for training is lower than a preset threshold, increase the weight of the edge loss function in the loss function.

[0085] Specifically, during the model training phase, in the initial coarse training process, the weights are first... and weight Parameters are frozen, and weights are unfrozen after the initial training is completed. and weight The weights of the loss function are updated inversely based on the structural complexity of the nodules. For example, when the structural complexity of the nodules used in training exceeds a preset threshold, indicating that the lung nodules have a more complex morphology and a higher probability of being malignant, the weights are adjusted. When the structural complexity of the training nodules is below a preset threshold, indicating that the lung nodules have clear boundaries and are more likely to be benign, the weights are adjusted. And with the help of curriculum-based training, stable convergence is achieved.

[0086] Step 14: Acquire CT images of lung nodules; segment the CT images of lung nodules to obtain a nodule segmentation mask. Specifically, the lung nodule CT images are processed using a pre-trained U2-Net segmentation network to obtain the nodule segmentation mask. The U2-Net segmentation network includes an encoder, a decoder, skip connection layers that are sequentially connected to the encoder and decoder, and an output layer. Step 14 includes:

[0087] Step 141: The CT image of lung nodules is downsampled layer by layer using an encoder to obtain semantic features;

[0088] Step 142: Pass the downsampled semantic features layer by layer to the decoder of the corresponding layer through skip connection layers;

[0089] Step 143: Upsample the semantic features layer by layer using the decoder and fuse them with the semantic features of the corresponding layer to obtain the upsampled feature map;

[0090] Step 144: Determine the lung nodule category probability of the pixels in the upsampled feature map through the output layer to obtain the nodule segmentation probability map, and convert the nodule segmentation probability map into a nodule segmentation mask.

[0091] Specifically, the U2-Net segmentation network, such as Figure 4 As shown, it includes a segmentation module, a classification module, and a loss module activated during model training. The segmentation module is a symmetric encoder-decoder structure.

[0092] In the encoder and decoder structure of the first and second layers, the complete data processing flow is as follows: First, convolution (Conv), batch normalization (BN), and ReLU activation are performed. After repeating this process twice, downsampling to half size (Downsample × 1 / 2) is performed three times consecutively using convolution (Conv), batch normalization (BN), and ReLU activation. Then, the dilation rate is set to 2 (dilation=2), and dilated convolution is performed, again using convolution (Conv), batch normalization (BN), and ReLU activation. Next, convolution (Conv), batch normalization (BN), and ReLU activation are performed once more. Finally, upsampling to twice size (Upsample × 2) is performed three times consecutively using convolution (Conv), batch normalization (BN), and ReLU activation, completing the upsampling process. In the first-layer encoder, after dilated convolution, subsequent upsampling is not activated. Instead, the result is passed to the second-layer encoder for further convolution (Conv), batch normalization (BN), ReLU activation, and downsampling. In the corresponding decoder, the results from the corresponding layer encoder and the previous layer decoder are received via a skip connection mechanism. After concatenation, upsampling is performed. For example, the first-layer decoder receives the results from the first-layer encoder and the second-layer decoder, concatenates them, and then resamples to obtain the CT image of the lung nodules.

[0093] In the encoder and decoder structure of the third layer, the complete data processing flow is as follows: first, convolution (Conv), batch normalization (BN), and ReLU activation are performed. After repeating this process twice, the dilation rate is set to 2 (dilation=2) and dilated convolution is performed, specifically, convolution (Conv), batch normalization (BN), and ReLU activation are performed again. Then, the dilation rate is set to 4 (dilation=4) and dilated convolution is performed again, followed by a dilation rate of 8 (dilation=8) and dilated convolution. Subsequently, the dilation rate is set to 4 (dilation=4) and dilated convolution is performed, then the dilation rate is set to 2 (dilation=2) and dilated convolution is performed again, and finally, convolution (Conv), batch normalization (BN), and ReLU activation are performed once more. Furthermore, the deepest or bottleneck layer of the segmentation network samples the same structural design as the encoder and decoder structure of the third layer, thus receiving information from the encoder's end. After a series of internal convolutions and nonlinear transformations, the global features are deeply processed and refined, and then the processed features are passed to the decoder to begin the upsampling process. In the segmentation module, the bottleneck layer and each upsampling layer output the processing result or upsampling result of an image. These are then concatenated and processed through convolution and the Sigmoid activation function to achieve upsampling to the input size of the nodule image, thereby obtaining the final nodule segmentation mask, which is then output to the classification module.

[0094] Subsequently, in the classification module, along the main segmentation path, the nodule segmentation mask is received and processed through a single convolution (Conv), batch normalization (BN), and ReLU activation, followed by max pooling. This is then repeated with multiple convolutions, batch normalizations, and ReLU activations. Finally, it is processed through a fully connected layer before being passed to the KCCAF feature fusion module in the classification module. Furthermore, upon receiving the nodule segmentation mask, it is passed to the GLCM-FD feature encoding module in the classification module for image enhancement and feature extraction. This mask is then passed to the KCCAF feature fusion module, which performs kernel transformation and correlation analysis based on the data processing results from both paths. Finally, the soft classifier in the classification module outputs the benign / malignant probability of the lung nodule CT image.

[0095] Furthermore, during the model training phase, the classification loss function is adjusted through co-learning. Sobel Loss (marginal loss function) Gaussian clustering loss function Each has its own weight.

[0096] Specifically, first obtain, such as Figure 5 The single-channel grayscale image shown is a chest CT image, which has been normalized and subjected to random enhancement processing such as cropping, flipping, and standardization to obtain the image shown. Figure 6 The image shown is a CT image of a lung nodule. Subsequently, the enhanced image was segmented using a U2-Net segmentation network to obtain the following results: Figure 7 The nodule segmentation mask shown.

[0097] Step 15: Extract semantic features of nodule structure from nodule segmentation mask; extract lung nodule texture features from nodule segmentation mask based on gray-level co-occurrence matrix; extract lung nodule shape features from nodule segmentation mask using Fourier descriptor; and concatenate lung nodule texture features and lung nodule shape features to obtain handcrafted features. The nodule segmentation mask is processed using a ResNet residual network to obtain the nodule structural complexity. The ResNet residual network includes a sequentially connected feature extraction module, a kernel-related feature fusion module, and a classification output module. The feature extraction module includes parallel deep feature extraction and handcrafted feature extraction modules. The handcrafted feature extraction module includes parallel texture extraction and shape extraction modules, as well as a stitching module. The deep feature extraction module extracts semantic features of the nodule structure from the nodule segmentation mask. The texture extraction module enhances the nodule segmentation mask using contrast-constrained adaptive histogram equalization to obtain a Clahe-enhanced image. The gray-level co-occurrence matrix of the Clahe-enhanced image is calculated to obtain the lung nodule texture features. The shape extraction module enhances the nodule segmentation mask using an unsharp mask to obtain an Unsharp Mask-enhanced image. The nodule boundary contour is extracted from the Unsharp Mask-enhanced image based on the Canny edge detection algorithm, and a discrete Fourier transform is performed on the nodule boundary contour to construct lung nodule shape features. The stitching module stitches the lung nodule texture features and lung nodule shape features dimensionally to obtain handcrafted features.

[0098] Specifically, such as Figure 8 As shown, to effectively capture explicit statistical structures in medical images, such as density, edge sharpness, and directional texture, this invention designs a heterogeneous handcrafted feature extraction module to supplement the insufficient expression of depth features from both texture and shape perspectives. On one hand, a contrast-limited adaptive histogram equalization (Clahe) method is introduced to enhance local detail representation, resulting in a Clahe-enhanced image. As shown in formula (7):

[0099] (7)

[0100] in, This is the original grayscale image of the lung nodules. and These control the degree of contrast enhancement and the size of the local window, respectively. To compare the constrained adaptive histogram equalization function, the Clahe-enhanced image was then processed. Calculate the gray-level co-occurrence matrix This matrix is ​​used to represent grayscale values. and grayscale value In direction ,distance The frequency of joint occurrence under the following conditions is used to determine the pulmonary nodule texture features .

[0101] On the other hand, an unsharp mask is used to enhance the edges of the original lung nodule grayscale image, as shown in formula (8):

[0102] (8)

[0103] in, For Gaussian blur operation, Enhance images with Unsharp Mask. To control the sharpening intensity parameter, the possible values ​​are: Then, the nodule boundary contours of the Unsharp Mask-enhanced image were extracted using the Canny edge detection algorithm. and the nodule boundary profile Perform Discrete Fourier Transform (DFT) processing, as shown in Equation (9):

[0104] (9)

[0105] in, For the frequency index of the Discrete Fourier Transform, This is the frequency domain result obtained after the Discrete Fourier Transform. These are discrete sampling points representing the nodule boundary contour. Then, the first M low-frequency Fourier coefficients are selected to construct... morphological characteristics of pulmonary nodules Finally, the feature vectors from both sides are concatenated in a different dimension to obtain the final handcrafted feature. .

[0106] Step 16: Based on nuclear canonical analysis, the deep features and manual features are fused to obtain fused features, and the benign or malignant probability of the CT image of the lung nodule is determined according to the fused features.

[0107] Step 16 includes:

[0108] Step 161: Map the semantic features and handcrafted features of the nodule structure into the feature space using kernel functions, respectively.

[0109] Step 162: Determine the weight coefficients corresponding to the semantic features and handcrafted features of the nodule structure with the goal of maximizing the correlation coefficient after projection of the two feature spaces.

[0110] Step 163: Determine the projection representations of the deep features and the handcrafted features based on the weight coefficients corresponding to the semantic features and handcrafted features of the nodule structure, respectively.

[0111] Step 164: Perform feature fusion on the projected representations of depth features and handcrafted features to obtain fused features.

[0112] Specifically, after the manual features are constructed, considering the different methods used for manual features and deep features, the two sets of features often have complex nonlinear correlations. Simply selecting features directly cannot guarantee their representational power. The lung nodule CT image processing method provided in this invention implements manual features through kernel transformation, as shown in the figure. with depth features The efficient integration of [various elements]. Among them, handcrafted features... Deep features For example, introducing Mapping handcrafted features to a high-dimensional feature space In, that is , Introduction Mapping handcrafted features to a high-dimensional feature space In, that is As shown in formulas (10) and (11):

[0113] (10)

[0114] (11)

[0115] in, For handcrafted characteristics After mapping in high-dimensional feature space The representation in the text, It is the first High-dimensional mapping results of each handcrafted feature component For handcrafted characteristics After mapping in high-dimensional feature space The representation in the text, It is the first The high-dimensional mapping result of each handcrafted feature component.

[0116] Then the projection vector is introduced. and and calculate and correlation coefficient As shown in formulas (12) and (13):

[0117] (12)

[0118] (13)

[0119] make ,remember and The processing is as shown in formula (14):

[0120] (14)

[0121] After that, maximization And solve for and Ultimately and The final fused features are determined by the equation shown in Equation (15), which serves as the weights for the deep features and the handcrafted features. :

[0122] (15)

[0123] Furthermore, in model training, the classification loss function The standard binary cross-entropy (BCE) loss can be chosen to measure the difference between the predicted probability distribution and the true label, as shown in Equation (16):

[0124] (16)

[0125] Therefore, during the model training phase, As a classification loss function Weights, total loss As shown in formula (17):

[0126] (17)

[0127] Based on the above implementation methods, such as Figure 10As shown, the lung nodule CT image processing method provided by this invention predicts the benign or malignant probability of multiple lung nodule CT images that have been pre-segmented and labeled manually. The results are largely consistent with manual predictions, indicating that the lung nodule CT image processing method provided by this invention possesses a certain degree of accuracy. Furthermore, comparing the lung nodule CT image processing method provided by this invention with the existing U2-Net model in image segmentation tasks, the lung nodule CT image processing method provided by this invention improves the segmentation IoU by 76.3% on the LIDC-IDRI dataset compared to the existing U2-Net model, and reduces the MAE by 0.8% compared to the existing U2-Net model, demonstrating more accurate region localization and more refined boundary prediction. In the lung nodule benign or malignant analysis task, the lung nodule CT image processing method provided by this invention achieves a classification AUC of 98.97% and an accuracy of 96.53%, outperforming existing CNN or Transformer models.

[0128] Example 2

[0129] This application also provides a CT image processing system for lung nodules, including:

[0130] The nodule image segmentation module is used to acquire CT images of lung nodules; and to segment the CT images of lung nodules to obtain a nodule segmentation mask.

[0131] The nodule feature extraction module is used to extract semantic features of nodule structure from the nodule segmentation mask; extract lung nodule texture features from the nodule segmentation mask based on the gray-level co-occurrence matrix; extract lung nodule shape features from the nodule segmentation mask through Fourier descriptors; and concatenate the lung nodule texture features and lung nodule shape features to obtain handcrafted features.

[0132] The structural complexity output module is used to fuse semantic features and manual features of nodule structure based on kernel canonical analysis to obtain fused features, and to determine the structural complexity of nodule CT images corresponding to lung nodules based on the fused features.

[0133] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in the embodiments of the lung nodule CT image processing method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0134] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in the embodiments of the lung nodule CT image processing method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0135] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0137] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0138] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0139] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for processing CT images of lung nodules, characterized in that, include: Acquire CT images of lung nodules; The lung nodule CT image is segmented to obtain a nodule segmentation mask. Specifically, the lung nodule CT image is processed by a pre-trained U2-Net segmentation network to obtain the nodule segmentation mask. The U2-Net segmentation network includes an encoder, a decoder, skip connection layers connected layer-by-layer to the encoder and decoder, and an output layer. The encoder downsamples the lung nodule CT image layer-by-layer to obtain semantic features. The skip connection layers pass the downsampled semantic features layer-by-layer to the decoder of the corresponding layer. The decoder upsamples the semantic features layer-by-layer and fuses them with the semantic features of the corresponding layer to obtain an upsampled feature map. The output layer determines the lung nodule category probability of the pixels in the upsampled feature map to obtain a nodule segmentation probability map, and the nodule segmentation probability map is converted into the nodule segmentation mask. Semantic features of nodule structure are extracted from the nodule segmentation mask; texture features of lung nodules are extracted from the nodule segmentation mask based on the gray-level co-occurrence matrix; shape features of lung nodules are extracted from the nodule segmentation mask using Fourier descriptors; the texture features and shape features of lung nodules are concatenated to obtain handcrafted features, specifically including processing the nodule segmentation mask through a ResNet residual network to obtain the nodule structure complexity; wherein, the ResNet residual network includes a feature extraction module, a kernel correlation feature fusion module, and a classification output module connected in sequence, and the feature extraction module includes parallel... The system comprises a deep feature extraction module and a manual feature extraction module. The manual feature extraction module includes a parallel texture extraction module and a shape extraction module, as well as a stitching module. The deep feature extraction module extracts the semantic features of the nodule structure from the nodule segmentation mask. The texture extraction module enhances the nodule segmentation mask using contrast-restricted adaptive histogram equalization to obtain a Clahe-enhanced image. The gray-level co-occurrence matrix of the Clahe-enhanced image is calculated to obtain the lung nodule texture features. The shape extraction module enhances the nodule segmentation mask using an unsharp mask to obtain an Unsharp Mask-enhanced image. Based on the Canny edge detection algorithm, the nodule boundary contour is extracted from the Unsharp Mask-enhanced image, and a discrete Fourier transform is performed on the nodule boundary contour to construct the lung nodule shape features. The stitching module stitches the lung nodule texture features and lung nodule shape features dimensionally to obtain the manual features. Based on kernel canonical analysis, the semantic features of the nodule structure and the handcrafted features are fused to obtain fused features. Specifically, this includes mapping the semantic features of the nodule structure and the handcrafted features to feature spaces using kernel functions; determining the weight coefficients corresponding to the semantic features of the nodule structure and the handcrafted features respectively with the goal of maximizing the correlation coefficient after projection of the two feature spaces; determining the projection representations of the semantic features of the nodule structure and the handcrafted features respectively based on the weight coefficients; fusing the projection representations of the semantic features of the nodule structure and the handcrafted features to obtain the fused features; and determining the nodule structure complexity corresponding to the pulmonary nodule CT image based on the fused features.

2. The lung nodule CT image processing method of claim 1, wherein, Before processing the lung nodule CT images using a pre-trained U2-Net segmentation network, the following steps are also included: Acquire CT training images of lung nodules and their corresponding real-world masks; The lung nodule CT training images are input into an untrained U2-Net segmentation network to obtain a nodule segmentation training mask; The loss function is calculated based on the nodule segmentation training mask and the real mask. The untrained U2-Net segmentation network is then adjusted based on the loss function to obtain the pre-trained U2-Net segmentation network. The loss function includes a Gaussian distribution clustering loss function, which is determined by weighting the difference between the corresponding pixels in the nodule segmentation training mask and the real mask. The weight of the corresponding pixel is determined by assigning the Euclidean distance from the pixel to the center of the image using a Gaussian distribution.

3. The lung nodule CT image processing method of claim 2, wherein, The loss function also includes an edge loss function, and the calculation of the loss function based on the nodule segmentation training mask and the real mask further includes: The gradients of the nodule segmentation training mask in the horizontal and vertical directions are calculated based on the Sobel operator, and the edge intensity of each element in the nodule segmentation training mask is determined by the gradients of the nodule segmentation training mask in the horizontal and vertical directions. The gradients of the real mask in the horizontal and vertical directions are calculated based on the Sobel operator, and the edge intensity of each element in the real mask is determined by the gradients of the real mask in the horizontal and vertical directions. The edge loss function is calculated based on the edge strength of each element in the training mask for nodule segmentation and the edge strength of each element in the real mask.

4. The lung nodule CT image processing method of claim 3, wherein, Before processing the nodule segmentation mask using a ResNet residual network to obtain the nodule structural complexity, the process further includes: The nodule segmentation training mask is processed by an untrained ResNet structure residual network to obtain the nodule structure complexity for training. When the structural complexity of the training nodules exceeds a preset threshold, the weight of the Gaussian distribution clustering loss function in the loss function is increased. When the complexity of the training nodule structure is lower than the preset threshold, the weight of the edge loss function in the loss function is increased.

5. A lung nodule CT image processing system characterized by, include: A nodule image segmentation module is used to acquire CT images of lung nodules; segment the lung nodule CT images to obtain a nodule segmentation mask, specifically by processing the lung nodule CT images using a pre-trained U2-Net segmentation network to obtain the nodule segmentation mask; the U2-Net segmentation network includes an encoder, a decoder, skip connection layers connected layer-by-layer to the encoder and decoder, and an output layer; the encoder performs layer-by-layer downsampling on the lung nodule CT images to obtain semantic features; the skip connection layers pass the downsampled semantic features layer-by-layer to the decoder of the corresponding layer; the decoder performs layer-by-layer upsampling on the semantic features and fuses them with the semantic features of the corresponding layer to obtain an upsampled feature map; the output layer determines the lung nodule category probability of the pixels in the upsampled feature map to obtain a nodule segmentation probability map, and the nodule segmentation probability map is converted into the nodule segmentation mask; The nodule feature extraction module is used to extract semantic features of nodule structure from the nodule segmentation mask; extract lung nodule texture features from the nodule segmentation mask based on the gray-level co-occurrence matrix; extract lung nodule shape features from the nodule segmentation mask using Fourier descriptors; and concatenate the lung nodule texture features and lung nodule shape features to obtain handcrafted features, specifically including processing the nodule segmentation mask through a ResNet residual network to obtain the nodule structure complexity; wherein, the ResNet residual network includes a feature extraction module, a kernel correlation feature fusion module, and a classification output module connected in sequence, the feature extraction module... The system includes a parallel deep feature extraction module and a manual feature extraction module. The manual feature extraction module includes a parallel texture extraction module and a shape extraction module, as well as a stitching module. The deep feature extraction module extracts the semantic features of the nodule structure from the nodule segmentation mask. The texture extraction module enhances the nodule segmentation mask using contrast-constrained adaptive histogram equalization to obtain a Clahe-enhanced image. The gray-level co-occurrence matrix of the Clahe-enhanced image is calculated to obtain the lung nodule texture features. The shape extraction module enhances the nodule segmentation mask using an anti-sharp mask to obtain an Unsharp Mask-enhanced image. Based on the Canny edge detection algorithm, the nodule boundary contour is extracted from the Unsharp Mask-enhanced image, and a discrete Fourier transform is performed on the nodule boundary contour to construct the lung nodule shape features. The stitching module stitches the lung nodule texture features and lung nodule shape features dimensionally to obtain the manual features. The structural complexity output module is used to fuse the nodule structural semantic features and the handcrafted features based on kernel canonical analysis to obtain fused features. Specifically, it includes mapping the nodule structural semantic features and the handcrafted features to feature spaces using kernel functions respectively; determining the weight coefficients corresponding to the nodule structural semantic features and the handcrafted features with the goal of maximizing the correlation coefficient after projection of the two feature spaces; determining the projection representations of the nodule structural semantic features and the handcrafted features based on the weight coefficients corresponding to the nodule structural semantic features and the handcrafted features respectively; fusing the projection representations of the nodule structural semantic features and the handcrafted features to obtain the fused features; and determining the nodule structural complexity corresponding to the lung nodule CT image based on the fused features.

6. A computer device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program comprises instructions that, when executed by the processor, cause the processor to perform the method of any one of claims 1-5. The processor executes the computer program to implement the steps of the lung nodule CT image processing method according to any one of claims 1 to 4.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the lung nodule CT image processing method according to any one of claims 1 to 4.

Images