Self-adaptive weight lightweight semi-supervised low-dose CT image denoising method
Through the lightweight semi-supervised denoising method of adaptive weights, multi-scale convolution feature extraction, recursive refinement and soft maximum feature fusion modules are used to denoise the label-free clinical low-dose CT images, solving the clinical low-dose The problem of CT image denoising achieves efficient noise removal and artifact suppression, improving diagnostic accuracy.
Patent Information
- Application Number
- CN202411034427.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2025-01-03
AI Technical Summary
The prior art is difficult to effectively denoise and label-free clinical low-dose CT images, and the unsupervised methods have limited denoising performance, making it difficult to directly apply complex network structures to clinical data.
A lightweight semi-supervised low-dose CT image denoising method with adaptive weights is proposed. By dividing medical image data into training sets, verification sets and test sets, data augmentation preprocessing is performed, and a student model and teacher model are constructed. The adaptive weights are embedded in a lightweight semi-supervised denoising network, and the images are denoised using a multi-scale convolution feature extraction module, a recursive refinement module and a soft maximum feature fusion module.
Effective denoising and artifact suppression of label-free clinical low-dose CT images is achieved, reducing the number of model parameters and calculation complexity, and improving processing efficiency and diagnostic accuracy.
Smart Images
Figure CN119251076A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of image processing, and in particular to a lightweight semi-supervised low-dose CT image denoising method with adaptive weights. Background Art
[0002] CT (Computed Tomography) is a medical imaging technology that can provide three-dimensional images of internal organs and abnormal tissues of the human body. Although CT imaging provides valuable information for clinical diagnosis, it may also pose potential health risks to patients. Specifically, excessive X-ray radiation may damage normal tissues, induce gene mutations, and even lead to the formation of malignant tumors, and also have adverse effects on the reproductive and skeletal systems. In order to reduce this risk, the use of low-dose CT is advocated, but this will bring problems of image noise and artifacts, which in turn affect the accuracy of diagnosis.
[0003] At present, supervised low-dose CT denoising methods based on simulated low-dose data have surpassed traditional low-dose CT reconstruction algorithms in performance. However, these methods have limitations in practical applications because it is difficult to obtain paired low-dose-high-dose CT data during actual clinical CT data acquisition, and there are inherent differences between simulated data and real data, which makes it difficult to directly apply these methods to unlabeled clinical low-dose CT images. Although unsupervised methods are not subject to the above limitations and can utilize unlabeled clinical low-dose CT images, these methods often involve complex network structures and their denoising performance is limited.
[0004] Therefore, for unlabeled clinical low-dose CT images, how to achieve effective denoising of CT images is a technical problem that needs to be urgently solved in this field. Summary of the invention
[0005] In order to solve the above technical problems, this application proposes the following technical solutions:
[0006] In a first aspect, an embodiment of the present application provides a lightweight semi-supervised low-dose CT image denoising method with adaptive weights, comprising:
[0007] Dividing medical image data into a training set, a validation set, and a test set, wherein the medical image data includes a labeled simulated low-dose CT data set and an unlabeled clinical low-dose CT data set;
[0008] Performing data augmentation preprocessing on the training set to expand the data set and improve the generalization ability of the model;
[0009] A lightweight semi-supervised denoising network with adaptive weight embedding is constructed and trained using the preprocessed dataset, wherein the network includes a student model and a teacher model;
[0010] Clinical low-dose CT images are denoised by embedding the adaptive weights into a lightweight semi-supervised denoising network.
[0011] In a possible implementation, dividing the medical image data into a training set, a validation set, and a test set includes:
[0012] Poisson noise of a preset proportion is added to the conventional dose CT image to simulate low dose CT data to obtain a set of labeled simulated low dose CT data sets;
[0013] Then, another two groups are selected to be unlabeled clinical low-dose CT datasets, wherein the image resolution of the unlabeled clinical low-dose CT datasets is the same as that of the labeled simulation low-dose CT datasets;
[0014] The labeled simulated low-dose CT dataset and the unlabeled clinical low-dose CT dataset are divided into a training set, a validation set, and a test set in a ratio of 8:1:1, respectively.
[0015] In a possible implementation, performing data augmentation preprocessing on the training set to expand the data set and improve the generalization ability of the model includes:
[0016] First, each image with the original resolution of the first pixel is randomly cropped into 16 small patch images of the second pixel;
[0017] Normalizing the pixel values of the small patch image so that their value range falls between 0 and 1;
[0018] Finally, during the training process, online data enhancement is performed using random horizontal flipping, vertical flipping, and random rotation in the range of -90 degrees to 90 degrees, so as to achieve the adaptability and robustness of the network trained with preprocessed data to various transformations.
[0019] In a possible implementation, the student model includes: a supervised branch and an unsupervised branch, which share a consistent network structure, the supervised branch is trained on a labeled simulation data set, and the unsupervised branch is trained using an unlabeled data set; the teacher model uses the unsupervised branch in the student model to update its own parameters through EMA; the supervised branch and the unsupervised branch are both composed of a constructed multi-scale convolutional feature extraction module, a recursive refinement module and a soft maximum feature fusion module, and feature extraction, feature refinement and adaptive weight adjustment are performed through the multi-scale convolutional feature extraction module, the recursive refinement module and the soft maximum feature fusion module respectively.
[0020] In a possible implementation, the multi-scale convolution feature extraction module, the recursive refinement module and the soft maximum feature fusion module are used to perform feature extraction, feature refinement and adaptive weight adjustment respectively, including:
[0021] Firstly, the deep features of labeled simulated low-dose CT dataset and unlabeled clinical low-dose CT dataset are extracted through multi-scale convolutional feature extraction module;
[0022] Then, the recursive refinement module is used to optimize the extracted features to reduce the model size and improve the processing efficiency of the network;
[0023] Finally, the soft maximum feature fusion module adopts an adaptive weighted method to achieve the fusion of features from different recursive modules, so as to fully mine the hierarchical information with low parameter overhead and construct respective feature spaces for labeled simulated images and unlabeled images.
[0024] In a possible implementation, a multi-scale convolution feature extraction module, a recursive refinement module, and a soft maximum feature fusion module are constructed, including:
[0025] Construct a new multi-scale convolutional feature extraction module to fully exploit the multi-source features of labeled simulation data and unlabeled clinical low-dose CT data;
[0026] Construct a recursive refinement module to further refine the features of the multi-scale convolutional feature extraction module and reduce the number of parameters in the network training process;
[0027] The soft maximum feature fusion module is constructed to utilize the hierarchical features generated by different recursive refinement modules, while achieving effective feature fusion at a small parameter cost.
[0028] In one possible implementation, a new multi-scale convolutional feature extraction module is constructed to fully exploit the multi-source features of labeled simulation data and unlabeled clinical low-dose CT data, including:
[0029] First, a 1×1 convolutional layer is used to expand the number of channels of the input data to enhance the expressiveness of subsequent network processing;
[0030] The obtained feature map is then divided into four parts and sent to branch convolution layers with different convolution kernel sizes for more detailed and differentiated feature extraction;
[0031] Finally, the features extracted by each branch are aggregated and input into a 3×3 convolutional layer, achieving deep fusion of features.
[0032] In a possible implementation, a recursive refinement module is constructed to further refine the features of the multi-scale convolutional feature extraction module, while reducing the number of parameters in the network training process, including: the recursive refinement module includes two interconnected recursive units in the network structure, each recursive unit is composed of three parallel convolutional layers: a 1×1 convolutional layer, and two asymmetric convolutional layers with convolutional kernel sizes of 1×11 and 11×1; more detailed and diverse feature information is captured through convolutional kernels of different shapes, and the two recursive units share the same parameters during the network training process.
[0033] In one possible implementation, a soft maximum feature fusion module is constructed to utilize the hierarchical features generated by different recursive refinement modules, while achieving effective feature fusion at a low parameter cost, including:
[0034] First, the features output by each recursive refinement module are input into a pooling layer to obtain a global channel feature representation;
[0035] Then a linear layer is used to dig deeper and analyze the potential relationships between internal channels;
[0036] Finally, the soft maximum function Softmax is used to assign weights to different features of each channel.
[0037] In the embodiment of the present application, the proposed lightweight semi-supervised network not only considers the correlation between labeled data and unlabeled data, but also considers the difference between them. It can well solve the domain difference problem existing in CT images, and can realize denoising and artifact suppression of labeled simulation and unlabeled CT images. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 A schematic flow chart of a lightweight semi-supervised low-dose CT image denoising method with adaptive weights provided in an embodiment of the present application;
[0039] Figure 2 A schematic diagram of a lightweight semi-supervised denoising network with adaptive weight embedding provided in an embodiment of the present application;
[0040] Figure 3 A schematic diagram of denoising results on a labeled simulated image provided in an embodiment of the present application;
[0041] Figure 4 A schematic diagram of the denoising results on an unlabeled image provided in an embodiment of the present application. DETAILED DESCRIPTION
[0042] The present solution is described below in conjunction with the accompanying drawings and specific implementation methods.
[0043] See also Figure 1This embodiment provides a lightweight semi-supervised low-dose CT image denoising method with adaptive weights, including:
[0044] S101, dividing medical image data into a training set, a validation set and a test set, wherein the medical image data includes a labeled simulated low-dose CT data set and an unlabeled clinical low-dose CT data set.
[0045] In this embodiment, a total of three sets of medical image data sets were used for evaluation. The first is a set of labeled simulated low-dose CT data sets, in which the image resolution is 512×512 pixels. 25% Poisson noise is added to the conventional dose CT image to simulate low-dose CT data. The data set includes scan data of 10 anonymous patients, from which the data of 8 patients are selected as training sets, and the data of the remaining 2 patients are used as validation sets and test sets respectively. The other two groups are unlabeled clinical low-dose CT data sets, which also have an image resolution of 512×512 pixels. In order to match the number of samples in the labeled data set, the data of 8 patients are also selected from each of the two unlabeled data sets as training sets, and the data of the remaining 2 patients are used as validation sets and test sets respectively.
[0046] S102, performing data enhancement preprocessing on the training set to expand the data set and improve the generalization ability of the model.
[0047] In order to expand the data set and improve the generalization ability of the model, this embodiment adopts a data enhancement strategy. Specifically, each image with an original resolution of 512×512 pixels is first randomly cropped into 16 small patch images of 128×128 pixels, and the pixel values of these images are normalized so that their value range falls between 0 and 1. Then, several online data enhancement techniques are used in the training process, including random horizontal flipping, vertical flipping, and random rotation in the range of -90 degrees to 90 degrees, to further enhance the adaptability and robustness of the network to various transformations.
[0048] S103, constructing an adaptive weight embedding lightweight semi-supervised denoising network and training it through the preprocessed data set, wherein the network includes a student model and a teacher model.
[0049] like Figure 2 As shown, this embodiment constructs an adaptive weight embedding lightweight semi-supervised denoising network, which includes a student model and a teacher model. The student model consists of two main branches: a supervised branch and an unsupervised branch, which share a consistent network structure. The supervised branch is trained on a labeled simulation data set, while the unsupervised branch is trained using an unlabeled data set. The teacher model uses the unsupervised branch in the student model to update its own parameters through EMA.
[0050] Both the supervised branch and the unsupervised branch are composed of a constructed multi-scale convolutional feature extraction module, a recursive refinement module, and a soft maximum feature fusion module, which are responsible for feature extraction, feature refinement, and adaptive weight adjustment, respectively. Specifically, the multi-scale convolutional feature extraction module (MCFEM) is first used to effectively extract deep features of labeled simulation and unlabeled data. Subsequently, the extracted features are optimized using the recursive refinement module (RECM) to reduce the model size and improve the processing efficiency of the network. Finally, the soft maximum feature fusion module (SFFM) is used to achieve the fusion of features from different recursive modules using an adaptive weighting method, thereby fully mining hierarchical information with a low parameter overhead and constructing respective feature spaces for labeled simulation images and unlabeled images.
[0051] This embodiment proposes an innovative multi-scale convolutional feature extraction module for the problem of multi-source feature extraction of labeled simulation data and unlabeled clinical low-dose CT data. The module can fully mine and utilize the rich information contained in these two types of data.
[0052] In the design of the network structure, a 1×1 convolution layer is first used to expand the number of channels of the input data to enhance the expressive power of the network's subsequent processing. Then, the obtained feature map is divided into four parts and sent to branch convolution layers with different convolution kernel sizes for more detailed and differentiated feature extraction. These branch structures can capture multi-scale features from subtle to macroscopic in parallel. Finally, the features extracted by each branch are aggregated and input into a 3×3 convolution layer, achieving deep fusion of features.
[0053] The advantage of this design structure lies in its parallel multi-branch structure, which not only effectively extracts multi-scale features, but also maintains a wider feature map to retain more spatial information, thereby providing strong spatial context information support for subsequent high-level semantic analysis. In addition, the modular network design greatly improves the flexibility and generalization ability of the entire network structure, enabling the network to better adapt to diverse data distributions. At the same time, by introducing a 1×1 convolution kernel into the network, the computational complexity and number of parameters of the model are effectively reduced, which not only reduces the computational burden during model training, but also alleviates the risk of overfitting to a certain extent, thereby significantly improving the overall computational efficiency and scalability of the model.
[0054] In order to further refine the features of the multi-scale convolutional feature extraction module and reduce the number of parameters in the network training process, the present invention designs an efficient recursive refinement module. The module includes two interconnected recursive units in the network structure. Each recursive unit consists of three parallel convolutional layers: a 1×1 convolutional layer, and two asymmetric convolutional layers with kernel sizes of 1×11 and 11×1. This design aims to capture more detailed and diverse feature information through convolutional kernels of different shapes. It is worth noting that the two recursive units share the same parameters during the training of the network. This parameter sharing mechanism not only effectively reduces the number of parameters of the overall model and reduces the risk of overfitting, but also significantly improves the efficiency of model training. Through this ingenious design, the recursive refinement module can achieve deep mining and fine processing of features while maintaining a low computational cost, thereby providing important support for improving the performance of the entire network model.
[0055] Based on the in-depth consideration of the unique information of the multi-level feature matrices output by different recursive refinement modules, this embodiment introduces an innovative soft maximum feature fusion module in order to achieve complementary fusion of this information and thus achieve a richer and more efficient reconstruction representation effect. The design of this module aims to focus on and utilize the hierarchical features generated by different recursive refinement modules, while realizing an effective feature fusion mechanism at a low parameter cost.
[0056] In the specific design of the network structure, the features output by each recursive refinement module are first input into a pooling layer to obtain a global channel feature representation. Then, a linear layer is used to deeply explore and analyze the potential relationship between internal channels. Finally, a soft maximum function (Softmax) is used to assign weights to different features of each channel. This process not only achieves effective weighting of features, but also enhances the model's ability to identify key information while maintaining feature diversity. Through this ingeniously designed feature fusion strategy, the network model of the present invention can enhance the integrity and accuracy of feature representation while reducing the number of parameters, thereby significantly improving the entire network's processing capability and reconstruction quality for complex data.
[0057] S104, denoising the clinical low-dose CT image by embedding the adaptive weight into a lightweight semi-supervised denoising network.
[0058] In this embodiment, when testing the denoising network, only labeled and unlabeled noisy images need to be input, and there is no need to input labeled gold standards. The semi-supervised network will automatically perform image denoising and artifact suppression based on the test images. Figure 3 and Figure 4, are the gold standard, input and output images of this embodiment on the labeled simulation image, and the input and output images on the unlabeled image, respectively. Figure 3 and Figure 4 It can be observed that there is a lot of noise and artifacts in the input image, which has a negative impact on the doctor's diagnosis. However, the denoised image (output) not only eliminates noise and artifacts as much as possible, but also better preserves the background texture and structural details in the image. The above results show that the proposed method can indeed effectively process low-dose CT images with labeled simulation and unlabeled clinical.
[0059] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.
[0060] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A lightweight semi-supervised low-dose CT image denoising method with adaptive weights, characterized in that: include: Dividing medical image data into a training set, a validation set, and a test set, wherein the medical image data includes a labeled simulated low-dose CT data set and an unlabeled clinical low-dose CT data set; Performing data augmentation preprocessing on the training set to expand the data set and improve the generalization ability of the model; A lightweight semi-supervised denoising network with adaptive weight embedding is constructed and trained using the preprocessed dataset, wherein the network includes a student model and a teacher model; Clinical low-dose CT images are denoised by embedding the adaptive weights into a lightweight semi-supervised denoising network.
2. The lightweight semi-supervised low-dose CT image denoising method with adaptive weights according to claim 1, characterized in that: The step of dividing the medical image data into a training set, a validation set, and a test set includes: Poisson noise of a preset proportion is added to the conventional dose CT image to simulate low dose CT data to obtain a set of labeled simulated low dose CT data sets; Then, another two groups are selected to be unlabeled clinical low-dose CT datasets, wherein the image resolution of the unlabeled clinical low-dose CT datasets is the same as that of the labeled simulation low-dose CT datasets; The labeled simulated low-dose CT dataset and the unlabeled clinical low-dose CT dataset are divided into a training set, a validation set, and a test set in a ratio of 8:1:1, respectively.
3. The lightweight semi-supervised low-dose CT image denoising method with adaptive weights according to claim 1, characterized in that: The training set is subjected to data augmentation preprocessing to expand the data set and improve the generalization ability of the model, including: First, each image with the original resolution of the first pixel is randomly cropped into 16 small patch images of the second pixel; Normalizing the pixel values of the small patch image so that their value range falls between 0 and 1; Finally, during the training process, online data enhancement is performed using random horizontal flipping, vertical flipping, and random rotation in the range of -90 degrees to 90 degrees, so as to achieve the adaptability and robustness of the network trained with preprocessed data to various transformations.
4. The lightweight semi-supervised low-dose CT image denoising method with adaptive weights according to claim 1, characterized in that: The student model includes: a supervised branch and an unsupervised branch, which share a consistent network structure. The supervised branch is trained on a labeled simulation data set, while the unsupervised branch is trained using an unlabeled data set. The teacher model uses the unsupervised branch in the student model to update its own parameters through an exponential moving average (EMA). The supervised branch and the unsupervised branch are both composed of a constructed multi-scale convolutional feature extraction module, a recursive refinement module, and a soft maximum feature fusion module, and feature extraction, feature refinement, and adaptive weight adjustment are performed through the multi-scale convolutional feature extraction module, the recursive refinement module, and the soft maximum feature fusion module.
5. The lightweight semi-supervised low-dose CT image denoising method with adaptive weights according to claim 4, characterized in that: The multi-scale convolution feature extraction module, the recursive refinement module and the soft maximum feature fusion module are used to perform feature extraction, feature refinement and adaptive weight adjustment respectively, including: Firstly, the deep features of labeled simulated low-dose CT dataset and unlabeled clinical low-dose CT dataset are extracted through multi-scale convolutional feature extraction module; Then, the recursive refinement module is used to optimize the extracted features to reduce the model size and improve the processing efficiency of the network; Finally, the soft maximum feature fusion module adopts an adaptive weighted method to achieve the fusion of features from different recursive modules, so as to fully mine the hierarchical information with low parameter overhead and construct respective feature spaces for labeled simulated images and unlabeled images.
6. The lightweight semi-supervised low-dose CT image denoising method with adaptive weights according to claim 4 or 5, characterized in that: Construct a multi-scale convolution feature extraction module, a recursive refinement module, and a soft maximum feature fusion module, including: Construct a new multi-scale convolutional feature extraction module to fully exploit the multi-source features of labeled simulation data and unlabeled clinical low-dose CT data; Construct a recursive refinement module to further refine the features of the multi-scale convolutional feature extraction module and reduce the number of parameters in the network training process; The soft maximum feature fusion module is constructed to utilize the hierarchical features generated by different recursive refinement modules, while achieving effective feature fusion at a small parameter cost.
7. The lightweight semi-supervised low-dose CT image denoising method with adaptive weights according to claim 6, characterized in that: A new multi-scale convolutional feature extraction module is constructed to fully exploit the multi-source features of labeled simulation data and unlabeled clinical low-dose CT data, including: First, a 1×1 convolutional layer is used to expand the number of channels of the input data to enhance the expressiveness of subsequent network processing; The obtained feature map is then divided into four parts and sent to branch convolution layers with different convolution kernel sizes for more detailed and differentiated feature extraction; Finally, the features extracted by each branch are aggregated and input into a 3×3 convolutional layer, achieving deep fusion of features.
8. The lightweight semi-supervised low-dose CT image denoising method with adaptive weights according to claim 6, characterized in that: A recursive refinement module is constructed to further refine the features of the multi-scale convolutional feature extraction module and reduce the number of parameters in the network training process, including: the recursive refinement module contains two interconnected recursive units in the network structure, each recursive unit consists of three parallel convolutional layers: a 1×1 convolutional layer, and two asymmetric convolutional layers with kernel sizes of 1×11 and 11×1; more detailed and diverse feature information is captured through convolutional kernels of different shapes, and the two recursive units share the same parameters during the network training process.
9. The lightweight semi-supervised low-dose CT image denoising method with adaptive weights according to claim 6, characterized in that: Constructing a soft maximum feature fusion module utilizes the hierarchical features generated by different recursive refinement modules and realizes effective feature fusion at a small parameter cost, including: First, the features output by each recursive refinement module are input into a pooling layer to obtain a global channel feature representation; Then a linear layer is used to dig deeper and analyze the potential relationships between internal channels; Finally, the soft maximum function Softmax is used to assign weights to different features of each channel.