A self-supervised learning method and its application

Through self-supervised learning methods, noise modeling and image block replacement are carried out in low-dose CT images, and the network is trained using the Unet structure and Adam optimizer, which solves the problems of poor imaging quality and lack of data in low-dose CT, and achieves high-quality CT Imaging and low radiation doses, suitable for clinical applications.

CN111860836BActive Publication Date: 2024-12-03NAT INST OF ADVANCED MEDICAL DEVICES SHENZHEN
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010693415.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-07-17
Publication Date
2024-12-03
Estimated Expiration
2040-07-17

AI Technical Summary

Technical Problem

The imaging quality is poor under the existing low-dose CT imaging technology, and due to the lack of paired data, it is difficult for neural networks to effectively learn the mapping of low-dose CT images to normal-dose CT images in clinical applications, resulting in difficulty in reducing radiation dose and image reconstruction speed. Slow, there are problems with quantum noise and metal artifacts.

Method used

Using self-supervised learning method, image noise modeling on low-dose CT images, randomly crop image blocks, replace pixel points, train the network using Unet structure and Adam optimizer to gradually reach the convergence state, and use mean square error loss function to realize the image Noise reduction, training neural networks without normal dose CT images.

Benefits of technology

Significantly improve the quality of CT imaging under low dose conditions, reduce radiation dose, realize the sparse angle of image reconstruction, break through the requirements of traditional convolutional neural networks for paired data, improve the quality and speed of low dose CT imaging, and is suitable for clinical lack Paired data situation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111860836B_ABST
    Figure CN111860836B_ABST
Patent Text Reader

Abstract

The present application belongs to the technical field of computer tomography (CT) systems in the medical and industrial fields, and in particular, to a self-supervised learning method and application. The existing CT imaging quality under low-dose conditions is poor. The present application provides a self-supervised learning method, which includes: modeling image noise; generating an approximate target pixel value of the current pixel from a low-dose CT image to obtain a target pixel point; randomly cropping an image block from a low-dose CT input image, randomly selecting N pixels from the image block, replacing the currently selected pixel point with the target pixel point to obtain a target image; training the network to gradually reach a convergence state. The network can be trained end-to-end without manual intervention, and noise reduction can be achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the technical field of computer tomography (CT) systems in the medical and industrial fields, and in particular to a self-supervised learning method and application. Background Art

[0002] Computed Tomography (CT) is a non-invasive imaging method that uses computers and X-rays to obtain tomographic images of the patient's body. It has the advantages of short scanning time, low cost and wide range of disease monitoring, and is suitable for early screening of diseases and routine physical examinations. However, a large amount of X-ray exposure will have a cumulative effect of radiation dose, greatly increasing the possibility of various diseases, thereby affecting the physiological functions of the human body, damaging human tissues and organs, and even endangering the life safety of patients.

[0003] The rational application of low-dose CT imaging technology requires that the clinical diagnostic requirements of CT images be met while reducing the radiation dose of X-rays to patients as much as possible. Therefore, research and development of CT imaging with higher imaging quality under low-dose conditions have important scientific significance and broad application prospects for the current medical diagnosis field. However, it is currently difficult to obtain paired CT images (low-dose CT images and corresponding normal-dose CT images) in the clinical stage. In the case of only low-dose CT images, how to use neural networks to learn the mapping from low-dose CT images to normal-dose CT images has great development prospects in the field of clinical applications.

[0004] Reducing X-ray radiation during CT imaging will cause a large amount of quantum noise and metal artifacts in the reconstructed image; the amount of data required to collect normal CT imaging is large, resulting in slow image reconstruction; the scanning time is long, and the movement of the patient's physiological functions causes image artifacts; the neural network-based method must be trained using paired data, and it is difficult to obtain paired data in the clinical stage, which greatly reduces the possibility of clinical application.

[0005] The existing CT imaging quality under low-dose conditions is poor. Summary of the invention

[0006] 1. Technical problems to be solved

[0007] Based on the problem of poor CT imaging quality under existing low-dose conditions, the present application provides a self-supervised learning method and application.

[0008] 2. Technical solution

[0009] In order to achieve the above-mentioned purpose, the present application provides a self-supervised learning method, which comprises the following steps:

[0010] Step 1: Model image noise;

[0011] Step 2: Generate an approximate target pixel value of the current pixel from the low-dose CT image to obtain the target pixel point;

[0012] Step 3: randomly crop an image block from the low-dose CT input image, randomly select N pixels from the image block, and replace the currently selected pixels with the target pixels to obtain the target image;

[0013] Step 4: Train the network and gradually reach convergence.

[0014] Another implementation manner provided by the present application is: the noisy image in step 1 is composed of a clean image and noise.

[0015] Another implementation method provided by the present application is: in step 2, the current pixel point x of the input low-dose CT image is i j A pixel point is randomly selected from the 5×5 neighborhood of as the target pixel point of the current pixel point.

[0016] Another implementation method provided by the present application is: in step 3, a 64×64 pixel image block x is randomly cut out from the low-dose CT input image. j .

[0017] Another implementation method provided by the present application is that the image block in step 3 is larger than the receptive field of the selected convolutional neural network.

[0018] Another implementation manner provided by the present application is: in step 3, N is one tenth of the size of the image block.

[0019] Another implementation manner provided by the present application is: the training network in the step 4 includes calculating the loss of the N pixels.

[0020] Another implementation manner provided by the present application is that the loss function is a mean square error loss function.

[0021] Another implementation method provided by the present application is: the entire network framework adopts a Unet structure, and the network is optimized using an Adam optimizer.

[0022] The present application also provides an application of self-supervised learning, which applies the self-supervised learning method described in any one of claims 1 to 9 to CT image denoising or MRI image denoising.

[0023] 3. Beneficial effects

[0024] Compared with the prior art, the self-supervised learning method provided by the present application has the following beneficial effects:

[0025] The self-supervised learning method provided in this application is a self-supervised learning method for low-dose CT image denoising.

[0026] The self-supervised learning method provided in the present application is based on the idea of ​​the self-supervised learning method. In the absence of labels (normal-dose CT images), the method of the present application can generate label values ​​from input data (low-dose CT images), thereby meeting the training requirements of the neural network, so that the neural network can acquire the ability to reduce noise in CT images under low-dose conditions through learning.

[0027] The use of image prior information in the self-supervised learning method provided in this application will facilitate the reconstruction of sparse angle low-dose CT images, thereby obtaining high-quality CT images while significantly reducing the radiation dose.

[0028] The self-supervised learning method provided in this application generates a target image by randomly replacing the neighborhood of some pixels in the input low-dose CT image, thereby breaking through the requirements of traditional convolutional neural networks for training integrated data. While abandoning the shortcomings of supervised learning, it fully utilizes the powerful feature extraction capabilities of convolutional neural networks, greatly improving the quality of CT imaging under low-dose conditions.

[0029] The self-supervised learning method provided in this application utilizes the powerful feature extraction capability of convolutional neural networks, and can train the network end-to-end and achieve noise reduction without human intervention.

[0030] The self-supervised learning method provided in this application does not require normal dose CT images and has low data requirements, which has great advantages in the absence of paired data in clinical practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is the first schematic diagram of the convolutional neural network based on supervised learning of the present application;

[0032] Figure 2 This is the second schematic diagram of the convolutional neural network based on supervised learning of this application. DETAILED DESCRIPTION

[0033] Hereinafter, the specific embodiments of the present application will be described in detail with reference to the accompanying drawings. According to these detailed descriptions, those skilled in the art can clearly understand the present application and can implement the present application. Without violating the principles of the present application, the features in the various embodiments can be combined to obtain new implementations, or certain features in certain embodiments can be replaced to obtain other preferred implementations.

[0034] In 2017, Hu Chen et al. published an article titled "Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)" in the journal IEEE Transactions on Medical Imaging. They applied the deep residual convolutional neural network with encoder-decoder structure to the problem of improving the quality of low-dose CT images for the first time, making the structure of CT images acquired under low-dose conditions clearer. The encoder part consists of 5 layers of convolution with a kernel size of 5×5 plus ReLU activation function, and the decoder part consists of 5 layers of deconvolution corresponding to the encoder plus ReLU activation function. In addition, the convolution and deconvolution layers corresponding to the encoder and decoder use residual connections similar to the residual network, and the final network output is the denoised CT image.

[0035] In 2017, Jelmer M. Wolterink et al. published an article titled "Generative Adversarial Networks for Noise Reduction in Low-Dose CT" in the journal IEEE Transactions on Medical Imaging, successfully applying generative adversarial networks (GANs) to the field of low-dose CT imaging. The convolution kernel size used by the generator was 3×3×3, and the number of convolution kernels gradually increased from 32 at the beginning to 64, and finally to 128. The pooling operation was removed, and the LeakyReLU activation function was used after all convolution layers to improve training stability. In addition, to ensure that the generator learns the noise part in the low-dose CT image, the final output of the generator is the input image minus the output image of the last layer of convolution of the generator, to obtain the denoised image; the discriminator part uses a convolution kernel of size 3×3×3, a LeakyReLU activation function, and a batch normalization operation, and finally outputs the prediction through a fully connected layer and a Sigmoid activation function.

[0036] See also Figures 1-2 , the present application provides a self-supervised learning method, the method comprising the following steps:

[0037] Step 1: Model image noise;

[0038] Step 2: Generate an approximate target pixel value of the current pixel from the low-dose CT image to obtain the target pixel point;

[0039] Step 3: randomly crop an image block from the low-dose CT input image, randomly select N pixels from the image block, and replace the currently selected pixels with the target pixels to obtain the target image;

[0040] Step 4: Train the network and gradually reach convergence.

[0041] Input means that the low-dose CT image is used as the input of the network, the image obtained after replacing the pixels is used as the target image, the loss between the output of the network and the target image is calculated, and the network is trained.

[0042] To some extent, the noisy image can be considered as a combination of the clean image and noise, that is:

[0043] x=s+n (1)

[0044] Among them, x represents the noisy image, s represents the clean image, and n represents the noise and artifacts of the image. So simply put, the image denoising task is to separate the noisy image x into two parts: s and n, and remove the noise n to get the clean image s.

[0045] Denoising methods are usually based on the assumption that pixels i and j in the clean image s are not statistically independent, that is:

[0046] p(s i |s j )≠p(s i ) (2)

[0047] That is to say, the pixel value of any point in the image is related to the values ​​of other pixels in the image. In other words, by observing the size and distribution of the pixel values ​​around the current pixel, the pixel value of the current pixel can be accurately predicted. This interdependence between pixels has become the basis of this application.

[0048] Furthermore, the noisy image in step 1 is composed of a clean image and noise.

[0049] Furthermore, in step 2, the current pixel point of the input low-dose CT image is A pixel point is randomly selected from the 5×5 neighborhood of as the target pixel point of the current pixel point.

[0050] Furthermore, in step 3, a 64×64 pixel image block x is randomly cropped from the low-dose CT input image. j .

[0051] Furthermore, in step 3, the image block is larger than the receptive field of the selected convolutional neural network.

[0052] Furthermore, in step 3, N is one tenth of the size of the image block.

[0053] Furthermore, the training of the network in step 4 includes calculating the loss of the N pixels.

[0054] Furthermore, the loss function is a mean square error loss function.

[0055] Furthermore, the overall network framework adopts a Unet structure, and the network is optimized using an Adam optimizer.

[0056] When using supervised learning methods to achieve low-dose CT denoising tasks, low-dose CT noise images are generally regarded as the input of the convolutional neural network, and normal-dose CT images are used as targets. The mapping from low-dose CT images to normal CT dose images is learned through the back-propagation algorithm. When the network converges, the learned convolutional neural network can achieve low-dose CT denoising.

[0057] Schematic diagram of denoising method based on convolutional neural network Figure 1 As shown, each predicted pixel value output by the convolutional neural network They are all composed of a fixed size receptive field area x RF(i) That is, the pixels that affect the predicted pixels are the set of pixels in the receptive field area. At this time, the convolutional neural network can be regarded as the input of the receptive field area x RF(i) , the output is the predicted value of the central pixel of the receptive field The function is:

[0058]

[0059] The parameters θ are the learnable parameters of the convolutional neural network.

[0060] In supervised learning, paired training data (x j ,s j ), where x j and j represent the j-th low-dose CT image and the corresponding normal-dose CT image, respectively. represents the area centered on pixel i in the j-th low-dose CT image in the training dataset, represents the pixel i in the jth normal dose CT image in the training data set, Input to the convolutional neural network, As the target value, minimize the predicted value and target value of the convolutional neural network, that is:

[0061]

[0062] in:

[0063]

[0064] By minimizing the above formula, the neural network parameters θ can gradually meet the requirements of the denoising task.

[0065] Convolutional neural networks based on supervised learning must be trained using paired training data, but it is difficult to obtain paired CT images in the clinical stage. Usually we can only obtain CT images under low-dose conditions.

[0066] Self-supervised learning is to generate targets from inputs through certain methods, so that the training of the neural network can proceed normally. This application is based on the idea of ​​self-supervised learning algorithm. In the case of Therefore, the network can be trained using the following objective function:

[0067]

[0068] How to generate an approximate target pixel value for the current pixel from a low-dose CT image This is the focus of this application. As mentioned above, image pixels are not statistically independent, that is, there is a certain correlation between pixels, and the closer two pixels are in space, the higher the correlation between them. In layman's terms, in the low-frequency area of ​​the image, the difference between the pixel values ​​of the adjacent pixel and the current pixel is small, and there are a large number of low-frequency areas in the CT image. Therefore, it is considered that in the training stage, the current pixel of the input low-dose CT image can be Randomly select a pixel point in the 5×5 neighborhood of as the target pixel point of the current pixel point, that is:

[0069]

[0070] Now that we know how to obtain the target pixel points, we can describe the entire training process.

[0071] First, a 64×64 pixel image patch x is randomly cropped from the low-dose CT input image. j , the size of the image block is larger than the receptive field size of the network we selected, then we randomly select N pixels from the image block, where the size of N is set to one tenth of the image block size, and then randomly select pixels in the 5×5 neighborhood of the selected pixels to replace the currently selected pixels to obtain the target image x j~N At this time, the low-dose CT image block x j As input, the replaced image patch x j~N As a goal, only the loss of these N pixels is calculated during training, that is:

[0072]

[0073] The loss function L uses the mean square error loss function;

[0074] The overall network framework uses the common Unet structure;

[0075] The entire network is optimized using the Adam optimizer;

[0076] Extract image patches from a low-dose CT image dataset as input and generate targets through the input;

[0077] Train the network and gradually reach convergence.

[0078] The present application also provides an application of self-supervised learning, wherein the self-supervised learning method described in any one of claims 1 to 9 is applied to CT image denoising or MRI image denoising. For MRI images, it is MRI noise image. Of course, it can also be applied to other types of medical image denoising.

[0079] This application is not targeted at a specific network structure, but is universal and can be applied to any network structure.

[0080] This application can implement the denoising task end-to-end and train the network without paired data.

[0081] Although the present application is described above with reference to specific embodiments, it should be understood by those skilled in the art that many modifications may be made to the configurations and details disclosed in the present application within the principles and scope disclosed in the present application. The scope of protection of the present application is determined by the appended claims, and the claims are intended to cover all modifications included in the equivalent literal meaning or scope of the technical features in the claims.

Claims

1. A self-supervised learning method, characterized in that: The method comprises the following steps: Step 1: Model the image noise. The image consists of a clean image and noise, expressed as: x=s+n Among them, x represents the noisy image, s represents the clean image, and n represents the noise and artifacts of the image; Step 2: Generate an approximate target pixel value of the current pixel from the image, and obtain the target pixel point including: obtaining an approximate target pixel value of the current pixel point from the input low-dose CT image Randomly select a pixel point in the 5×5 neighborhood of as the target pixel point of the current pixel point; Step 3: Randomly crop a 64×64 pixel image patch x from the low-dose CT input image j , the image block is larger than the receptive field of the selected convolutional neural network, randomly select N pixels from the image block, N is one tenth of the size of the image block, and replace the target pixel with the currently selected pixel to obtain the target image x j~N ; Step 4: Train the network to gradually reach a convergence state. The training network includes calculating the loss of the N pixels, expressed as: Among them, the loss function L uses the mean square error loss function.

2. The self-supervised learning method according to any one of claims 1, characterized in that: The overall network framework adopts the Unet structure, and the network is optimized using the Adam optimizer.

3. An application of self-supervised learning, characterized in that: The self-supervised learning method according to any one of claims 1 to 2 is applied to CT image denoising or MRI image denoising.