Improved reggan low-dose ct image denoising method and related device

Abstract

The application discloses an improved RegGAN low-dose CT image denoising method and related device, which comprises the following steps: collecting chest and abdominal CT images formed by using a low-dose technology and paired chest and abdominal CT images formed by using a normal dose which are not strictly aligned; randomly dividing the paired CT image data set into a training set and a test set; giving a model loss function and an optimizer, and setting appropriate hyperparameters to train a low-dose CT image denoising model; and using the trained model to perform high-quality image denoising and restoration on the low-dose CT image. The application makes the training more stable by using spectral normalization to constrain the weight parameters of the discriminator network, and enables the denoised image to retain more texture information by adding a perception loss function. The application provides a better choice for applying deep learning technology to clinical low-dose CT image denoising, and not only has no strict requirements for the data set, but also can meet the diagnostic requirements in terms of image quality.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning and medical CT, and particularly relates to an improved RegGAN low-dose CT image denoising method and a related device. BACKGROUND

[0002] CT (Computed Tomography), namely, electronic computed tomography, is a technology of reconstructing a tomographic image of an object according to X-ray attenuation in multiple directions penetrating the object, and has the characteristics of short scanning time and clear formed image, and is widely used in clinical screening, diagnosis and auxiliary surgery at present. However, the CT has a certain carcinogenic risk due to the excessively high X-ray radiation dose. One of the main methods to reduce the CT radiation dose level is to reduce the exposure level of each projection angle, but the lower X-ray dose will introduce a certain amount of noise in the CT image, thereby affecting the accuracy of the doctor's diagnosis. Therefore, low-dose CT image denoising has become a hot issue in recent years.

[0003] At present, most of the image denoising methods based on deep learning are performed by learning the mapping from a low-quality image to a high-quality image, however, these methods need to collect a large number of pixel-by-pixel aligned paired images in advance as the input and output of network training. For medical images, it is unrealistic to obtain enough qualified paired training samples. Some scholars have proposed a RegGAN network for image denoising. The RegGAN regards the noise of the non-strictly aligned paired images as a deformation error, and trains an additional registration network after the generator to predict a deformable vector field. However, in the actual scene, the image denoised by the RegGAN network still produces a large blur on the edge and part of the content structure, and the training process is unstable and prone to mode collapse problems. SUMMARY

[0004] The technical problem to be solved by the present application is to provide an improved RegGAN low-dose CT image denoising method and a related device to solve the problems that the image denoised by the RegGAN network still produces a large blur on the edge and part of the content structure, and the training process is unstable and prone to mode collapse problems.

[0005] To achieve the above-mentioned purpose, the following technical solutions are adopted in the present application:

[0006] An improved RegGAN low-dose CT image denoising method, comprising the following steps:

[0007] Collecting a chest and abdominal CT image formed using a low-dose technology and a non-strictly aligned paired chest and abdominal CT image formed using a normal dose;

[0008] The paired CT image data sets are randomly divided into a training set and a test set;

[0009] Given the model loss function and the optimizer, and set appropriate hyperparameters, train the low-dose CT image denoising model;

[0010] Using the trained model to perform high-quality image denoising and restoration on low-dose CT images.

[0011] Further, the size of the collected single image needs to be the same, and the CT images collected using low-dose technology and normal dose need to be paired one by one, that is, each low-dose CT image has a corresponding normal-dose CT image at the same position.

[0012] Further, the network structure of the generator in the low-dose CT image denoising model is divided into three parts: the down-sampling stage, the feature extraction stage, and the up-sampling stage; In the down-sampling stage of the model generator, a Sobel convolution operator is added, the extracted CT image edge information is fused with the input and sent to the convolution network, and in the feature extraction stage, a channel attention mechanism is used to give different weights to the channels to model the correlation between features.

[0013] Further, the network structure of the down-sampling stage is sequentially a Sobel convolution operator layer, a convolution layer, a normalization layer, an activation function layer, a convolution layer, a normalization layer, an activation function layer, a convolution layer, a normalization layer, and an activation function layer.

[0014] Further, the feature extraction stage is stacked by nine residual modules and an attention module, and the structure of the attention module is sequentially a global average pooling layer, a fully connected layer, an activation function layer, a fully connected layer, and an activation function layer; The structure of the residual module is sequentially an edge padding layer, a convolution layer, a normalization layer, an activation function layer, an edge padding layer, a convolution layer, and a normalization layer, and is spliced with the input to obtain the output.

[0015] Further, a deeper convolution network is used in the low-dose CT image denoising model instead of the original discriminator, and a self-attention mechanism is added to make the discriminator pay more attention to important information in the image; The structure of the model discriminator with the self-attention mechanism is sequentially seven spectral normalization convolution modules, a self-attention mechanism layer, a spectral normalization convolution module, a self-attention mechanism layer, and a spectral normalization convolution module, wherein the spectral normalization convolution module is composed of a convolution layer and a LeakyReLU activation function layer, and the weight tensor of the convolution layer is adjusted through spectral normalization.

[0016] Further, the loss function part adds a perception loss, so that the model discriminator pays more attention to the texture information when distinguishing the denoised CT image generated by the generator from the real noise-free image; after the model training is completed, the model weight file saved during the training process is called, the low-dose CT image in the test set is input, and the high-quality CT image after denoising is obtained through the generator.

[0017] Further, an improved RegGAN low-dose CT image denoising system comprises:

[0018] The acquisition module is used for acquiring the chest and abdominal CT images formed by using the low-dose technology and the paired chest and abdominal CT images formed by using the normal dose without strict alignment;

[0019] The classification module is used for randomly dividing the paired CT image dataset into a training set and a test set;

[0020] The training module is used for setting appropriate hyperparameters for the given model loss function and optimizer, and training the low-dose CT image denoising model;

[0021] The denoising module is used for performing high-quality image denoising and restoration on the low-dose CT image by using the trained model.

[0022] Further, a computer readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform an improved RegGAN low-dose CT image denoising method.

[0023] Further, a computing device comprises:

[0024] One or more processors, memories and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise steps for performing an improved RegGAN low-dose CT image denoising method.

[0025] Compared with the prior art, the present application has at least the following beneficial effects:

[0026] The improved RegGAN low-dose CT image denoising method dynamically measures the similarity between the denoised image and the noise-free image in the training process through the GAN framework, so that the low-dose CT image can fully participate in the deep network training without the guidance of the corresponding pixel-by-pixel alignment high-quality image. The present application is based on the RegGAN network and makes improvements: a Sobel convolution operator is added to enhance the edge information of the CT image in the model, and a channel attention mechanism is used to give different weights to the feature map channels to model the correlation between the features; a deeper convolution network is used instead of the original model discriminator, and a self-attention mechanism is added to make the discriminator pay more attention to important information in the image; the spectral normalization is used to constrain the weight parameters of the discriminator network to make the training more stable; the perception loss function is added to make the denoised image retain more texture information. The present application provides a better choice for the application of deep learning technology in clinical low-dose CT image denoising, which not only has no strict requirements for the data set, but also can meet the diagnostic requirements in terms of image quality.

[0027] Further, for each scan object, two sets of CT image data sets are collected using low-dose and normal-dose, the low-dose CT image with more noise is used as the input data, and the normal-dose CT image with less noise is used as the label, i.e. the training target of the model, then the error between the model output and the expected output is calculated according to the loss function, and the training is carried out through the back propagation algorithm. If the input image and the label are not paired, the model training purpose cannot be well achieved, therefore, during the model training stage, the input image and the label should be paired.

[0028] Further, the network structure of the generator in the low-dose CT image denoising model is divided into three parts: the down-sampling stage, the feature extraction stage and the up-sampling stage. The input image first goes through the down-sampling stage, i.e. an encoding process, which improves the receptive field and perceives the image through the stacking of convolutional layers, learns the implicit features of the data; the output feature map goes through the feature extraction stage, which fully extracts and fuses the detailed information and contour information extracted by the network; finally, it goes through the up-sampling stage, i.e. a decoding process, which uses the learned new features to restore the CT image, achieving the denoising effect. The generator network can effectively suppress noise and at the same time maintain the resolution as much as possible by modeling the image degradation model.

[0029] Further, the Sobel convolution operator added in the down-sampling stage of the model generator can extract the edge structure of the CT image to enhance the effect of edge information on the model; the channel attention mechanism added through the network automatic learning method obtains the importance of each channel of the feature map, and gives each channel a weight value, so that the network focuses on the effective feature map channel and suppresses the useless feature map channel, explicitly modeling the interdependence between the feature channels to enhance the representation ability of the network and improve the denoising effect of the model.

[0030] Further, deeper networks are prone to vanishing and exploding gradients, while residual connections make the propagation of gradients more stable in the network, and the model training effect is better. On the other hand, the attention mechanism module can help the model select a small part of useful information from a large amount of input information for intensive processing by the weight parameter allocation mechanism, and ignore other invalid information. Therefore, through the stacking of residual modules and attention modules, the ability of the generator to extract features is greatly improved.

[0031] Further, the network based on the convolutional layer depends on the cascaded convolutional layer to gradually improve the receptive field, and as the number of network layers increases, the expression ability and abstraction ability of the network will also increase, and the deeper the network, the more semantic information the extracted features have. However, the receptive field of the convolutional layer is still limited, and the cascaded convolutional layer is inefficient in modeling various structural information. In order to overcome this limitation, the added self-attention mechanism captures global dependencies by calculating the interaction between any two positions, compensates for the shortcomings of convolution, and enhances the discrimination ability of the discriminator network.

[0032] Further, in the loss function, although the root mean square error loss function commonly used in the denoising task helps to improve the peak signal-to-noise ratio of the super-resolution task, it also makes part of the details of the image blurred. The added perceptual loss function sends the denoised image generated by the generator and the real noise-free image into the VGG network with fixed parameters for feature extraction, and then uses the root mean square error on the extracted feature map. Therefore, the perceptual loss makes the CT image reconstructed by the model closer to the real normal dose CT image in terms of low-level pixel values and high-level abstract features.

[0033] In summary, the CT image denoising model built by the application can better fit the noise distribution between the paired images, and find the common optimal solution for the image-to-image conversion and registration task. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 It is a flowchart of the low-dose CT image denoising method of the application;

[0035] Figure 2 It is a CT sample schematic diagram provided by the application, wherein (a) is a chest and abdominal low-dose CT sample, and (b) is a corresponding normal dose CT sample (b) schematic diagram;

[0036] Figure 3 It is a schematic diagram of the overall structure of the low-dose CT image denoising model provided by the application;

[0037] Figure 4 It is a schematic diagram of the generator structure in the low-dose CT image denoising model provided by the application;

[0038] Figure 5 A schematic diagram of a registration network structure in a low-dose CT image denoising model provided by the present application is shown in FIG. 1.

[0039] Figure 6 A schematic diagram of a discriminator structure in a low-dose CT image denoising model provided by the present application is shown in FIG. 2.

[0040] Figure 7 A CT sample provided by the present application is shown in FIG. 3, where (a) is a low-dose CT sample, (b) is a denoised sample output by the denoising model, and (c) is a corresponding normal-dose CT sample. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0042] In the description of the present application, it should be understood that the terms “include” and “contain” indicate the presence of described features, whole, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, whole, steps, operations, elements, components and / or sets thereof.

[0043] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms “a”, “an” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0044] It should be further understood that the term “and / or” used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character “ / ” herein generally represents an “or” relationship between the front and rear associated objects.

[0045] It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe the preset ranges, etc., these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from each other. For example, the first preset range can also be referred to as the second preset range, and similarly, the second preset range can also be referred to as the first preset range without departing from the scope of the embodiments of the present application.

[0046] Depending on the context, the word "if" as used herein can be interpreted to mean "when" or "while" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be interpreted to mean "when it is determined" or "in response to determining" or "when [a stated condition or event] is detected" or "in response to detecting [a stated condition or event]."

[0047] Various structural diagrams according to the disclosed embodiments of the present application are shown in the accompanying drawings. These drawings are not drawn to scale, in which certain details are exaggerated for the purpose of clarity and some details may be omitted. The shapes of various regions, layers and their relative sizes and positional relationships shown in the drawings are only exemplary, and in actuality may be deviated due to manufacturing tolerances or technical limitations, and regions / layers with different shapes, sizes and relative positions can be additionally designed according to actual needs by those skilled in the art.

[0048] The application provides an improved RegGAN low-dose CT image denoising method, which builds a CT image denoising model based on a RegGAN network, and dynamically measures the similarity between the denoised image and the noise-free image in the training process through a GAN framework. In the context of low-dose CT denoising, the goal of the generator is to simulate the generation of a low-dose CT denoising result to deceive the discriminator; while the goal of the discriminator is to distinguish between the real low-dose CT image and the denoised image generated by the generator. The RegGAN network regards the noise of the unaligned paired images as a deformation error, and trains an additional registration network after the generator to predict the deformable vector field; the application solves the problem of difficulty in training task caused by the incomplete alignment between the low-dose CT image and the paired normal-dose CT image in the CT image data set in the actual scene to some extent. The application builds a low-dose CT image denoising model based on the RegGAN network, which can dynamically measure the similarity between the denoised image and the noise-free image in the training process, and makes certain improvements to the problems that occur in practice.

[0049] Please refer to Figure 1 The improved RegGAN low-dose CT image denoising method of the application comprises the following steps:

[0050] S1, collecting chest and abdominal CT images generated using low-dose technology and paired chest and abdominal CT images generated using normal dose but not strictly aligned;

[0051] The size of the collected single image needs to be the same, and the CT images collected under low-dose technology and normal dose need to be paired one by one, that is, each low-dose CT image has a corresponding normal-dose CT image at the same position, and the pixels between the paired images can not be completely aligned.

[0052] S2, randomly dividing the paired CT image data set into a training set and a test set;

[0053] S3, given the model loss function and the optimizer, and setting appropriate hyperparameters, training the low-dose CT image denoising model;

[0054] The network structure of the generator of the low-dose CT image denoising model can be divided into three parts: the down-sampling stage, the feature extraction stage, and the up-sampling stage. The down-sampling stage needs to pass through the Sobel convolution operator first, and then be sent to the subsequent convolution network. The Sobel operator is a common image edge detection operator used to calculate the approximate gradient of image gray scale. The larger the gradient, the more likely it is an edge. It contains a set of filters for extracting horizontal edges and a set of filters for extracting vertical edges, as shown in (1):

[0055]

[0056] The network structure of the generator of the low-dose CT image denoising model can be divided into three parts: the down-sampling stage, the feature extraction stage, and the up-sampling stage. The feature extraction stage adds the SE (Squeeze-and-Excitation) channel attention mechanism, which automatically learns the importance of each channel of the feature map through the network, and gives each channel a weight value, so that the network focuses on the effective feature map channel and suppresses the useless feature map channel, explicitly modeling the interdependence between feature channels to improve the representation ability of the network. The structure of the SE channel attention module is a global pooling layer, a fully connected layer, a ReLU activation function layer, a fully connected layer, and a Sigmoid activation function layer.

[0057] In the discriminator of the low-dose CT image denoising model, all convolution layers use spectral normalization to re-adjust the weight tensor of the convolution layer to stabilize the training of the discriminator, so that the neural network has good non-sensitivity to input disturbance, thereby making the training process of the GAN more stable and easier to converge.

[0058] The loss function includes four parts: the smoothness loss of the deformable vector field learned by the registration network, the adversarial loss of the GAN network, the L1 loss between the image denoised by the model and the corresponding real image, and the perceptual loss of the image generated by the GAN generator.

[0059] The commonly used neural network training method is the back propagation algorithm, and the learning process of the back propagation algorithm is composed of a forward propagation process and a back propagation process. In the forward propagation process, the input is calculated through the neural network and the output is obtained, and the error between the network output and the expected output is calculated according to the loss function; the error calculated in the forward propagation process is transmitted into the back propagation process, and the partial derivative of the loss function with respect to each neuron weight is calculated layer by layer as the gradient of the objective function with respect to the weight, and the network weight is updated according to the gradient. The weights of the neural network model are adjusted in the iterative back propagation process until convergence.

[0060] S4, using the trained model to perform high-quality image denoising and restoration on the low-dose CT image.

[0061] After the model training is completed, the model weight file saved in the training process is called, and the low-dose CT image in the test set is input, and only the generator part is needed to obtain the denoised high-quality CT image.

[0062] The low-dose CT image denoising model network structure comprises:

[0063] (1) The generator network structure can be divided into three parts.

[0064] The first part is a down-sampling stage, and the structure is sequentially a Sobel convolution operator layer, a convolution layer, a normalization layer, a ReLU activation function layer, a convolution layer, a normalization layer, a ReLU activation function layer, a convolution layer, a normalization layer and a ReLU activation function layer.

[0065] The second part is a feature extraction stage, which is sequentially stacked by nine residual modules and an attention module, and the structure of the attention module is sequentially a global average pooling layer, a full connection layer, a ReLU activation function layer, a full connection layer and a Sigmoid activation function layer; the structure of the residual module is sequentially an edge padding layer, a convolution layer, a normalization layer, a ReLU activation function layer, an edge padding layer, a convolution layer and a normalization layer, and the input is spliced to obtain the output.

[0066] The third part is an up-sampling stage, and the structure is sequentially a transposed convolution layer, a normalization layer, a ReLU activation function layer, a transposed convolution layer, a normalization layer, a ReLU activation function layer, an edge padding layer, a convolution layer and a Tanh activation function layer.

[0067] (2) The structure of the registration network is based on the ResUNet network, and can be divided into two parts.

[0068] The first part is a down-sampling stage, which is stacked by seven down-sampling modules with the same structure. The structure of a single down-sampling module is sequentially a convolution layer, a LeakyReLU activation function layer and a maximum pooling layer, and a skip layer connection structure is added.

[0069] The second part is an upsampling stage, which is stacked by seven structure-same upsampling modules. The structure of a single upsampling module is in turn a convolution layer, a LeakyReLU activation function layer, and a bilinear interpolation upsampling layer, and the feature maps corresponding to the upsampling stage are fused.

[0070] (3) The structure of the discriminator network is in turn seven spectral normalization convolution modules, a self-attention layer, a spectral normalization convolution module, a self-attention layer, a spectral normalization convolution module, wherein the spectral normalization convolution module is composed of a convolution layer and a LeakyReLU activation function layer, and the weight tensor of the convolution layer is adjusted by spectral normalization to improve the stability of model training.

[0071] In another embodiment of the present application, an improved RegGAN low-dose CT image denoising system is provided, which can be used to implement the improved RegGAN low-dose CT image denoising method described above. Specifically, the improved RegGAN low-dose CT image denoising system comprises

[0072] The acquisition module is used to acquire the chest and abdominal CT images formed by using the low-dose technology and the non-strictly aligned paired chest and abdominal CT images formed by using the normal dose.

[0073] The classification module is used to randomly divide the paired CT image dataset into a training set and a test set.

[0074] The training module is used to give a model loss function and an optimizer, and set appropriate hyperparameters to train the low-dose CT image denoising model.

[0075] The denoising module is used to use the trained model to perform high-quality image denoising and restoration on the low-dose CT image.

[0076] In still another embodiment of the present application, a terminal device is provided, which comprises a processor and a memory, the memory being configured to store a computer program, the computer program comprising program instructions, and the processor being configured to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and are particularly suitable for loading and executing one or more instructions to implement a corresponding method flow or a corresponding function; the processor in the embodiments of the present application can be used for the operation of the improved RegGAN low-dose CT image denoising method, including:

[0077] Collecting chest and abdominal CT images formed using a low-dose technique and paired chest and abdominal CT images formed using a normal dose which are not strictly aligned; randomly dividing the paired CT image data sets into a training set and a test set; given a model loss function and an optimizer, and setting appropriate hyperparameters, training a low-dose CT image denoising model; using the trained model to perform high-quality image denoising and restoration on the low-dose CT images.

[0078] In still another embodiment of the present application, the present application further provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in the terminal device, and is used for storing programs and data. It can be understood that the computer readable storage medium herein can include an internal storage medium in the terminal device, and of course can also include an expansion storage medium supported by the terminal device. The computer readable storage medium provides a storage space, and the storage space stores an operating system of the terminal. Furthermore, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory.

[0079] The one or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the improved RegGAN low-dose CT image denoising method in the above embodiments; the one or more instructions stored in the computer readable storage medium are loaded and executed by the processor to implement the following steps:

[0080] Collecting chest and abdominal CT images formed using a low-dose technique and paired chest and abdominal CT images formed using a normal dose that are not strictly aligned; randomly dividing the paired CT image data sets into a training set and a test set; given a model loss function and an optimizer, and setting appropriate hyperparameters, training a low-dose CT image denoising model; using the trained model to perform high-quality image denoising and restoration on the low-dose CT images.

[0081] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art without creative labor based on the embodiments in the present application belong to the scope of protection of the present application.

[0082] Embodiments

[0083] S1, collecting chest and abdominal CT images formed using a low-dose technique and paired chest and abdominal CT images formed using a normal dose, as shown in Figure 2 (a) and (b); the collected low-dose CT images and the corresponding normal-dose CT images are each 9524, and the size of a single image is 512x512, which together form a 9524-paired CT image data set, and the pixels between the paired images are not completely aligned.

[0084] S2, randomly dividing the paired CT image data set into a training set and a test set according to 8:2, wherein the training set has a total of 7700 pairs of CT images, and the test set has a total of 1924 pairs of CT images;

[0085] S3, training a low-dose CT image denoising model using an Adam optimizer and cooperating with an adaptive learning rate decay strategy;

[0086] The set hyperparameters include: training rounds (epoch) of 50, data batch size (batchsize) of 4, and initial learning rate of 0.0001.

[0087] The overall structure of the low-dose CT image denoising model built is as shown in Figure 3 The specific network structure and parameters are as follows:

[0088] (1) The generator network structure can be divided into three parts.

[0089] The first part is the down-sampling stage, and the structure is Sobel convolutional operator layer, 7x7 convolutional layer with a step of 1, normalization layer, ReLU activation function layer, 3x3 convolutional layer with a step of 2 and padding of 1, normalization layer, ReLU activation function layer, 3x3 convolutional layer with a step of 2 and padding of 1, normalization layer, and ReLU activation function layer.

[0090] The second part is the feature extraction stage, which is stacked by nine residual modules and an attention module. The structure of the attention module is global average pooling layer, full connection layer, ReLU activation function layer, full connection layer, and Sigmoid activation function layer. The structure of the residual module is edge padding layer, 3x3 convolutional layer, normalization layer, ReLU activation function layer, edge padding layer, 3x3 convolutional layer, and normalization layer, which are spliced with the input to obtain the output.

[0091] The third part is the up-sampling stage, and the structure is 3x3 transpose convolutional layer with a step of 2 and padding of 1, normalization layer, ReLU activation function layer, 3x3 transpose convolutional layer with a step of 2 and padding of 1, normalization layer, ReLU activation function layer, edge padding layer, 7x7 convolutional layer with a step of 1, and Tanh activation function layer.

[0092] Please refer to Figure 4 for the structure of the generator.

[0093] (2) The structure of the registration network can be divided into two parts.

[0094] The first part is the down-sampling stage, which is stacked by seven down-sampling modules with the same structure. The structure of the down-sampling module is 3x3 convolutional layer with a step of 1 and padding of 1, LeakyReLU activation function layer, and maximum pooling layer with a window size of 2. The output channel numbers of the seven down-sampling modules are 32, 64, 64, 64, 64, 64, and 64, respectively, and the skip connection structure is added.

[0095] The second part is the up-sampling stage, which is stacked by seven up-sampling modules with the same structure. The structure of the up-sampling module is 3x3 convolutional layer with a step of 1 and padding of 1, LeakyReLU activation function layer, and bilinear interpolation up-sampling layer. The output channel numbers of the seven down-sampling modules are 64, 64, 64, 64, 64, 64, and 32, respectively, and the corresponding feature maps of the up-sampling stage are fused.

[0096] The registration network structure is as shown in Figure 5 .

[0097] (3) The discriminator network structure is in turn a 3x3 convolutional layer with a step size of 1, a 3x3 convolutional layer with a step size of 2, a 3x3 convolutional layer with a step size of 1, a 3x3 convolutional layer with a step size of 1, a 3x3 convolutional layer with a step size of 2 and padding of 2, a 3x3 convolutional layer with a step size of 1 and padding of 2, a 3x3 convolutional layer with a step size of 1 and padding of 2, a self-attention mechanism layer, a 3x3 convolutional layer with a step size of 1 and padding of 2, a self-attention mechanism layer, and a 3x3 convolutional layer with a step size of 1 and padding of 2, and spectral normalization is used for all the convolutional layers to re-adjust the weight tensor of the convolutional layer to stabilize the training of the discriminator. The discriminator structure is as shown in Figure 6 .

[0098] The model loss function includes four parts: the smooth loss of the deformable vector field learned by the registration network, the adversarial loss of the GAN network, the L1 loss between the image denoised by RegGAN and the corresponding real image, and the perceptual loss of the image generated by the GAN generator. The proportions of the smooth loss of the deformable vector field learned by the registration network, the adversarial loss of the GAN network, the L1 loss between the image denoised by RegGAN and the corresponding real image, and the perceptual loss of the image generated by the GAN generator are set to 10:5:2:20.

[0099] S4, after the model training is completed, the model weight file saved during the training process is called, and a low-dose CT image in the test set is input, and only the generator part is needed to obtain a high-quality CT image after denoising. The effect comparison before and after denoising is as shown in Figure 7 .

[0100] The low-dose CT image is denoised by using the method, and the experimental results are as follows:

[0101] On the test set, the PSNR (Peak Signal to Noise Ratio) of the model built reaches 26.56, and the SSIM (Structure Similarity Index Measure) reaches 0.714. The results show that this method can better denoise the low-dose CT image.

[0102] In summary, the improved RegGAN low-dose CT image denoising method and related device can dynamically measure the similarity between the denoised image and the noise-free image during the training process, and make improvements to the problems that occur in practice, to a certain extent, solve the problem that the low-dose CT image in the data set and the paired normal-dose CT image are not strictly aligned, causing difficulty in training tasks. The present application provides a better choice for the application of deep learning technology in clinical low-dose CT image denoising, not only solving the problem of difficult acquisition of strictly paired CT image data sets, but also meeting the diagnostic requirements in terms of image quality.

[0103] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or software. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0104] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in a certain embodiment can be referred to the related description of other embodiments.

[0105] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0106] In the embodiments of the present application, it should be understood that the disclosed apparatus / terminal and method can be implemented in other manners. For example, the embodiments of the apparatus / terminal described above are merely schematic, and the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0107] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0108] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically independent unit, or two or more units can be integrated into a unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0109] The integrated module / unit, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include or exclude content according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0110] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0111] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0112] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0113] The above merely provides the technical idea of the present application, and cannot be used to limit the protection scope of the present application. Any modification made according to the technical idea of the present application, on the basis of the technical solutions, falls within the protection scope of the claims of the present application.

Claims

1. An improved RegGAN low-dose CT image denoising method, characterized in that, The method comprises the following steps: Collecting chest and abdomen CT images taken by low-dose technology and non-strictly aligned paired chest and abdomen CT images taken by normal dose; Randomly dividing the paired CT image data set into a training set and a test set; Given the model loss function and the optimizer, and set the appropriate hyperparameters, train the low-dose CT image denoising model; Using the trained model to perform high-quality image denoising and restoration on the low-dose CT image; Add Sobel convolution operator to enhance the edge information of the CT image in the model, and use channel attention mechanism to give different weights to the feature map channels to model the correlation between features; Use a deeper convolutional network to replace the original model discriminator, and add a self-attention mechanism to make the discriminator pay more attention to important information in the image; by spectral normalization constraint discriminator network weight parameters, the training is more stable; add perceptual loss function to make the denoised image retain more texture information.

2. The improved RegGAN low-dose CT image denoising method of claim 1, wherein, The size of the collected single image needs to be the same, and the CT images taken by low-dose technology and normal dose need to be paired one by one, that is, each low-dose CT image has a corresponding normal-dose CT image at the same position.

3. The improved RegGAN low-dose CT image denoising method of claim 1, wherein, The network structure of the generator in the low-dose CT image denoising model is divided into three parts: downsampling stage, feature extraction stage, and upsampling stage; in the downsampling stage of the model generator, a Sobel convolution operator is added, the extracted CT image edge information is fused with the input and sent to the convolutional network, and in the feature extraction stage, a channel attention mechanism is used to give different weights to the channels to model the correlation between features.

4. The improved RegGAN low-dose CT image denoising method of claim 3, wherein, The network structure of the downsampling stage is sequentially a Sobel convolution operator layer, a convolution layer, a normalization layer, an activation function layer, a convolution layer, a normalization layer, an activation function layer, a convolution layer, a normalization layer, and an activation function layer.

5. The improved RegGAN low-dose CT image denoising method of claim 3, wherein, The feature extraction stage is stacked by nine residual modules and an attention module, and the structure of the attention module is sequentially a global average pooling layer, a full connection layer, an activation function layer, a full connection layer, and an activation function layer; the structure of the residual module is sequentially an edge padding layer, a convolution layer, a normalization layer, an activation function layer, an edge padding layer, a convolution layer, and a normalization layer, and the output is obtained by splicing the input.

6. The improved RegGAN low-dose CT image denoising method of claim 1, wherein, In the low-dose CT image denoising model, a deeper convolutional network is used to replace the original discriminator, and a self-attention mechanism is added to make the discriminator pay more attention to important information in the image; the structure of the model discriminator with the self-attention mechanism is sequentially seven spectral normalization convolution modules, a self-attention mechanism layer, a spectral normalization convolution module, a self-attention mechanism layer, and a spectral normalization convolution module, wherein the spectral normalization convolution module is composed of a convolution layer and a LeakyReLU activation function layer, and the weight tensor of the convolution layer is adjusted by spectral normalization.

7. The improved RegGAN low-dose CT image denoising method of claim 1, wherein, The perceptual loss is added to the loss function part, so that the model discriminator pays more attention to the texture information when distinguishing between the denoised CT image generated by the generator and the real noise-free image; after the model training is completed, the model weight file saved during the training process is called, the low-dose CT image in the test set is input, and the denoised high-quality CT image is obtained through the generator.

8. An improved RegGAN low-dose CT image denoising system, characterized in that, It comprises: The acquisition module is configured to acquire a chest and abdomen CT image formed by using a low-dose technique and a pair of chest and abdomen CT images formed by using a normal dose, which are not strictly aligned; The classification module is configured to randomly divide the paired CT image dataset into a training set and a test set; The training module is configured to train a low-dose CT image denoising model given a model loss function and an optimizer, and set appropriate hyperparameters; The denoising module is configured to use the trained model to perform high-quality image denoising and restoration on the low-dose CT image; A Sobel convolution operator is added to enhance the edge information of the CT image in the model, and a channel attention mechanism is used to give different weights to the feature map channels to model the correlation between features. A deeper convolutional network is used instead of the original model discriminator, and a self-attention mechanism is added to make the discriminator pay more attention to important information in the image; spectral normalization is used to constrain the weight parameters of the discriminator network to make the training more stable; a perception loss function is added to make the denoised image retain more texture information.

9. A computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that when executed by a computer cause the computer to perform a method of any of claims 1-8. The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform the method of any one of claims 1-7.

10. A computing device, comprising: Comprise: One or more processors, memories, and one or more programs, wherein the one or more programs are stored in the memories and configured to be executed by the one or more processors, the one or more programs comprising steps for performing the method of any one of claims 1-7.

Images