Denoising Method, System and Storage Medium for Multi-Level CT Sparse Views

Abstract

The present invention provides a denoising method, system and storage medium for multi-energy CT sparse views. A generator and a discriminator with multiple channels are set in a neural network model for denoising multi-energy CT sparse views, and the inputs and outputs of the generator and the discriminator in the neural network model are set as multi-dimensional matrices, so that the neural network model can be applied to denoising multi-energy CT sparse views, saving training time and taking into account the correlation between different energy levels in the multi-energy reconstruction images of the same scanning section, thereby improving the clarity of the output image at one time with better results.

Description

Technical Field

[0001] The present invention relates to the technical field of lossy compressed image restoration, and in particular to a denoising method, system and storage medium for multi-energy CT sparse views. Background Art

[0002] Electron computed tomography (CT) technology refers to using an X-ray beam to perform cross-sectional scanning on a certain thickness layer of an imaging object, and according to the obtained projection data, using a specific mathematical algorithm to reconstruct the cross-sectional image of the object; it is an important imaging means to obtain the internal structure information of the object in a non-destructive manner. Currently, methods for low-dose CT imaging include adjusting the x-ray tube current, sparse data sampling, etc. Among them, directly reducing the tube current will lead to an increase in the noise level; sparse data sampling is divided into uniform angular sparse sampling, limited angular sparse sampling, and hybrid sparse sampling. Compared with adjusting the tube current, sparse sampling reduces the number of projections, so the radiation dose is lower than that of traditional CT, the radiation hazard to the human body is also lower, it has a lower noise level, and at the same time realizes the function of data compression, reducing the requirement for the transmission bandwidth of the slip ring antenna. However, since the projection data obtained by sparse sampling is incomplete projection data and does not meet the Tuy-Smith data completeness condition, the images obtained by using traditional reconstruction techniques and the images reconstructed after interpolation and completion will have serious noise and artifacts, and relevant techniques need to be used in the later stage to repair the image quality.

[0003] At the same time, with the development of multi-energy CT, current mainstream commercial multi-energy CT systems can all provide dual-energy data. Since dual-energy data has higher material identification ability, can effectively suppress or eliminate beam hardening artifacts, and can provide quantitative imaging results, dual-energy or multi-energy CT is gradually becoming the mainstream of clinical imaging. When using dual-energy or multi-energy CT to scan the same position, there is a correlation between the reconstructed images of different energy levels. Therefore, the information missing in the reconstructed image of one energy level can be supplemented by the reconstructed images of other energy levels. However, most of the existing neural networks for CT sparse view denoising are single-energy levels. For example, when using a generative adversarial neural network (GAN architecture) for image processing, the generator in the GAN architecture learns the distribution of the input image under the supervision of the discriminator, so that the distribution of the image generated by the generator is close to the distribution of the real image. However, since the input data of the generator in the prior art does not consider the correlation between energy levels, the restoration ability is limited. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a denoising method, system and storage medium for multi-energy CT sparse views to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present invention provides a denoising method for multi - energy - level CT sparse views, the method comprising the following steps:

[0006] Construct an image dataset for training a pre - built neural network model based on the complete projection data obtained by a multi - energy - level CT scanner; the image dataset includes corresponding multi - dimensional original slices and multi - dimensional noisy slices; use the image dataset to train the pre - built neural network model to obtain a multi - energy - level CT sparse - view denoising model; wherein the pre - built neural network model includes a multi - channel generator and a multi - channel discriminator; perform interpolation and completion on the multi - energy - level sparse projection data obtained by sparse data scanning of the cross - section of the object to be measured by the multi - energy - level CT scanner to obtain noisy complete projection data, and reconstruct the noisy complete projection data to obtain a corresponding noisy multi - energy - level reconstructed image; pass the noisy multi - energy - level reconstructed image through the generator of the multi - energy - level CT sparse - view denoising model to obtain a denoised multi - energy - level reconstructed image. In some embodiments of the present invention, the neural network model adopts an image translation network architecture based on a generative adversarial network, wherein the number of input channels of the generator is the same as the number of output channels, the number of output channels of the discriminator is the same as the number of output channels of the corresponding generator, and the number of input channels of the discriminator is twice the number of output channels.

[0007] In some embodiments of the present invention, the pre - built neural network model adopts a neural network model with an image translation network architecture based on a generative adversarial network, wherein the number of input channels of the generator is the same as the number of output channels, the number of output channels of the discriminator is the same as the number of output channels of the corresponding generator, and the number of input channels of the discriminator is twice the number of output channels.

[0008] In some embodiments of the present invention, in the multi - energy - level sparse projection data obtained by sparse data scanning of the cross - section of the object to be measured by the multi - energy - level CT scanner, each multi - energy - level sparse projection data includes single - energy - level sparse projection data corresponding to multiple different energy levels obtained by sparse data scanning of the same cross - section with the same compression ratio for each energy level in the multi - energy - level CT scanner.

[0009] In some embodiments of the present invention, the step of constructing an image dataset for training a pre-built neural network model based on the complete projection data obtained by a multi-energy CT scanner includes: reconstructing the complete projection data to obtain a corresponding original multi-energy reconstructed image, effectively processing the original multi-energy reconstructed image to remove the invalid scanning area around the scanned object, and obtaining a corresponding multi-dimensional original slice; performing interpolation processing on the complete projection data according to a predetermined compression ratio, reconstructing the noisy complete projection data after interpolation replacement to obtain a corresponding noisy multi-energy reconstructed image, effectively processing the noisy multi-energy reconstructed image to remove the invalid scanning area around the scanned object, and obtaining a corresponding multi-dimensional noisy slice; using the corresponding multi-dimensional original slice and multi-dimensional noisy slice to form an image dataset for training the pre-built neural network model.

[0010] In some embodiments of the present invention, in the step of constructing an image dataset for training a pre-built neural network model, the data volume of the multi-dimensional original slices and multi-dimensional noisy slices in the image dataset is increased by means of data augmentation.

[0011] In one embodiment of the present invention, the step of training the pre-built neural network model with the image dataset to obtain a multi-energy CT sparse-view denoising model includes: cropping the corresponding multi-dimensional original slices and multi-dimensional noisy slices in the image dataset into a plurality of multi-dimensional original sub-slices and multi-dimensional noisy sub-slices of the same size; respectively iteratively training the generator and discriminator in the neural network model with the corresponding multi-dimensional original sub-slices and multi-dimensional noisy sub-slices until the loss functions of the discriminator and the generator both converge, and obtaining a trained multi-energy CT sparse-view denoising model.

[0012] In some embodiments of the present invention, the step of inputting the noisy multi-energy reconstructed image into the generator of the multi-energy CT sparse-view denoising model to obtain a denoised multi-energy reconstructed image includes: effectively processing the noisy multi-energy reconstructed image to remove the invalid scanning area around the scanned object, obtaining a corresponding multi-dimensional noisy slice, and cropping the multi-dimensional noisy slice to obtain a plurality of multi-dimensional noisy sub-slices of the same size; inputting the obtained multi-dimensional noisy sub-slices into the generator of the multi-energy CT sparse-view denoising model, splitting the multi-dimensional denoised sub-slices output by the generator according to energy levels, and splicing the multiple single-energy denoised sub-slices of the same energy level to obtain a complete denoised single-energy reconstructed image at each energy level.

[0013] In some embodiments of the present invention, the multi-dimensional noisy sub-slices input into the generator of the multi-energy CT sparse-view denoising model are of the same size as the multi-dimensional original sub-slices and / or multi-dimensional noisy sub-slices in the image dataset.

[0014] Another aspect of the present invention provides a denoising system for multi - energy - level CT sparse views, including a processor and a memory. Computer instructions are stored in the memory, and the processor is configured to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the above - mentioned denoising method for multi - energy - level CT sparse views.

[0015] Another aspect of the present invention provides a computer - readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the steps of the denoising method for multi - energy - level CT sparse views.

[0016] The present invention discloses a denoising method for multi - energy - level CT sparse views. In a neural network model for denoising multi - energy - level CT sparse views, a generator and a discriminator with multiple channels are set, and the inputs and outputs of the generator and the discriminator in the neural network model are set as multi - dimensional matrices, so that the neural network model can be applied to denoising multi - energy - level CT sparse views. This not only saves training time but also takes into account the correlation between different energy levels in the multi - energy - level reconstructed images of the same scanning section, thereby improving the clarity of the output image at one time with better results. And a Laplacian operator is introduced into the neural network model, so as to ensure better contrast in the low - density regions under the condition of low compression ratio while improving the clarity.

[0017] The additional advantages, objects, and features of the present invention will be partially described below, and will become partially apparent to those of ordinary skill in the art after studying the following text, or can be learned from the practice of the present invention. The objects and other advantages of the present invention can be achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0018] Those skilled in the art will understand that the objects and advantages that can be achieved by the present invention are not limited to the above - mentioned specifically, and the above - mentioned and other objects that the present invention can achieve will be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and do not limit the present invention. In the drawings:

[0020] Figure 1 is a flowchart for denoising multi - energy - level CT sparse views.

[0021] Figure 2 is a diagram of the generator architecture.

[0022] Figure 3 is a diagram of the discriminator architecture.

[0023] Figure 4 This is a flowchart for denoising multi - energy - level CT sparse views in an embodiment of the present invention.

[0024] Figure 5 This is a flowchart for training a neural network model in an embodiment of the present invention.

[0025] Figure 6 These are the original reconstructed images at 80kv, 100kv, and 120kv energy levels in another embodiment of the present invention.

[0026] Figure 7 These are the quadruple - compressed reconstructed images at 80kv, 100kv, and 120kv energy levels in another embodiment of the present invention.

[0027] Figure 8 These are the denoised reconstructed images output by the multi - energy - level CT sparse - view denoising model with low compression ratio - quadruple compression at 80kv, 100kv, and 120kv energy levels in another embodiment of the present invention.

[0028] Figure 9 These are the octuple - compressed reconstructed images at 80kv, 100kv, and 120kv energy levels in another embodiment of the present invention.

[0029] Figure 10 These are the denoised reconstructed images output by the multi - energy - level CT sparse - view denoising model with high compression ratio - octuple compression at 80kv, 100kv, and 120kv energy levels in another embodiment of the present invention.

[0030] Figure 11 These are the local details of the denoised reconstructed images at 80kv, 100kv, and 120kv energy levels output by the multi - energy - level CT sparse - view denoising model with low compression ratio - quadruple compression, with and without the Laplacian operator, in another embodiment of the present invention. Detailed implementation manners

[0031] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in conjunction with the implementation manners and the accompanying drawings. Here, the illustrative implementation manners of the present invention and their descriptions are used to explain the present invention, but do not limit the present invention.

[0032] Here, it should also be noted that in order to avoid obscuring the present invention due to unnecessary details, only the structures and / or processing steps closely related to the solution of the present invention are shown in the drawings, while other details less related to the present invention are omitted.

[0033] It should be emphasized that the term "including / containing" when used herein refers to the presence of features, elements, steps, or components, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0034] Here, it should also be noted that, unless otherwise specified, the term "connection" in this text can not only refer to direct connection, but also represent indirect connection with intermediates.

[0035] In the following, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0036] Aiming at the denoising problem of multi-energy CT sparse views in the prior art, the present invention discloses a denoising method for multi-energy CT sparse views. A generator and a discriminator with multiple channels are set in the neural network model for denoising multi-energy CT sparse views, and the inputs and outputs of the generator and the discriminator in the neural network model are set as multi-dimensional matrices, so that the neural network model can be applied to the denoising of multi-energy CT sparse views, saving training time and taking into account the correlation between different energy levels in the multi-energy reconstruction images of the same scanning section, thereby improving the clarity of the output image at one time with better effect.

[0037] A denoising method for multi-energy CT sparse views provided by the present invention, as Figure 1 shown, includes steps S110 - S140:

[0038] In step S110, according to the complete projection data scanned by a multi-energy CT scanner, an image data set for training a pre-built neural network model is constructed; the image data set includes corresponding multi-dimensional original slices and multi-dimensional noisy slices.

[0039] In the above step S110, the pre-built neural network model adopts a neural network model based on the image translation network architecture of a generative adversarial network, wherein the number of input channels of the generator is the same as the number of output channels, the number of output channels of the discriminator is the same as the number of output channels of the corresponding generator, and the number of input channels of the discriminator is twice the number of output channels.

[0040] The step of constructing an image data set for training a pre-built neural network model according to the complete projection data scanned by a multi-energy CT scanner, as Figure 4As shown in the figure, it includes: reconstructing the complete projection data to obtain the corresponding original multi-level reconstructed image, effectively processing the original multi-level reconstructed image to remove the invalid scanning area around the scanned object, and obtaining the corresponding multi-dimensional original slice; performing interpolation processing on the complete projection data according to a predetermined compression ratio, reconstructing the noisy complete projection data after interpolation replacement to obtain the corresponding noisy multi-level reconstructed image, and effectively processing the noisy multi-level reconstructed image to remove the invalid scanning area around the scanned object, and obtaining the corresponding multi-dimensional noisy slice; using the corresponding multi-dimensional original slice and multi-dimensional noisy slice to form an image dataset for training a pre-built neural network model. Among them, reconstructing the complete projection data and reconstructing the noisy complete projection data after interpolation replacement means converting the detector data obtained by scanning an object with a multi-level CT scanner into image data that we can understand by using CT image reconstruction technology; CT image reconstruction technology is an imaging technology that obtains the cross-sectional information of an object by performing ray projection measurements on the object from different angles; effective processing means, based on the complete multi-level reconstructed image obtained by scanning with a multi-level CT scanner, removing the invalid scanning area around the scanned object and only retaining a part of the multi-level reconstructed image that includes the scanned object, so as to obtain the multi-dimensional original slice corresponding to the effective scanning area that includes the scanned object.

[0041] Specifically, based on the complete projection data of multiple energy levels obtained by the above-mentioned multi-energy-level CT scanner, each complete projection data is directly reconstructed through CT image reconstruction technology to obtain the corresponding original multi-energy-level reconstruction images. In addition, each original multi-energy-level reconstruction image is effectively processed to retain the effective scanning area of each original multi-energy-level reconstruction image and remove the invalid scanning area around the scanned object, resulting in the corresponding multi-dimensional original slices. Each multi-dimensional original slice includes the non-destructive reconstruction slices of each energy level corresponding to a scanning section. By uniformly interpolating each complete projection data according to a specific compression ratio, the corresponding noisy complete projection data is obtained after replacing each complete projection data through equidistant interpolation. Each noisy complete projection data is reconstructed through CT image reconstruction technology to obtain the noisy multi-energy-level reconstruction image after interpolation processing of each complete projection data. In addition, each noisy multi-energy-level reconstruction image is effectively processed to retain the effective scanning area of each noisy multi-energy-level reconstruction image and remove the invalid scanning area around the scanned object, resulting in the corresponding multi-dimensional noisy slices. Each multi-dimensional noisy slice includes the noisy reconstruction slices of each energy level corresponding to a scanning section. According to one or more pairs of corresponding multi-dimensional original slices and multi-dimensional noisy slices, an image dataset for training the neural network model is created. The image dataset includes multiple pairs of multi-dimensional original slices and multi-dimensional noisy slices that are the same in size and corresponding to each other. The non-destructive reconstruction slices corresponding to each energy level in the multi-dimensional original slices are stacked in ascending order of energy level. The noisy reconstruction slices corresponding to each energy level in the multi-dimensional noisy slices are stacked in ascending order of energy level. The step of uniformly interpolating each complete projection data according to a specific compression ratio and obtaining the corresponding noisy complete projection data after replacing each complete projection data through equidistant interpolation includes: compressing the complete projection data according to a specific compression ratio and then complementing the compressed projection data to obtain the noisy complete projection data. For example, when the complete projection data (1, 2, 4, 6, 8, 8, 3, 3) is uniformly interpolated according to a compression ratio of 2 times, in the obtained corresponding noisy complete projection data, the conversion process of the complete projection data is as follows: the compressed projection data obtained after compressing the complete projection data (1, 2, 4, 6, 8, 8, 3, 3) according to a compression ratio of 2 times is (1, 4, 8, 3), and the noisy complete projection data obtained after complementing the compressed projection data is (1, 2.5, 4, 6, 8, 5.5, 3, 2).

[0042] Divide the corresponding multi-dimensional original slices and multi-dimensional noisy slices in the above manner according to a certain allocation ratio into a training set and a test set in the image dataset; when the number of corresponding multi-dimensional original slices and multi-dimensional noisy slices used to train the pre-built neural network model is small, although the neural network model for multi-energy-level CT sparse-view denoising can achieve good training results with less training data, increasing the training data can effectively improve the generalization ability and robustness of the neural network model; therefore, in the step of constructing the image dataset for training the pre-built neural network model, the data volume of the corresponding multi-dimensional original slices and multi-dimensional noisy slices in the image dataset can be increased by means of data augmentation. Specifically, various data augmentation methods such as 180° rotation, horizontal mirroring, and / or vertical mirroring are used to process the corresponding multi-dimensional original slices and multi-dimensional noisy slices in the image dataset, so as to increase the training data volume of the pre-built neural network model, and further improve the generalization ability and robustness of the neural network model.

[0043] In addition, synchronous cropping can be performed on each pair of corresponding multi-dimensional original slices and multi-dimensional noisy slices in the image dataset, and each pair of corresponding multi-dimensional original slices and multi-dimensional noisy slices is divided into the same number of corresponding multi-dimensional original sub-slices and multi-dimensional noisy sub-slices of the same size; and in order to improve the robustness of the trained neural network model, when synchronously cropping the multi-dimensional original slices and multi-dimensional noisy slices, it can be ensured that there is a certain overlapping area between the adjacent sub-slices after cropping. Among them, each multi-dimensional original sub-slice includes lossless reconstruction sub-slices of multiple energy levels stacked in ascending order of energy level at the same cropping position, and each multi-dimensional noisy sub-slice includes noisy reconstruction sub-slices of multiple energy levels stacked in ascending order of energy level at the same cropping position.

[0044] In one embodiment, the process of constructing an image dataset for training a pre-built neural network model is as follows: A multi-energy CT scanner scans cross-sections of various objects to obtain multiple complete projection data with multiple energy levels, whose size is (H, W, C), where H represents the number of channels in each row of the multi-energy CT scanner, W represents the total number of frames obtained by scanning the cross-section of the object to be measured in one scan of the multi-energy CT scanner, and C represents the number of energy levels of the rays used to scan the projection data. In the specific implementation process, the data size of the projection data of each energy level in each complete projection data is set to (880, 2320, 1); the size of the reconstructed image of each energy level in the original multi-energy reconstructed image obtained by reconstructing each complete projection data is 512×512. After removing the invalid scan area around the scanned object from the original multi-energy reconstructed image, the size of the multi-dimensional original slice containing only the valid scan area is (256, 384, n), where n represents the number of dimensions of the multi-dimensional original slice and corresponds to the number of energy levels in the corresponding complete projection data; during the interpolation and replacement process of each complete projection data, the size of the compressed projection data obtained by removing according to the r-fold compression ratio is (H, W / r, C), and then the size of the noisy complete projection data obtained by complementing the compressed projection data is (H, W, C). The size of the reconstructed image of each energy level in the noisy multi-energy reconstructed image obtained by reconstructing each noisy complete projection data is 512×512; after removing the invalid scan area around the scanned object from the noisy multi-energy reconstructed image, the size of the multi-dimensional noisy slice containing only the valid scan area is (256, 384, n). In the corresponding embodiment, according to the ratio of 10:1, the corresponding multi-dimensional original slices and multi-dimensional noisy slices are divided into the training set and the test set of the pre-built neural network model; and the corresponding multi-dimensional original slices and multi-dimensional noisy slices are synchronously cropped into multiple pairs of corresponding multi-dimensional original sub-slices and multi-dimensional noisy sub-slices with a size of (64, 64, n), and the overlapping area between adjacent sub-slices in the multi-dimensional original slices or multi-dimensional noisy slices is set to 48. Then, the corresponding multi-dimensional original slices and multi-dimensional noisy slices are cropped every 16 pixel points in the W / H direction. Finally, each pair of corresponding multi-dimensional original slices and multi-dimensional noisy slices is cropped and divided into 273 pairs of corresponding multi-dimensional original sub-slices and multi-dimensional noisy sub-slices with a size of (64, 64, n).

[0045] In another embodiment, the original reconstructed image of each energy level obtained by performing a complete projection scan and reconstruction on the lungs of the subject using a multi-energy CT scanner including 80 kv, 100 kv, and 120 kv is as Figure 6As shown, the noisy complete projection data obtained by interpolating and replacing the multi-energy complete projection data obtained by performing a complete projection scan on the above multi-energy CT scanner at a four-fold compression ratio is reconstructed to obtain the noisy reconstructed images at each energy level as Figure 7 shown; the noisy complete projection data obtained by interpolating and replacing the multi-energy complete projection data obtained by performing a complete projection scan on the above multi-energy CT scanner at an eight-fold compression ratio is reconstructed to obtain the noisy reconstructed images at each energy level as Figure 9 shown; for the original multi-energy reconstructed images corresponding to Figure 6 and Figure 7 shown or Figure 6 and Figure 9 shown, the original multi-energy reconstructed images and the noisy multi-energy reconstructed images of other multiple scan sections similar to the corresponding original multi-energy reconstructed images and noisy multi-energy reconstructed images are effectively processed together to obtain an image dataset for training the neural network model of the pix2pix GAN architecture. The image dataset of the neural network model corresponding to each target sparse view includes the corresponding original multi-energy reconstructed images for multiple scan sections and the noisy multi-energy reconstructed images corresponding to the target sparse view; the corresponding original multi-energy reconstructed images and the noisy multi-energy reconstructed images for multiple scan sections in the image dataset of the neural network model corresponding to each sparse view are divided into a training set and a test set according to a ratio of 10∶1.

[0046] In step S120, the pre-built neural network model is trained using the image dataset to obtain a multi-energy CT sparse view denoising model.

[0047] Specifically, a neural network model using an image translation network architecture based on a generative adversarial network (pix2pix GAN architecture) is used as the pre-built neural network model. This pre-built neural network model includes a multi-channel generator and a multi-channel discriminator. The number of input channels of the generator is the same as the number of output channels. The number of output channels of the discriminator is the same as the number of output channels of the corresponding generator, and the number of input channels of the discriminator is twice the number of output channels. Among them, the generator is used to generate corresponding restored data according to the input noisy data, so that the restored data is infinitely close to the corresponding original data, and the discriminator is used to distinguish other sample data that does not belong to the original data from the true and false mixed samples.

[0048] In an embodiment of the present invention, the generator in the neural network model of the pix2pix GAN architecture is a segmentation network architecture with residual connections (Unet architecture), as Figure 2 shown. The number of input channels and the number of output channels of this generator are the same; the discriminator is a neural network architecture including 5 convolutional layers, as Figure 3 shown.Figure 3 Here, "k3n64s2" indicates that the convolution kernel size is 3x3, the number of convolution kernels is 64, and the convolution step is 2; the number of output channels of the discriminator is the same as that of the generator, and the number of input channels of the discriminator is twice the number of output channels. In addition, each channel in the generator corresponds to an energy level when a multi-energy CT scanner is used for scanning; the reconstructed image corresponding to each energy level in the multi-energy reconstructed image obtained by scanning the same object cross-section by the multi-energy CT scanner is input into the generator and / or the discriminator from a separate input channel. In the embodiment, the neural network model of the pix2pix GAN architecture adopts a leaky linear rectifier function (Leaky Relu function) as the activation function of the generator in the neural network model, and sets the input end of the generator to a 3x3 convolution with an input channel of n, and the output end to a 1x1 convolution with an output channel of n. The corresponding mathematical expression of the activation function used in the generator is: y=max(0,x)+0.01*min(0,x); the Leaky Relu function is used as the activation function of the discriminator, and the number of input channels of the discriminator is set to 2n, and the number of output channels is set to n. The corresponding mathematical expression of the activation function used in the discriminator is: y=max(0,x)+0.01*min(0,x), wherein x represents the input of the activation function, and y represents the output of the activation function.

[0049] The neural network model of the pix2pix GAN architecture is trained using the training data in the image data set formed above. During the training process of the neural network model of the pix2pix GAN architecture, the noisy multi-level reconstructed image passes through the generator in the neural network model of the pix2pix GAN architecture, and the output multi-level denoised image is infinitely close to the original multi-level reconstructed image corresponding to the noisy multi-level reconstructed image. The discriminator in the neural network model of the pix2pix GAN architecture filters out images that do not belong to the original multi-level reconstructed image between the multi-level denoised image and the original multi-level reconstructed image.

[0050] Among them, the steps of training the pre-built neural network model using the training data in the image data set to obtain the multi-level CT sparse view denoising model are as follows: Figure 4 As shown, it includes: cutting the corresponding multidimensional original slices and multidimensional noisy slices in the image data set into multiple multidimensional original sub-slices and multidimensional noisy sub-slices of the same size and corresponding to each other; using the corresponding multidimensional original sub-slices and multidimensional noisy sub-slices to iteratively train the generator and the discriminator in the neural network model respectively until the loss functions of the discriminator and the generator converge, thereby obtaining a trained multi-level CT sparse view denoising model.

[0051] Specifically, the steps of iteratively training the generator and discriminator in the neural network model by using the corresponding multi-dimensional original sub-slices and multi-dimensional noisy sub-slices in the training set are as follows Figure 5 shown in the figure. When training the discriminator, keeping the parameters of the generator unchanged, concatenating the corresponding multi-dimensional original sub-slices and multi-dimensional noisy sub-slices according to energy levels as the input data of the discriminator, calculating the first term of the loss function of the discriminator, that is, L 1 (D)=-E x，y [logD(x,y)], and then performing the first backpropagation to calculate the gradient value of the current discriminator parameter θ Dis and retaining the currently generated gradient value; then using the multi-dimensional noisy sub-slices as the input data of the generator, and outputting the corresponding multi-dimensional denoised sub-slices through the generator; after concatenating the multi-dimensional noisy sub-slices and the corresponding multi-dimensional denoised sub-slices output by the generator according to energy levels and inputting them into the discriminator, calculating the second term of the discriminator loss function, that is, L (D)=-E 2 (D)=-E x，z [log(1 - D(x,G(x)))] and then performing the second backpropagation to calculate the gradient value of the current discriminator parameter θ Dis and retaining the currently generated gradient value; updating the discriminator parameters by combining the gradient value of the discriminator with the learning rate where α is the learning rate of the discriminator, Dis and the loss function of the discriminator is: The loss function of the discriminator is:

[0052]

[0053] where x is the multi-dimensional noisy sub-slice; y is the multi-dimensional original sub-slice; z is the multi-dimensional denoised sub-slice output by the generator, with the multi-dimensional noisy sub-slice x as the condition for the discriminator input; D is the discriminator, -E x，y [logD(x,y)] means that when the discriminator input is the multi-dimensional noisy sub-slice and the multi-dimensional original sub-slice, calculating the binary cross-entropy element by element between the output matrix of the discriminator and the identity matrix (all elements are 1) with the same dimension as the output matrix, and then summing and taking the average; -E x，z [log(1 - D(x,G(x)))] means that when the discriminator input is the multi-dimensional noisy sub-slice and the output of the generator, calculating the binary cross-entropy element by element between the output matrix of the discriminator and the zero matrix (all elements are 0) with the same dimension as the output matrix, and then summing and taking the average.

[0054] When training the generator, keep the discriminator parameters unchanged. Use the multi-dimensional noisy slices as the input data for the generator. After passing through the generator, the corresponding multi-dimensional denoised slices are output. Concatenate the multi-dimensional noisy slices input to the generator and the corresponding multi-dimensional denoised slices output by the generator according to energy levels, and input them into the discriminator through the input channels of the discriminator. Calculate the loss function of the generator, and then perform backpropagation to calculate the gradient value of the current generator parameter θ Gen of the gradient value Use the gradient value of the generator combined with the learning rate to update the generator parameter α Gen is the learning rate of the generator. Among them, the Laplace operator is introduced in the generator to perform spatial sharpening and edge detection on the input image. Write the second-order differential operator of the Laplace operator in matrix form as the convolution kernel of a 3x3 convolution. Perform convolution (stride = 1) on the multi-dimensional denoised slices and multi-dimensional original slices output by the generator with this convolution kernel respectively. Then, through the loss function, narrow the gap between the two, make their second-order differentials as close as possible, and thus achieve the purpose of increasing the contrast of the low-density region and suppressing the over-smoothing caused by the loss term L L2 (G). The loss function of this generator is:

[0055]

[0056] L L2 (G)=E[||y - G(x)|| 2

[0057] L LAP (G)=E[||Laplace(y)-Laplace(G(x))|| 2

[0058] Among them, G is the generator, λ 1 and λ 2 are the weight coefficients of the two losses of L L2 (G) and L LAP (G) respectively; L L2 (G) represents the mean square error between the multi-dimensional original slices and the multi-dimensional denoised slices output by the generator, that is, E[||y - G(x)|| 2 ; L LAP (G) is the loss function of the Laplace algorithm, which represents the mean square error between the result obtained by performing convolution (stride = 1) on the multi-dimensional original slices and the Laplace operator and the result obtained by performing convolution (stride = 1) on the multi-dimensional denoised slices output by the generator and the Laplace operator, that is, E[||Laplace(y)-Laplace(G(x))|| 2 ​​; Laplace(y) represents the result obtained by convolving the multi-dimensional original sub-slice with the Laplace operator (stride = 1), and Laplace(G(x)) represents the result obtained by convolving the multi-dimensional denoised sub-slice output by the generator with the Laplace operator; the dimension of the Laplace operator is (3, 3, n), where n is the number of energy levels. The matrix [[0, -1, 0], [-1, 5, -1], [0, -1, 0]] with dimension (3, 3) can be extended to (3, 3, 1) and then replicated n times in the third dimension.

[0059] Until the loss functions of both the discriminator and the generator reach convergence, stop training the neural network model of the pix2pix GAN architecture to obtain the trained multi-energy-level CT sparse-view denoising model of the pix2pix GAN architecture.

[0060] Corresponding to the above embodiments, the pre-built neural network model is trained using the corresponding multi-dimensional original slices and multi-dimensional noisy slices in the training set of the image dataset; in the embodiments, the pix2pix GAN network architecture is used as the network architecture of the pre-built neural network model, the initial learning rate of the generator of the pix2pix GAN network architecture is set to 0.0001, the initial learning rate of the discriminator is 0.00001, the Adam optimization algorithm is used, the first-order moment estimation exponential decay rate beta1 = 0.9, the second-order moment estimation exponential decay rate beta2 = 0.999, and batch size = 128. During the training of the neural network model with the pix2pix GAN architecture, the corresponding multi-dimensional original sub-slices and multi-dimensional noisy sub-slices are concatenated according to the channels to obtain a multi-dimensional matrix of size (64, 64, 2n). According to the one-to-one correspondence between the dimension and the input channels of the discriminator, the multi-dimensional matrix of size (64, 64, 2n) is input into the discriminator, and the first term of the loss function of the discriminator is calculated through the output result of the discriminator, and then the first backpropagation is performed, and the gradient value of the current discriminator is retained. The output data sizes of the multi-dimensional matrix of size (64, 64, 2n) in each convolutional layer of the discriminator are (32, 32, 64), (16, 16, 128), (8, 8, 256), (8, 8, 512), (8, 8, n) in turn; the multi-dimensional matrix of size (64, 64, n) composed of multi-dimensional noisy sub-slices is input into the generator, and the generator is made to output the corresponding multi-dimensional sub-denoised slices of the multi-dimensional noisy sub-slices. Among them, the image data sizes output by each convolutional layer during downsampling of the multi-dimensional noisy sub-slices in the generator are (32, 32, 128), (16, 16, 256), (8, 8, 512), (4, 4, 1024) in turn; the image data sizes output by each convolutional layer during upsampling are (8, 8, 512), (16, 16, 256), (32, 32, 128), (64, 64, 64) in turn; finally, after 1x1 convolution and residual operations, an output matrix with the same dimension as the input is obtained, with a size of (64, 64, n); the multi-dimensional noisy sub-slices input into the generator and the corresponding multi-dimensional denoised slices output by the generator are concatenated according to the channels to obtain a multi-dimensional tensor of size (64, 64, 2n), and the multi-dimensional tensor of size (64, 64, 2n) is input into the discriminator, and the second term of the loss function of the current discriminator is calculated according to the discrimination result of the discriminator, and then the second backpropagation is performed, thereby updating the discriminator parameters.

[0061] Input the multi-dimensional noisy slice into the generator, and let the generator output the corresponding multi-dimensional sub-denoised slice of the multi-dimensional noisy slice; splice the multi-dimensional noisy slice input into the generator and the corresponding multi-dimensional denoised slice output by the generator according to the channels to obtain a multi-dimensional tensor of size (64, 64, 2n), input the multi-dimensional tensor of size (64, 64, 2n) into the discriminator, calculate the loss function of the current generator according to the discrimination result of the discriminator, and then perform backpropagation to update the generator parameters.

[0062] Through the above iterative training of the generator and discriminator in the neural network model of the pix2pix GAN architecture, when training the discriminator, the generator parameters remain fixed, and the discriminator parameters are updated after two consecutive backpropagations; when training the generator, the discriminator parameters remain fixed, and the generator parameters are updated after each backpropagation; stop training until the loss functions of both converge, and obtain the final multi-level CT sparse view denoising model of the pix2pix GAN architecture after training.

[0063] Corresponding to another embodiment of the present invention, using Figure 6 and Figure 7 shown or Figure 6 and Figure 9 The original multi-level reconstruction images and noisy multi-level reconstruction images corresponding to a variety of mutually corresponding scanning cross-sections similar to those shown are used together with the original multi-level reconstruction images and noisy multi-level reconstruction image pairs corresponding to multiple scanning cross-sections in the training set of the image dataset to train the neural network model of the pre-built pix2pix GAN architecture respectively, and obtain a low compression rate - four-fold compression multi-level CT sparse view denoising model and a high compression rate - eight-fold compression multi-level CT sparse view denoising model. Take Figure 7 The shown noisy multi-level reconstruction image as the noisy multi-level reconstruction image in the test set. After denoising by the trained low compression rate - four-fold compression multi-level CT sparse view denoising model, obtain the denoised reconstruction images of each energy level corresponding to the scanning cross-section as shown in Figure 8 ; Take Figure 9 The shown noisy multi-level reconstruction image as the noisy multi-level reconstruction image in the test set. After denoising by the trained high compression rate - eight-fold compression multi-level CT sparse view denoising model, obtain the denoised reconstruction images of each energy level corresponding to the scanning cross-section as shown in Figure 10 .

[0064] Take Figure 7 The shown noisy multi-level reconstruction image as the noisy multi-level reconstruction image in the test set. After denoising by the trained low compression rate - four-fold compression multi-level CT sparse view denoising model, obtainFigure 11 The local details of the denoised slices at each energy level output by the multi-level CT sparse-view denoising model with low compression ratio - quadruple compression, which introduces the Laplacian operator in the loss function shown, and the local details of the denoised slices at each energy level output by the multi-level CT sparse-view denoising model with low compression ratio - quadruple compression, which does not introduce the Laplacian operator in the loss function, are compared Figure 11 From the images shown, it can be seen that under the condition of low compression ratio, introducing the Laplacian operator in the loss function of the pre-built neural network model can increase the contrast of the low-density region of the restored image, which helps to suppress the over-smoothing caused by the loss term L L2 (G).

[0065] In step S130, the multi-level sparse projection data obtained by sparsely scanning the cross-section of the object to be measured by the multi-level CT scanner is interpolated and completed to obtain noisy complete projection data, and the noisy complete projection data is reconstructed to obtain the corresponding noisy multi-level reconstructed image.

[0066] Specifically, in the multi-level sparse projection data obtained by sparsely scanning the cross-section of the object to be measured by the multi-level CT scanner, each multi-level sparse projection data includes single-level sparse projection data corresponding to multiple different energy levels obtained by sparsely scanning the same cross-section with the same compression ratio at each energy level in the multi-level CT scanner. And the bandwidth occupied by this multi-level sparse projection data is the same as that of the compressed projection data obtained after being compressed according to a specific compression ratio during the training process of the multi-level CT sparse-view denoising model; the bandwidth occupied by the noisy complete projection data obtained after interpolating and completing this multi-level sparse projection data is the same as that of the complete projection data obtained by the multi-level CT scanner through full scanning.

[0067] In step S140, the noisy multi-level reconstructed image is passed through the generator of the multi-level CT sparse-view denoising model to obtain a denoised multi-level reconstructed image.

[0068] The step of passing the noisy multi-level reconstructed image through the generator of the multi-level CT sparse-view denoising model to obtain a complete multi-level reconstructed image is as Figure 4As shown, it includes: effectively processing the multi-level reconstructed image with noise to remove the invalid scanning area around the scanned object, obtaining the corresponding multi-dimensional noisy slice, and cropping the multi-dimensional noisy slice to obtain multiple multi-dimensional noisy sub-slices of the same size; inputting the obtained multi-dimensional noisy sub-slices into the generator of the multi-level CT sparse-view denoising model, and splitting the multi-dimensional denoised sub-slices output by the generator according to energy levels, and splicing multiple single-level denoised sub-slices of the same energy level to obtain a complete denoised single-level reconstructed image at each energy level. Specifically, each multi-dimensional denoised sub-slice corresponds to multiple single-level denoised sub-slices of different energy levels. After splicing the single-level denoised sub-slices at the same energy level of the same scanning section along the H and W directions according to the cropping positions of the corresponding multi-dimensional noisy sub-slices, a complete denoised single-level reconstructed image at each energy level of this scanning section is obtained. Among them, the size of the multi-dimensional noisy sub-slices input into the generator of the multi-level CT sparse-view denoising model is the same as that of the multi-dimensional original sub-slices and / or multi-dimensional noisy sub-slices in the image dataset.

[0069] Correspondingly to the above method, the present invention also provides a denoising system for multi-level CT sparse views. The system includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the denoising method for multi-level CT sparse views as described above.

[0070] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the foregoing denoising method for multi-level CT sparse views. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0071] Those of ordinary skill in the art should understand that the various exemplary components, systems, and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Specifically, whether to implement in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave on a transmission medium or a communication link.

[0072] It should be clear that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present invention.

[0073] In the present invention, the features described and / or exemplified for one embodiment can be used in the same or a similar manner in one or more other embodiments, and / or combined with the features of other embodiments or replace the features of other embodiments.

[0074] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, various changes and modifications can be made to the embodiments of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A denoising method for multi - energy - level CT sparse views, characterized in that, the method comprises the following steps: According to the complete projection data obtained by scanning with a multi - energy - level CT scanner, construct an image data set for training a pre - built neural network model; the image data set includes corresponding multi - dimensional original slices and multi - dimensional noisy slices; Use the image data set to train the pre - built neural network model to obtain a multi - energy - level CT sparse view denoising model; wherein the pre - built neural network model includes a multi - channel generator and a multi - channel discriminator; Interpolate and complete the multi - energy - level sparse projection data obtained by sparse data scanning of the cross - section of the object to be measured by the multi - energy - level CT scanner to obtain noisy complete projection data, and reconstruct the noisy complete projection data to obtain the corresponding noisy multi - energy - level reconstructed image; Pass the noisy multi - energy - level reconstructed image through the generator of the multi - energy - level CT sparse view denoising model to obtain a denoised multi - energy - level reconstructed image; Among them, the neural network model adopts an image translation network architecture based on an adversarial neural network. Among them, the generator adopts a segmentation network architecture with residual connections, the discriminator adopts a neural network structure including multiple convolutional layers, the number of input channels of the generator is the same as the number of output channels, the number of output channels of the discriminator is the same as the number of output channels of the corresponding generator, and the number of input channels of the discriminator is twice the number of output channels; Among the multi - energy - level sparse projection data obtained by sparse data scanning of the cross - section of the object to be measured by the multi - energy - level CT scanner, each multi - energy - level sparse projection data includes single - energy - level sparse projection data corresponding to multiple different energy levels obtained by sparse data scanning of the same cross - section with the same compression ratio for each energy level in the multi - energy - level CT scanner.

2. The method according to claim 1, characterized in that, the step of constructing an image data set for training a pre - built neural network model according to the complete projection data obtained by scanning with a multi - energy - level CT scanner includes: Reconstruct the complete projection data to obtain the corresponding original multi - energy - level reconstructed image, and perform effective processing on the original multi - energy - level reconstructed image to remove the invalid scanning area around the scanning object to obtain the corresponding multi - dimensional original slices; Perform interpolation processing on the complete projection data according to a predetermined compression ratio, and reconstruct the interpolated and replaced noisy complete projection data to obtain the corresponding noisy multi - energy - level reconstructed image, and perform effective processing on the noisy multi - energy - level reconstructed image to remove the invalid scanning area around the scanning object to obtain the corresponding multi - dimensional noisy slices; Use the corresponding multi - dimensional original slices and multi - dimensional noisy slices to form an image data set for training a pre - built neural network model.

3. The method according to claim 1, characterized in that, In the step of constructing an image data set for training a pre - built neural network model, the data volume of the multi - dimensional original slices and multi - dimensional noisy slices in the image data set is increased by means of data augmentation.

4. The method according to claim 1, characterized in that, The step of training the pre-established neural network model using the image dataset to obtain a multi-level CT sparse view denoising model includes: cropping the corresponding multi-dimensional original slices and multi-dimensional noisy slices in the image dataset into multiple multi-dimensional original sub-slices and multi-dimensional noisy sub-slices with the same size; using the corresponding multi-dimensional original sub-slices and multi-dimensional noisy sub-slices to iteratively train the generator and discriminator in the neural network model until the loss functions of the discriminator and the generator both converge, obtaining the trained multi-level CT sparse view denoising model.

5. The method according to claim 1, wherein, the step of obtaining the denoised multi-level reconstruction image by passing the noisy multi-level reconstruction image through the generator of the multi-level CT sparse view denoising model includes: effectively processing the noisy multi-level reconstruction image to remove the invalid scanning area around the scanned object, obtaining the corresponding multi-dimensional noisy slice, and cropping the multi-dimensional noisy slice to obtain multiple multi-dimensional noisy sub-slices of the same size; inputting the obtained multi-dimensional noisy sub-slices into the generator of the multi-level CT sparse view denoising model, splitting the multi-dimensional denoised sub-slices output by the generator according to energy levels, and splicing multiple single-level denoised sub-slices of the same energy level to obtain a complete denoised single-level reconstruction image at each energy level.

6. The method according to claim 5, wherein, the multi-dimensional noisy sub-slices input into the generator of the multi-level CT sparse view denoising model have the same size as the multi-dimensional original sub-slices and / or multi-dimensional noisy sub-slices in the image dataset.

7. A denoising system for multi-level CT sparse views, comprising a processor and a memory, wherein, computer instructions are stored in the memory, and the processor is configured to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, on which a computer program is stored, wherein, when the program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images