An unsupervised LDCT image restoration method based on neighborhood subgraph

Abstract

The application discloses a kind of unsupervised LDCT image restoration methods based on neighborhood subgraph, comprising the following steps: S1, obtains LDCT image data;S2, training LDCT image restoration network, the LDCT image restoration network includes 3 level subgraph restoration network, LDCT image data Y is generated neighborhood subgraph by sub-sampler G reconstruction loss L rec , LDCT image data Y is generated constraint graph by third level subgraph restoration network, constraint graph is generated constraint subgraph by sub-sampler G regular loss L reg , using the reconstruction loss L rec , regular loss L reg Loss function of LDCT image restoration network training is built.S3, the LDCT image to be restored is input to the 3rd level subgraph restoration network of the LDCT image restoration network of training completion, and the restored LDCT image is output.The application can train LDCT image restoration network in the absence of paired LDCT image-NDCT image data, and complete LDCT image restoration by trained LDCT image restoration network.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to an unsupervised LDCT image restoration method based on neighborhood subgraphs. Background Technology

[0002] Computed tomography (CT) is a crucial technology in medical imaging diagnosis, generating high-resolution tomographic images by projecting large amounts of X-rays and mathematically reconstructing the interior of the human body. Low-dose computed tomography (LDCT) has become a common practice in medical imaging diagnosis. LDCT significantly reduces the radiation dose received by patients, lowering potential radiation risks. However, LDCT images are often affected by noise, artifacts, and other factors, reducing image quality and thus impacting the accuracy and reliability of diagnosis. Therefore, improving the quality of LDCT images has become a significant challenge in the field of medical imaging technology.

[0003] Deep learning methods offer significant advantages in feature extraction and function fitting. Supervised learning methods use labeled, high-quality data to train models, thereby learning the mapping relationships between images. In LDCT image processing, this labeled data is typically normal dose CT images (NDCT). In LDCT image reconstruction, obtaining large-scale paired LDCT-NDCT image data is extremely difficult due to the need to consider patient radiation dose safety.

[0004] In recent years, unsupervised learning techniques have received widespread attention and application in medical image processing. Unsupervised learning methods can learn features and patterns from data using only unlabeled data. This invention aims to use an unsupervised learning method to train an LDCT image restoration network without using paired LDCT image-NDCT image data, and to complete LDCT image restoration using the trained LDCT image restoration network. Summary of the Invention

[0005] The purpose of this invention is to overcome the limitations of existing deep learning-based LDCT image restoration methods in terms of training data, and to propose an unsupervised LDCT image restoration method based on neighborhood subgraphs. This method, even in the absence of paired LDCT and NDCT images, constructs neighborhood subgraphs and constraint subgraphs to train an LDCT image restoration network, achieving noise removal and structural enhancement of LDCT images, thereby effectively restoring LDCT images.

[0006] To achieve the above objectives, an unsupervised LDCT image restoration method based on neighborhood subgraphs is proposed, comprising the following steps:

[0007] S1: Obtain LDCT image data Y(i,j) to construct a training dataset; where i and j are the x and y coordinates of the pixels in the LDCT image Y, respectively, and the spatial dimension of the LDCT image Y is N*N;

[0008] S2: Train the LDCT image restoration network, which includes a 3-level sub-image restoration network, denoted as f. θk k = 1, 2, 3, where θ is the network parameter and k is the sub-image restoration network level. The LDCT image data Y is processed by a subsampler G to generate four neighborhood sub-images g0(Y), g1(Y), g2(Y), and g3(Y). The reconstruction loss L is constructed using these neighborhood sub-images. rec The LDCT images corresponding to the neighborhood sub-images are processed by the third-level sub-image restoration network f. θ3 Obtain constraint graph f θ3 (Y), the constraint graph generates three constraint subgraphs g1(f) through the subsampler G. θ3 (Y)),g2(f θ3 (Y)),g3(f θ3 (Y)) constructs a regularized loss L through a constraint subgraph. reg During the generation of constrained subgraphs, the process of LDCT images being restored through the third-level subgraph reconstruction network is not included in the gradient calculation during network training. The reconstruction loss L is then used. rec Regular loss L reg The loss function L for constructing a subgraph restoration network k =L reg +λL reg , where λ is a hyperparameter;

[0009] S3: Input the LDCT image to be restored into the third-level sub-image restoration network of the trained LDCT image restoration network, and output the restored LDCT image.

[0010] Preferably, the LDCT image restoration network in S2 includes a 3-level sub-image restoration network. Each level of the sub-image restoration network is trained using the loss function of the sub-image restoration network. Specifically, the output of the next level of the second and third level sub-image processing networks is fused with the input of the current level at the pixel level and used as the input of the current sub-image restoration network.

[0011] Preferably, the sub-image restoration network in S2 includes two convolutional modules and four CFT-Transformer modules. Each CFT-Transformer module is sequentially connected to each other. The two convolutional modules are respectively connected at the beginning and end of the interconnected CFT-Transformer modules. The first convolutional module is used for shallow feature extraction, and the second convolutional module is used for image decoding. The CFT-Transformer module includes two Layer Normalization (LN) layers, one Window Multi-head Self-attention (W-MSA) layer, and one Channel Fusion (CFT-FFN) module, arranged in the order of LN, W-MSA, LN, and CFT-FFN. The input of the first LN is added to the output of W-MSA through a skip connection, and the input of the second LN is added to the output of CFT-FFN through a skip connection.

[0012] Preferably, the CFT-FFN module includes two branches. One branch includes a 3*3 convolutional layer and a sigmoid activation layer. The input of the convolutional layer is multiplied by the output of the sigmoid activation layer through a skip connection. The other branch includes only a 3*3 convolutional layer. The input and output of the convolutional layer are added by a skip connection. Finally, the two branches are added together to obtain the forward output.

[0013] Preferably, in S2, the subsampler G selects pixels from the LDCT image data Y in a non-overlapping 2*2 size subsampler window to generate four neighborhood sub-images g0(Y), g1(Y), g2(Y), and g3(Y). In each neighborhood sub-image, all pixels come from the same position in different 2*2 size subsampler windows. Given an LDCT image Y(i,j), the four neighborhood sub-images g0(Y), g1(Y), g2(Y), and g3(Y) are:

[0014]

[0015] Preferably, in S2, the constraint subgraph, given an LDCT image Y, is processed by the third-level subgraph processing network to obtain the constraint graph f. θ3 (Y), constraint graph f θ3 (Y) After passing through the subsampler G, three constraint subgraphs g1(f) are obtained. θ3 (Y)),g2(f θ3 (Y)),g3(f θ3 (Y) are respectively:

[0016]

[0017] Preferably, the training of the LDCT image restoration network in S2 uses the neighborhood sub-image g0(Y) as the restoration target, and g1(Y), g2(Y), and g3(Y) as the sub-image restoration networks at each level. θk The input, g1(f θ3 (Y)),g2(f θ3 (Y)),g3(f θ3 (Y) is used as a constraint subgraph and input into the subgraph restoration networks at each level for training. The subgraph restoration network f θk Reconstruction losses Regular loss is The loss function of the subgraph restoration network is:

[0018]

[0019] Each level of the subgraph network includes one forward propagation and one backward propagation. One forward propagation constitutes one loss constraint, and one backward propagation updates the parameters of the subgraph restoration network. The LDCT image restoration network L... cont The loss function is:

[0020]

[0021] Compared with existing technologies, the significant advantages of this invention are as follows: This invention discloses an unsupervised LDCT image restoration method based on neighborhood subgraphs. For unpaired LDCT image data, a subsampler is used to construct neighborhood subgraphs and constraint subgraphs for the LDCT image, and the LDCT image restoration network is trained using these neighborhood and constraint subgraphs. Compared with existing deep learning-based LDCT image restoration techniques, the neighborhood and constraint subgraphs constructed in this invention can replace paired LDCT-NDCT images, overcoming the limitation of deep learning-based LDCT image restoration methods that cannot collect corresponding NDCT images for LDCT images. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method of the present invention;

[0023] Figure 2 This is a schematic diagram of the LDCT image restoration network according to a preferred embodiment of the present invention;

[0024] Figure 3 yes Figure 2 A schematic diagram of the network reconstruction at each level of the subgraph;

[0025] Figure 4 yes Figure 3 Schematic diagram of the CFT-Transformer module;

[0026] Figure 5 yes Figure 4 Schematic diagram of the CFT-FFN fusion module for the middle channel. Detailed Implementation

[0027] The present invention will be further described in detail below with reference to specific implementation schemes and accompanying drawings, but the embodiments described are not intended to limit the present invention.

[0028] like Figure 1 As shown, the process of this invention is as follows:

[0029] Step 1: Acquire LDCT image data Y, with a size of 512*512;

[0030] Step 2: Train the LDCT image restoration network, the LDCT image restoration network as follows: Figure 2 As shown, it includes a 3-level subgraph processing network, denoted as f. θk Where θ is the network parameter and k is the sub-image restoration network level, the output of the second and third level sub-image processing networks of the LDCT image restoration network is fused with the input of the current level and used as the input of the current sub-image restoration network; the sub-image restoration network is as follows: Figure 3 As shown, it consists of two convolutional modules (Conv) and four CFT-Transformer modules. The first convolutional module is used to extract shallow features. Since the input to the CFT-Transformer module is a sequence, the output of the convolutional module is serialized into image patches and its position is encoded before being input into the CFT-Transformer module. Specifically, the image is processed by a 3x3 convolutional kernel to extract features, and the convolution result is unfolded into a sequence. The last convolutional module is used for decoding image reconstruction. Specifically, the feature sequence output by the CFT-Transformer module is first remapped into image features of image size, and then the image features are processed by a 3x3 convolution to obtain the output image. The CFT-Transformer module is as follows: Figure 4 As shown, it consists of two LN layers, one W-MSA layer, and one channel fusion module CFT-FFN, arranged in the order of LN, W-MSA, LN, and CFT-FFN. The first LN input is added to the W-MSA output through a skip connection, and the second LN input is added to the CFT-FFN output through a skip connection. Specifically, the feature Z input to the CFT-Transformer module of the l-th layer... l First, it is normalized using LN, then feature learned using W-MSA, and finally obtained through skip connections. Then, through an LN normalization, a CFT-FFN, and a skip connection, we obtain the feature output Z of layer l+1. l+1 The above process can be represented as: The channel fusion module CFT-FFN, as shown in... Figure 5 As shown, the structure comprises two branches. One branch includes a 3x3 convolutional layer and a sigmoid activation layer, with the input of the convolutional layer multiplied by the output of the sigmoid activation layer via skip connections. The other branch includes only a 3x3 convolutional layer, with the input and output of the convolutional layer added together via skip connections. Finally, the two branches are summed to obtain the forward output. Specifically, the module input is first reshaped to the image size, and then depthwise separable convolution is used. The Conv network outputs two branches. One branch performs a convolution operation on the input, then passes it through a sigmoid activation function, multiplies it by itself as a scaling factor to capture the influence of the local context. The other branch performs a convolution operation on the input, then adds it to itself through skip connections to enhance its own information. Finally, the two are fused along the channel dimension and reshaped into the module input size. The LDCT image restoration network uses sub-neighborhood maps and constraint sub-maps to form image pairs as training data. Given an LDCT image Y with a spatial dimension of 512*512, the four neighborhood sub-maps g0(Y), g1(Y), g2(Y), and g3(Y) of Y are as follows:

[0031]

[0032] After the LDCT image Y is processed by the third-level sub-image processing network, f is obtained. θ3 (Y); then we get 3 constraint subgraphs g1(f) θ3 (Y)),g2(f θ3 (Y)),g3(f θ3 (Y) are respectively:

[0033]

[0034] The loss function for each subgraph processing network is:

[0035]

[0036] Where λ is a hyperparameter, k is the level of the sub-image restoration network, and the loss function of the LDCT image restoration network is:

[0037]

[0038] Specifically, in the training of the LDCT image restoration network in this implementation, the hyperparameter λ is set to 1. The input image pixels and output image pixels of each sub-image restoration network are input into the loss function. The gradient of the loss function with respect to the network parameter θ is calculated by back-derivative calculation through the Adam optimizer. Then, the first-order momentum related to the gradient and the second-order momentum related to the square of the gradient are calculated. Finally, the network parameters are updated based on the first-order momentum and the second-order momentum.

[0039] Step 3: Input the LDCT image to be restored into the third-level sub-image restoration network of the trained LDCT image restoration network, and output the restored LDCT image.

[0040] The preferred embodiment of the present invention provides an LDCT image restoration method that, in the absence of paired LDCT and NDCT images, trains an LDCT image restoration network by constructing neighborhood subgraphs and constraint subgraphs. The LDCT image to be restored is then input into the third-level subgraph restoration network of the trained LDCT image restoration network to complete the LDCT image restoration.

Claims

1. An unsupervised LDCT image restoration method based on neighborhood subgraphs, characterized in that, Includes the following steps: Step S1: Acquire LDCT image data Construct the training dataset; where, Let x and y be the pixel x and y coordinates of the LDCT image Y, respectively. The spatial dimension of the LDCT image Y is N*N. Step S2: Train the LDCT image restoration network. The LDCT image restoration network includes a 3-level sub-image restoration network, denoted as... ,in Here, k represents the network parameters, and k is the subgraph restoration network level. The LDCT image data Y is processed by the subsampler G to generate 4 neighborhood subgraphs. Reconstruction loss is constructed using neighborhood subgraphs. The LDCT images corresponding to the neighboring sub-images are processed through the Level 3 sub-image restoration network. Obtain constraint diagram The constraint graph generates three constraint subgraphs through the subsampler G. Constructing a regularized loss through constrained subgraphs During the generation of constrained subgraphs, the process of LDCT images being restored through the third-level subgraph reconstruction network is not included in the gradient calculation during network training, and the reconstruction loss is used. Regular loss Loss function for constructing subgraph restoration networks ,in For hyperparameters; The LDCT image restoration network consists of three levels of sub-image restoration networks. Each level of sub-image restoration network is trained using the loss function of the sub-image restoration network. Specifically, for the second and third level sub-image processing networks, the output of the next level is fused with the input of the current level at the pixel level and used as the input of the current sub-image restoration network. The subgraph restoration network includes two convolutional modules and four CFT-Transformer modules. Each CFT-Transformer module is sequentially connected to each other. The two convolutional modules are connected at the beginning and end of the interconnected CFT-Transformer modules, respectively. The first convolutional module is used for shallow feature extraction, and the second convolutional module is used for image decoding. Each CFT-Transformer module includes two normalization layers (LN), a windowed multi-head self-attention layer (W-MSA), and a channel fusion module (CFT-FFN), arranged in the order of LN, W-MSA, LN, and CFT-FFN. The input of the first LN is added to the output of W-MSA through a skip connection, and the input of the second LN is added to the output of CFT-FFN through a skip connection. The channel fusion module CFT-FFN in the subgraph restoration network includes two branches. One branch includes a 3*3 convolutional layer and a sigmoid activation layer. The input of the convolutional layer is multiplied by the output of the sigmoid activation layer through a skip connection. The other branch includes only a 3*3 convolutional layer. The input and output of the convolutional layer are added through a skip connection. Finally, the two branches are added together to obtain the forward output. Step S3: Input the LDCT image to be restored into the third-level sub-image restoration network of the trained LDCT image restoration network, and output the restored LDCT image.

2. The unsupervised LDCT image restoration method based on neighborhood subgraphs according to claim 1, characterized in that, In step S2, the subsampler G selects LDCT image data from non-overlapping 2x2 subsample windows. Generate 4 neighborhood subgraphs from each pixel. In each of the neighborhood sub-graphs, all pixels come from the same location within different 2x2-sized sub-sampling windows, given an LDCT image. Then the four neighborhood subgraphs of Y for:

3. The unsupervised LDCT image restoration method based on neighborhood subgraphs according to claim 1, characterized in that, In step S2, the generation of the constraint sub-map is based on an LDCT image. The constraint graph is obtained after passing through the third-level subgraph processing network. constraint diagram Three constraint subgraphs are obtained after passing through the subsampler G. They are respectively:

4. The unsupervised LDCT image restoration method based on neighborhood subgraphs according to claim 1, characterized in that, In step S2, the training of the LDCT image restoration network uses the neighborhood subgraph. As a restoration target, As the subgraph restoration network at each level Input, As a constraint subgraph, it is input into the subgraph restoration network at each level for training. Reconstruction losses The regular loss is The loss function of the subgraph restoration network is: Each level of the subgraph network includes one forward propagation and one backward propagation. One forward propagation constitutes one loss constraint, and one backward propagation updates the parameters of the subgraph restoration network. This LDCT image restoration network... The loss function is:

Images