CT image efficient metal artifact removing system based on multi-domain joint filtering

Abstract

The invention discloses a CT image efficient metal artifact removing system based on multi-domain joint filtering, and belongs to the field of medical image restoration. The problems that an existing metal artifact removing system is low in processing speed and poor in artifact removing effect are solved. According to the method, metal artifacts are removed from a CT image through multi-domain combination, the CT image containing the metal artifacts serves as input by the system, the CT image is mapped to a feature space through input layer coding, step-by-step dimension compression, multi-domain filtering and deep feature coding are carried out on a multi-layer feature map through a coding sub-network, and the multi-layer feature map after dimension reduction is output; and step-by-step decoding is carried out on the multi-layer feature map after dimension reduction and a part of feature maps obtained by a jumping type transmission encoder in the encoding process through a decoding sub-network, an H * W-dimensional multi-layer feature map is output and decoded through an output layer to obtain a residual image, and the residual image and an input image are added to obtain a clear CT image without artifacts. The method is mainly used for removing the metal artifacts of the CT image.

Description

Technical Field

[0001] The invention belongs to the field of medical image restoration. Background Art

[0002] CT imaging technology is an indispensable core tool in the modern medical diagnosis system, and its image quality is directly related to the accuracy and efficiency of clinical decision-making. However, in the scanning of patients with metal implants (such as heart stents, joint replacements, etc.), CT images are inevitably seriously disturbed by metal artifacts. These artifacts are like visual noise, greatly reducing the clarity and contrast of the image, and thus posing a challenge to the accuracy of doctors' diagnosis. Therefore, the development of efficient and accurate CT image metal artifact removal technology is of immeasurable value for improving the reliability of medical diagnosis and promoting the optimization of patient treatment plans. At present, although academia and industry have proposed a variety of methods and models for removing metal artifacts from CT images, they still face significant bottlenecks in practical applications: First, most methods focus on the fine processing of local areas. Although they can reduce artifacts to a certain extent, they often ignore the coherence and integrity of the overall structural information of the image, limiting the further improvement of the artifact removal effect; second, in terms of time efficiency, many algorithms are difficult to meet the urgent needs of rapid imaging and immediate diagnosis in clinical environments due to their high computational complexity, affecting the overall efficiency of medical services. Summary of the invention

[0003] The purpose of the present invention is to solve the problems of slow processing speed and poor artifact removal effect of the existing metal artifact removal system; the present invention provides a CT image efficient metal artifact removal system based on multi-domain joint filtering.

[0004] An efficient metal artifact removal system for CT images based on multi-domain joint filtering includes an input layer, an encoding subnetwork, a decoding subnetwork and an output layer arranged in sequence;

[0005] The input layer is used to map the input H×W dimensional CT image containing metal artifacts into a multi-layer feature map; H and W are the height resolution and width resolution of the input image respectively;

[0006] The encoding subnetwork is used to perform step-by-step dimensional compression, multi-domain filtering, and deep feature encoding on the multi-layer feature maps, and output the multi-layer feature maps after dimensionality reduction;

[0007] The decoding subnetwork is used to decode the multi-layer feature map after dimensionality reduction and some feature maps obtained by the encoder of the jump transfer in the encoding process step by step, and output the H×W dimensional multi-layer feature map;

[0008] The output layer is used to decode the H×W dimensional multi-layer feature map output by the decoding subnetwork into a residual image R, and add it to the input CT image containing metal artifacts to obtain a CT image without metal artifacts.

[0009] Preferably, the encoding subnetwork includes N multi-domain Transformer integrated encoding layers and N / 2-1 down-converters, and two adjacent multi-domain Transformer integrated encoding layers are divided into a group of encoding units, and N / 2 groups of encoding units and N / 2-1 down-converters are arranged alternately, and the output end of the former is connected to the input end of the latter;

[0010] The decoding subnetwork includes N multi-domain Transformer integrated decoding layers and N / 2-1 up-converters, and two adjacent multi-domain Transformer integrated encoding layers are divided into a group of decoding units, N / 2 groups of decoding units and N / 2-1 down-converters are arranged alternately, a feature addition module is connected between the output end of any adjacent down-converter and the input end of the decoding unit, and the first input end of the feature addition module is connected to the output end of the down-converter, and the output end of the feature addition module is connected to the input end of the decoding unit;

[0011] There is a one-to-one correspondence between the N / 2 groups of encoding units and the N / 2 groups of decoding units;

[0012] The second input end of each feature sum module is connected to the output end of the corresponding multi-domain Transformer integrated encoding layer, and the dimensions of the feature maps inputted by the two input ends of the feature sum module are the same;

[0013] The multi-domain Transformer integrated encoding layer and the multi-domain Transformer integrated decoding layer are both implemented by a multi-domain Transformer network consisting of a multi-domain joint filtering module and an overall-granularity feature extraction module arranged in sequence, which is used to perform multi-domain filtering and overall-fine-grained feature extraction operations on the input multi-layer feature map in sequence;

[0014] The up-converter and the down-converter are used for up-sampling and down-sampling respectively.

[0015] Preferably, the implementation of the multi-domain joint filtering module to perform the multi-domain filtering operation includes the following process:

[0016] The convolution module G1 performs layer normalization and convolution operations on the input multi-layer feature map; and the input end of the convolution module G1 is used as the input end of the multi-domain joint filtering module in the multi-domain Transformer network; the convolution module G2 performs a convolution operation on the output of the convolution module G1;

[0017] The first feature separation module performs multi-layer feature map separation on the output of the convolution module G2 and divides it into 5 equal parts, which are recorded as feature maps F1 to F5 respectively;

[0018] The new feature map obtained by connecting the feature maps obtained by performing Fourier space filtering operation on the feature map F1 through the Fourier path branch module, performing Radon space filtering operation on the feature map F2 through the Radon path branch module, and performing spatial space filtering operation on the feature map F3 through the spatial path branch module is convolved by the convolution module G5, and then the shape is adjusted to obtain the feature vector v;

[0019] The convolution module G3 performs convolution operation on the feature map F4 and then adjusts the shape to obtain the feature vector k;

[0020] The convolution module G4 performs convolution operation on the feature map F5 and then adjusts the shape to obtain the feature vector q;

[0021] After matrix multiplication of feature vector k and feature vector q and softmax operation, attention matrix A is obtained;

[0022] After the feature vector v is matrix-multiplied with the feature attention matrix A and then reshaped, it is sent to the convolution module G6 for convolution operation, and then added with the multi-layer feature map received by the convolution module G1, and the multi-layer feature map is output as the output of the multi-domain joint filtering module;

[0023] The convolution operation performed by the convolution module G1 is 5C 1×1×C convolution operations with a stride of 1; C is the number of feature layers of the multi-layer feature map output by the input layer;

[0024] The convolution operation performed by the convolution module G2 is 5C 3×3 convolution operations with a stride of 1;

[0025] The convolution operation performed by the convolution module G3 is C 3×3 convolution operations with a stride of 1;

[0026] The convolution operation performed by the convolution module G4 is C 5×5 convolution operations with a stride of 1;

[0027] The convolution operation performed by the convolution module G5 is C 1×1×2C convolution operations with a stride of 1;

[0028] The convolution operation performed by the convolution module G6 is C 1×1×C convolution operations with a step size of 1.

[0029] Preferably, the implementation method of performing Fourier space filtering operation on the feature graph F1 through the Fourier path branch module includes: performing fast Fourier transform on the feature graph F1, filtering through a Fourier filter, and inverse Fourier transform in sequence.

[0030] Preferably, the implementation method of performing the Radon spatial filtering operation on the feature graph F2 through the Radon path branch module includes: performing Radon transform, filtering and inverse Radon transform on the feature graph F2 in sequence.

[0031] Preferably, the spatial filtering operation in the spatial domain is implemented using a spatial domain filter.

[0032] Preferably,

[0033] Among them, softmax(·) is the soft maximum function, d is the number of multiple heads in the channel self-attention mechanism, (·) T Represents matrix transpose.

[0034] Preferably, the overall-granularity feature extraction module performs overall-fine-granularity feature extraction operations in the following manners:

[0035] The multi-layer feature maps output by the convolution module H1 after multi-domain filtering are normalized and activated after convolution operations in turn;

[0036] The second feature separation module performs multi-layer feature map separation on the output of the convolution module H1 and divides it into 5 equal parts, which are recorded as feature maps FG1 to FG5 respectively;

[0037] The convolution module H2 performs convolution and activation operations on the feature map FG1;

[0038] The convolution module H3 performs a convolution operation on the feature map FG2;

[0039] The output of convolution module H2 is multiplied by the output of convolution module H3 to obtain a fine-grained feature map;

[0040] The feature maps FG3, FG4 and FG5 are subjected to fast Fourier transform to obtain the feature maps FG3', FG4' and FG5' respectively. The conjugate of the feature map FG3' is multiplied by the feature map FG4' and then subjected to inverse Fourier transform, activation operation and fast Fourier transform to obtain the overall selection coefficient C O ;

[0041] The overall selection coefficient Co is multiplied by the feature map FG5', and an inverse Fourier transform is performed to obtain the overall feature map;

[0042] After the fine-grained feature map and the overall feature map are connected by the convolution module H2, the feature map is added with the multi-layer feature map output after the multi-domain filtering operation to obtain the overall-fine-grained feature as the output of the overall-grained feature extraction module;

[0043] The convolution operation performed by the convolution module H1 is 5C / 2 1×1×C convolution operations with a stride of 1;

[0044] The convolution operation performed by the convolution module H2 is C 3×3 convolution operations with a stride of 1;

[0045] The convolution operation performed by the convolution module H3 is C 3×3 convolution operations with a stride of 1;

[0046] The convolution operation performed by the convolution module H4 is C 1×1×C convolution operations with a step size of 1.

[0047] Preferably, when N=10 and the number of feature layers of the feature map output by the input layer is C=16, the N multi-domain Transformer integrated encoding layers in the encoding subnetwork are defined as E1 to E10, and the N / 2-1 down-converters are defined as P1 to P4; the decoding subnetwork includes N multi-domain Transformer integrated decoding layers defined as S1 to S10, and the N / 2-1 up-converters are defined as D1 to D4;

[0048] The multi-domain Transformer integrated encoding layer E1 is used to perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the C-layer feature map output by the input layer in sequence;

[0049] The multi-domain Transformer integrated coding layer E2 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E1;

[0050] The multi-domain Transformer integrated coding layer E3 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E2;

[0051] The multi-domain Transformer integrated coding layer E4 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E3;

[0052] The multi-domain Transformer integrated coding layer E5 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E4;

[0053] The multi-domain Transformer integrated coding layer E6 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E5;

[0054] The multi-domain Transformer integrated coding layer E7 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E6;

[0055] The multi-domain Transformer integrated coding layer E8 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E7;

[0056] The multi-domain Transformer integrated coding layer E9 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E8;

[0057] The multi-domain Transformer integrated coding layer E10 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E9;

[0058] The down-converter P1 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated encoding layer E2;

[0059] The down-converter P2 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated coding layer E4;

[0060] The down-converter P3 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated coding layer E6;

[0061] The down-converter P4 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated coding layer E8;

[0062] The resolution of the operation object of the multi-domain Transformer integrated encoding layer E1 and E2 is H×W;

[0063] The operation object resolution of the multi-domain Transformer integrated coding layer E3 and E4 is H / 2×W / 2;

[0064] The operation object resolution of the multi-domain Transformer integrated coding layers E5 and E6 is H / 4×W / 4;

[0065] The operation object resolution of the multi-domain Transformer integrated coding layers E7 and E8 is H / 8×W / 8;

[0066] The operation object resolution of the multi-domain Transformer integrated coding layer E9 and E10 is H / 16×W / 16; the multi-domain Transformer integrated decoding layer S10 is used to perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E10 in sequence;

[0067] The multi-domain Transformer integrated decoding layer S9 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S10;

[0068] The up-converter D4 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S9;

[0069] The fourth feature addition module is used to add the output of the up-converter D4 and the output of the multi-domain Transformer integrated encoding layer E8 and send them to the multi-domain Transformer integrated decoding layer S8 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence;

[0070] The multi-domain Transformer integrated decoding layer S7 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S8;

[0071] The up-converter D3 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S7;

[0072] The third feature addition module is used to add the output of the up-converter D3 and the output of the multi-domain Transformer integrated encoding layer E6 and send them to the multi-domain Transformer integrated decoding layer S6 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence;

[0073] The multi-domain Transformer integrated decoding layer S5 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S6;

[0074] The up-converter D2 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S5;

[0075] The second feature summing module is used to sum the output of the up-converter D2 and the output of the multi-domain Transformer integrated encoding layer E4 and send them to the multi-domain Transformer integrated decoding layer S4 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence;

[0076] The multi-domain Transformer integrated decoding layer S3 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S4;

[0077] The up-converter D1 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S3;

[0078] The first feature summing module is used to sum the output of the up-converter D1 and the output of the multi-domain Transformer integrated encoding layer E2 and send them to the multi-domain Transformer integrated decoding layer S2 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence;

[0079] The multi-domain Transformer integrated decoding layer S1 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S2;

[0080] The operation object resolution size of the multi-domain Transformer integrated decoding layer S1 and S2 is H×W;

[0081] The operation object resolution size of the multi-domain Transformer integrated decoding layer S3 and S4 is H / 2×W / 2;

[0082] The operation object resolution size of the multi-domain Transformer integrated decoding layer S5 and S6 is H / 4×W / 4;

[0083] The operation object resolution of the multi-domain Transformer integrated decoding layers S7 and S8 is H / 8×W / 8;

[0084] The operation object resolution size of the multi-domain Transformer integrated decoding layer S9 and S10 is H / 16×W / 16;

[0085] The number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E1 and E2, and the multi-domain Transformer integrated decoding layers S1 and S2 is 16, and the number of multi-heads is 2; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E3 and E4, and the multi-domain Transformer integrated decoding layers S3 and S4 is 32, and the number of multi-heads is 4; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E5 and E6, and the multi-domain Transformer integrated decoding layers S5 and S6 is 64, and the number of multi-heads is 8; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E7 and E8, and the multi-domain Transformer integrated decoding layers S7 and S8 is 128, and the number of multi-heads is 16; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E9 and E10, and the multi-domain Transformer integrated decoding layers S9 and S10 is 256, and the number of multi-heads is 32.

[0086] Preferably, the loss function Loss of the metal artifact removal system in the training phase is expressed as:

[0087]

[0088] In the formula, l 2 ||·|| is L2 regularization, I is the high-definition original image, Y is the input CT image containing metal artifacts, is the mask of non-metallic areas, is the number of samples, PSNR(·) represents the PSNR loss function, represents the Radon transform.

[0089] The beneficial effects brought by the present invention are:

[0090] Compared with the existing CT image metal artifact removal system, the present invention proposes for the first time the use of multi-domain joint for CT image metal artifact removal, specifically the use of the Fourier-spatial domain multi-domain Transformer network for CT image metal artifact removal. It can effectively and efficiently utilize the overall and local information in the image, so as to quickly and quickly remove the metal artifacts in the image. The local information is mainly carried out in the spatial domain, and the global information is simultaneously reflected in the Fourier domain, the Radon domain and the spatial domain (channel self-attention mechanism).

[0091] The computational complexity of the Fourier transform and its inverse transform involved in the present invention is HWlog(HW), and the computational complexity of the channel self-attention mechanism is linearly related to the image resolution (H×W), so it can run efficiently.

[0092] The present invention significantly improves the visual quality of CT images containing metal artifacts through innovative image processing methods, making them clearer and easier to interpret. At the same time, the algorithm has extremely high computational efficiency and can process a large amount of medical imaging data in real time, providing clinicians with a fast and accurate basis for diagnosis. This not only helps to improve the accuracy of diagnosis, but also helps to shorten the waiting time of patients, optimize treatment plans, and ultimately improve the quality of life of patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 The overall architecture of the multi-domain joint filtering CT image efficient metal artifact removal system of the present invention;

[0094] Figure 2 A schematic diagram of the principle of multi-domain joint filtering module performing multi-domain filtering operation;

[0095] Figure 3 A schematic diagram of the principle of extracting the overall and fine-grained features of an image by the overall-grained feature extraction module;

[0096] Figure 4The figure is a comparison diagram before and after the artifact removal using the CT image efficient metal artifact removal system based on multi-domain joint filtering of the present invention; wherein, Figure 4 (a) is the CT image containing metal artifacts input to the system; Figure 4 (b) Clear CT image obtained after system processing. DETAILED DESCRIPTION

[0097] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0098] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0099] Specific implementation method 1. Figure 1 The present embodiment is described. The CT image efficient metal artifact removal system based on multi-domain joint filtering described in the present embodiment comprises an input layer, an encoding sub-network, a decoding sub-network and an output layer which are arranged in sequence;

[0100] The input layer is used to map the input H×W dimensional CT image containing metal artifacts into a multi-layer feature map; H and W are the height resolution and width resolution of the input image respectively;

[0101] The encoding subnetwork is used to perform step-by-step dimensional compression, multi-domain filtering, and deep feature encoding on the multi-layer feature maps, and output the multi-layer feature maps after dimensionality reduction;

[0102] The decoding subnetwork is used to decode the multi-layer feature map after dimensionality reduction and some feature maps obtained by the encoder of the jump transfer in the encoding process step by step, and output the H×W dimensional multi-layer feature map;

[0103] The output layer is used to decode the H×W dimensional multi-layer feature map output by the decoding subnetwork into a residual image R, and add it to the input CT image containing metal artifacts to obtain a CT image without metal artifacts.

[0104] The system of the present invention takes a CT image containing metal artifacts as input, maps it to a feature space through input layer encoding, and then decodes it through the encoding sub-network, decoding sub-network and output layer in sequence to obtain a residual image, which is then added to the input image to obtain an artifact-free and clear CT image. The present invention proposes for the first time to use multi-domain joint for CT image metal artifact removal, effectively and efficiently utilizing the overall and local information in the image, thereby better removing the metal artifacts in the image.

[0105] Furthermore, the use of multi-domain Transformer networks for CT image metal artifact removal can effectively and efficiently utilize the overall and local information in the image, thereby better removing the metal artifacts in the image; specifically,

[0106] The encoding subnetwork includes N multi-domain Transformer integrated encoding layers and N / 2-1 down-converters, and two adjacent multi-domain Transformer integrated encoding layers are divided into a group of encoding units, and N / 2 groups of encoding units and N / 2-1 down-converters are arranged alternately, and the output of the former is connected to the input of the latter;

[0107] The decoding subnetwork includes N multi-domain Transformer integrated decoding layers and N / 2-1 up-converters, and two adjacent multi-domain Transformer integrated encoding layers are divided into a group of decoding units, N / 2 groups of decoding units and N / 2-1 down-converters are arranged alternately, a feature addition module is connected between the output end of any adjacent down-converter and the input end of the decoding unit, and the first input end of the feature addition module is connected to the output end of the down-converter, and the output end of the feature addition module is connected to the input end of the decoding unit;

[0108] There is a one-to-one correspondence between the N / 2 groups of encoding units and the N / 2 groups of decoding units;

[0109] The second input end of each feature sum module is connected to the output end of the corresponding multi-domain Transformer integrated encoding layer, and the dimensions of the feature maps inputted by the two input ends of the feature sum module are the same;

[0110] The multi-domain Transformer integrated encoding layer and the multi-domain Transformer integrated decoding layer are both implemented by a multi-domain Transformer network consisting of a multi-domain joint filtering module and an overall-granularity feature extraction module arranged in sequence, which is used to perform multi-domain filtering and overall-fine-grained feature extraction operations on the input multi-layer feature map in sequence;

[0111] The up-converter and the down-converter are used for up-sampling and down-sampling respectively.

[0112] Furthermore, the multi-domain Transformer network in Fourier-spatial domain is used to remove metal artifacts from CT images. The overall and local information in the image is efficiently used to better remove metal artifacts in the image. The local information is mainly carried out in the spatial domain, while the global information is simultaneously reflected in the Fourier domain, Radon domain, and spatial domain (channel self-attention mechanism). For details, see Figure 2 Specifically, the implementation method of the multi-domain joint filtering module for multi-domain filtering operation includes the following process:

[0113] The convolution module G1 performs layer normalization and convolution operations on the input multi-layer feature map; and the input end of the convolution module G1 is used as the input end of the multi-domain joint filtering module in the multi-domain Transformer network; the convolution module G2 performs a convolution operation on the output of the convolution module G1;

[0114] The first feature separation module performs multi-layer feature map separation on the output of the convolution module G2 and divides it into 5 equal parts, which are recorded as feature maps F1 to F5 respectively;

[0115] The new feature map obtained by connecting the feature maps obtained by performing Fourier space filtering operation on the feature map F1 through the Fourier path branch module, performing Radon space filtering operation on the feature map F2 through the Radon path branch module, and performing spatial space filtering operation on the feature map F3 through the spatial path branch module is convolved by the convolution module G5, and then the shape is adjusted to obtain the feature vector v;

[0116] The convolution module G3 performs convolution operation on the feature map F4 and then adjusts the shape to obtain the feature vector k;

[0117] The convolution module G4 performs convolution operation on the feature map F5 and then adjusts the shape to obtain the feature vector q;

[0118] After matrix multiplication of feature vector k and feature vector q and softmax operation, attention matrix A is obtained; specifically, Among them, softmax(·) is the soft maximum function, d is the number of multiple heads in the channel self-attention mechanism, and (·) T Represents matrix transpose;

[0119] After the feature vector v is matrix-multiplied with the feature attention matrix A and then reshaped, it is sent to the convolution module G6 for convolution operation, and then added with the multi-layer feature map received by the convolution module G1, and the multi-layer feature map is output as the output of the multi-domain joint filtering module;

[0120] The convolution operation performed by the convolution module G1 is 5C 1×1×C convolution operations with a stride of 1; C is the number of feature layers of the multi-layer feature map output by the input layer;

[0121] The convolution operation performed by the convolution module G2 is 5C 3×3 convolution operations with a stride of 1;

[0122] The convolution operation performed by the convolution module G3 is C 3×3 convolution operations with a stride of 1;

[0123] The convolution operation performed by the convolution module G4 is C 5×5 convolution operations with a stride of 1;

[0124] The convolution operation performed by the convolution module G5 is C 1×1×2C convolution operations with a stride of 1;

[0125] The convolution operation performed by the convolution module G6 is C 1×1×C convolution operations with a step size of 1.

[0126] In this preferred embodiment, a channel self-attention operation is introduced, specifically: the feature vector k and the feature vector q are matrix multiplied and a softmax operation is performed to obtain the attention matrix A, and the feature vector v is matrix multiplied with the feature attention matrix A and then the shape is adjusted to output the vector feature v'.

[0127] See also Figure 2 As an example, the implementation method of performing Fourier space filtering operation on the feature map F1 through the Fourier path branch module includes: after performing fast Fourier transform on the feature map F1, filtering through the Fourier filter and inverse Fourier transform in sequence. And the Fourier space filtering operation is: filtering operation of the Fourier filter with the same feature resolution size;

[0128] The implementation method of performing the Radon spatial filtering operation on the feature map F2 through the Radon path branch module includes: performing Radon transform, filtering and inverse Radon transform on the feature map F2 in sequence; as an example, the Radon filtering operation is a 1×1 convolution filtering operation.

[0129] The spatial filtering operation in the spatial domain is implemented by using a spatial domain filter. Specifically, the spatial domain filtering operation is a filtering operation of a spatial domain filter having a size consistent with the feature resolution.

[0130] See also Figure 3 ,Furthermore, the overall-granularity feature extraction module implements the overall-fine-grained feature extraction operation in the following ways:

[0131] The multi-layer feature maps output by the convolution module H1 after multi-domain filtering are normalized and activated after the convolution operation.

[0132] The second feature separation module performs multi-layer feature map separation on the output of the convolution module H1 and divides it into 5 equal parts, which are recorded as feature maps FG1 to FG5 respectively;

[0133] The convolution module H2 performs convolution and activation operations on the feature map FG1;

[0134] The convolution module H3 performs a convolution operation on the feature map FG2;

[0135] The output of convolution module H2 is multiplied by the output of convolution module H3 to obtain a fine-grained feature map;

[0136] The feature maps FG3, FG4 and FG5 are subjected to fast Fourier transform to obtain the feature maps FG3', FG4' and FG5' respectively. The conjugate of the feature map FG3' is multiplied by the feature map FG4' and then subjected to inverse Fourier transform, activation operation and fast Fourier transform to obtain the overall selection coefficient C O ;

[0137] The overall selection coefficient Co is dot-multiplied with the feature map FG5', and an inverse Fourier transform is performed to obtain the overall feature map as the output of the overall-granularity feature extraction module;

[0138] After the fine-grained feature map and the overall feature map are connected by the convolution module H2, the feature map is added with the multi-layer feature map output after the multi-domain filtering operation to obtain the overall-fine-grained feature;

[0139] The convolution operation performed by the convolution module H1 is 5C / 2 1×1×C convolution operations with a stride of 1;

[0140] The convolution operation performed by the convolution module H2 is C 3×3 convolution operations with a stride of 1;

[0141] The convolution operation performed by the convolution module H3 is C 3×3 convolution operations with a stride of 1;

[0142] The convolution operation performed by the convolution module H4 is C 1×1×C convolution operations with a step size of 1.

[0143] Figure 3 In , the activation operation is implemented using Gaussian error linear units.

[0144] As an example see Figure 1 , when N=10 and the number of feature layers of the feature map output by the input layer is C=16, the N multi-domain Transformer integrated encoding layers in the encoding subnetwork are defined as E1 to E10, and the N / 2-1 down-converters are defined as P1 to P4; the decoding subnetwork includes N multi-domain Transformer integrated decoding layers defined as S1 to S10, and the N / 2-1 up-converters are defined as D1 to D4;

[0145] The multi-domain Transformer integrated encoding layer E1 is used to perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the C-layer feature map output by the input layer in sequence;

[0146] The multi-domain Transformer integrated coding layer E2 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E1;

[0147] The multi-domain Transformer integrated coding layer E3 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E2;

[0148] The multi-domain Transformer integrated coding layer E4 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E3;

[0149] The multi-domain Transformer integrated coding layer E5 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E4;

[0150] The multi-domain Transformer integrated coding layer E6 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E5;

[0151] The multi-domain Transformer integrated coding layer E7 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E6;

[0152] The multi-domain Transformer integrated coding layer E8 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E7;

[0153] The multi-domain Transformer integrated coding layer E9 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E8;

[0154] The multi-domain Transformer integrated coding layer E10 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E9;

[0155] The down-converter P1 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated encoding layer E2;

[0156] The down-converter P2 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated coding layer E4;

[0157] The down-converter P3 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated coding layer E6;

[0158] The down-converter P4 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated coding layer E8;

[0159] The resolution of the operation object of the multi-domain Transformer integrated encoding layer E1 and E2 is H×W;

[0160] The operation object resolution of the multi-domain Transformer integrated coding layer E3 and E4 is H / 2×W / 2;

[0161] The operation object resolution of the multi-domain Transformer integrated coding layers E5 and E6 is H / 4×W / 4;

[0162] The operation object resolution of the multi-domain Transformer integrated coding layers E7 and E8 is H / 8×W / 8;

[0163] The operation object resolution of the multi-domain Transformer integrated coding layer E9 and E10 is H / 16×W / 16; the multi-domain Transformer integrated decoding layer S10 is used to perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E10 in sequence;

[0164] The multi-domain Transformer integrated decoding layer S9 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S10;

[0165] The up-converter D4 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S9;

[0166] The fourth feature addition module is used to add the output of the up-converter D4 and the output of the multi-domain Transformer integrated encoding layer E8 and send them to the multi-domain Transformer integrated decoding layer S8 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence;

[0167] The multi-domain Transformer integrated decoding layer S7 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S8;

[0168] The up-converter D3 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S7;

[0169] The third feature addition module is used to add the output of the up-converter D3 and the output of the multi-domain Transformer integrated encoding layer E6 and send them to the multi-domain Transformer integrated decoding layer S6 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence;

[0170] The multi-domain Transformer integrated decoding layer S5 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S6;

[0171] The up-converter D2 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S5;

[0172] The second feature summing module is used to sum the output of the up-converter D2 and the output of the multi-domain Transformer integrated encoding layer E4 and send them to the multi-domain Transformer integrated decoding layer S4 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence;

[0173] The multi-domain Transformer integrated decoding layer S3 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S4;

[0174] The up-converter D1 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S3;

[0175] The first feature summing module is used to sum the output of the up-converter D1 and the output of the multi-domain Transformer integrated encoding layer E2 and send them to the multi-domain Transformer integrated decoding layer S2 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence;

[0176] The multi-domain Transformer integrated decoding layer S1 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S2;

[0177] The operation object resolution size of the multi-domain Transformer integrated decoding layer S1 and S2 is H×W;

[0178] The operation object resolution size of the multi-domain Transformer integrated decoding layer S3 and S4 is H / 2×W / 2;

[0179] The operation object resolution size of the multi-domain Transformer integrated decoding layer S5 and S6 is H / 4×W / 4;

[0180] The operation object resolution of the multi-domain Transformer integrated decoding layers S7 and S8 is H / 8×W / 8;

[0181] The operation object resolution size of the multi-domain Transformer integrated decoding layer S9 and S10 is H / 16×W / 16;

[0182] The number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E1 and E2, and the multi-domain Transformer integrated decoding layers S1 and S2 is 16, and the number of multi-heads is 2; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E3 and E4, and the multi-domain Transformer integrated decoding layers S3 and S4 is 32, and the number of multi-heads is 4; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E5 and E6, and the multi-domain Transformer integrated decoding layers S5 and S6 is 64, and the number of multi-heads is 8; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E7 and E8, and the multi-domain Transformer integrated decoding layers S7 and S8 is 128, and the number of multi-heads is 16; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E9 and E10, and the multi-domain Transformer integrated decoding layers S9 and S10 is 256, and the number of multi-heads is 32.

[0183] In this preferred embodiment, the efficient metal artifact removal system for CT images based on multi-domain joint filtering can be effective and the computational complexity of the Fourier transform and its inverse transform involved in the present invention is HWlog (HW), and the computational complexity of the channel self-attention mechanism is linearly related to the image resolution (H×W), so it can run efficiently.

[0184] Through simulation experiments, it is estimated that the time required for scoring a 416*416 CT image is about 20ms using the CT image efficient metal artifact removal system based on multi-domain joint filtering when N=10 and the number of feature layers of the feature map output by the input layer is C=16. The CT image efficient artifact removal system of this embodiment performs well in terms of speed and effect, see Figure 4 .

[0185] The CT image efficient metal artifact removal system based on multi-domain joint filtering described in the present invention introduces multi-domain filtering, Fourier correlation and channel self-attention mechanism on the basis of the existing deep convolutional neural network structure. The system of the present invention takes the CT image containing metal artifacts as input, maps it to the feature space through input layer encoding, and then decodes it through the encoding sub-network, decoding sub-network and output layer in sequence to obtain the residual image, and then adds it to the input image to obtain the output artifact-free and clear CT image.

[0186] Furthermore, in the training phase of the efficient metal artifact removal system for CT images based on multi-domain joint filtering, the AdamW optimization algorithm is used to perform end-to-end training on the weight prediction network.

[0187] The expression of the loss function Loss of the metal artifact removal system in the training stage is:

[0188]

[0189] In the formula, l 2 ||·|| is L2 regularization, I is the high-definition original image, Y is the input CT image containing metal artifacts, is the mask of non-metallic areas, is the number of samples, PSNR(·) represents the PSNR loss function, represents the Radon transform.

[0190] The present invention discloses a highly efficient CT image metal artifact removal system based on multi-domain joint filtering, which introduces spatial and frequency domain joint filters. The metal artifact removal deep network includes an input layer, an output layer, a coding subnetwork, and a decoding subnetwork; the coding subnetwork includes N multi-domain Transformer integrated coding layers and N / 2-1 down-converters; the decoding subnetwork includes N multi-domain Transformer integrated decoding layers and N / 2-1 up-converters, and the multi-domain Transformer integrated coding layers and the multi-domain Transformer integrated decoding layers are both implemented by a multi-domain Transformer network composed of a multi-domain joint filtering module and an overall-granularity feature extraction module arranged in sequence; the multi-domain joint filtering module performs weighted fusion of the features filtered by spatial and frequency domains through a channel self-attention mechanism to make full use of the complementarity of information in different frequency domains. The overall-granularity feature extraction module performs Fourier autocorrelation and pixel-by-pixel gating operations on the features to extract global and local features. The CT image containing metal artifacts is mapped to the feature space through the input layer encoding, and the feature map is input into the coding subnetwork. The encoding subnetwork gradually extracts deep features by alternating spatial-frequency domain joint filtering and feature selection, combined with downsampling operations. The decoding subnetwork restores the deep features to the original image size through upsampling and symmetrical decoding processes. Finally, the output layer and residual learning are used to obtain the CT image after removing metal artifacts.

[0191] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. An efficient metal artifact removal system for CT images based on multi-domain joint filtering, characterized in that: It includes an input layer, an encoding sub-network, a decoding sub-network and an output layer which are arranged in sequence; The input layer is used to map the input H×W dimensional CT image containing metal artifacts into a multi-layer feature map; H and W are the height resolution and width resolution of the input image respectively; The encoding subnetwork is used to perform step-by-step dimensional compression, multi-domain filtering, and deep feature encoding on the multi-layer feature maps, and output the multi-layer feature maps after dimensionality reduction; The decoding subnetwork is used to decode the multi-layer feature map after dimensionality reduction and some feature maps obtained by the encoder of the jump transfer in the encoding process step by step, and output the H×W dimensional multi-layer feature map; The output layer is used to decode the H×W dimensional multi-layer feature map output by the decoding subnetwork into a residual image R, and add it to the input CT image containing metal artifacts to obtain a CT image without metal artifacts.

2. The CT image efficient metal artifact removal system based on multi-domain joint filtering according to claim 1 is characterized in that: The encoding subnetwork includes N multi-domain Transformer integrated encoding layers and N / 2-1 down-converters, and two adjacent multi-domain Transformer integrated encoding layers are divided into a group of encoding units, and N / 2 groups of encoding units and N / 2-1 down-converters are arranged alternately, and the output of the former is connected to the input of the latter; The decoding subnetwork includes N multi-domain Transformer integrated decoding layers and N / 2-1 up-converters, and two adjacent multi-domain Transformer integrated encoding layers are divided into a group of decoding units, N / 2 groups of decoding units and N / 2-1 down-converters are arranged alternately, a feature addition module is connected between the output end of any adjacent down-converter and the input end of the decoding unit, and the first input end of the feature addition module is connected to the output end of the down-converter, and the output end of the feature addition module is connected to the input end of the decoding unit; There is a one-to-one correspondence between the N / 2 groups of encoding units and the N / 2 groups of decoding units; The second input end of each feature sum module is connected to the output end of the corresponding multi-domain Transformer integrated encoding layer, and the dimensions of the feature maps inputted by the two input ends of the feature sum module are the same; The multi-domain Transformer integrated encoding layer and the multi-domain Transformer integrated decoding layer are both implemented by a multi-domain Transformer network consisting of a multi-domain joint filtering module and an overall-granularity feature extraction module arranged in sequence, which is used to perform multi-domain filtering and overall-fine-grained feature extraction operations on the input multi-layer feature map in sequence; The up-converter and the down-converter are used for up-sampling and down-sampling respectively.

3. The CT image efficient metal artifact removal system based on multi-domain joint filtering according to claim 2 is characterized in that: The implementation of the multi-domain joint filtering module for multi-domain filtering operation includes the following process: The convolution module G1 performs layer normalization and convolution operations on the input multi-layer feature map; and the input end of the convolution module G1 is used as the input end of the multi-domain joint filtering module in the multi-domain Transformer network; the convolution module G2 performs a convolution operation on the output of the convolution module G1; The first feature separation module performs multi-layer feature map separation on the output of the convolution module G2 and divides it into 5 equal parts, which are recorded as feature maps F1 to F5 respectively; The new feature map obtained by connecting the feature maps obtained by performing Fourier space filtering operation on the feature map F1 through the Fourier path branch module, performing Radon space filtering operation on the feature map F2 through the Radon path branch module, and performing spatial space filtering operation on the feature map F3 through the spatial path branch module is convolved by the convolution module G5, and then the shape is adjusted to obtain the feature vector v; The convolution module G3 performs convolution operation on the feature map F4 and then adjusts the shape to obtain the feature vector k; The convolution module G4 performs convolution operation on the feature map F5 and then adjusts the shape to obtain the feature vector q; After matrix multiplication of feature vector k and feature vector q and softmax operation, attention matrix A is obtained; After the feature vector v is matrix-multiplied with the feature attention matrix A and then reshaped, it is sent to the convolution module G6 for convolution operation, and then added with the multi-layer feature map received by the convolution module G1, and the multi-layer feature map is output as the output of the multi-domain joint filtering module; The convolution operation performed by the convolution module G1 is 5C 1×1×C convolution operations with a stride of 1; C is the number of feature layers of the multi-layer feature map output by the input layer; The convolution operation performed by the convolution module G2 is 5C 3×3 convolution operations with a stride of 1; The convolution operation performed by the convolution module G3 is C 3×3 convolution operations with a stride of 1; The convolution operation performed by the convolution module G4 is C 5×5 convolution operations with a stride of 1; The convolution operation performed by the convolution module G5 is C 1×1×2C convolution operations with a stride of 1; The convolution operation performed by the convolution module G6 is C 1×1×C convolution operations with a step size of 1.

4. The CT image efficient metal artifact removal system based on multi-domain joint filtering according to claim 3 is characterized in that: The implementation method of performing Fourier space filtering operation on the feature map F1 through the Fourier path branch module includes: After the feature map F1 is fast Fourier transformed, it is filtered through a Fourier filter and inverse Fourier transformed in sequence.

5. The CT image efficient metal artifact removal system based on multi-domain joint filtering according to claim 3 is characterized in that: The implementation method of performing the Radon spatial filtering operation on the feature map F2 through the Radon path branch module includes: The feature map F2 is subjected to Radon transform, filtering and inverse Radon transform in sequence.

6. The CT image efficient metal artifact removal system based on multi-domain joint filtering according to claim 3 is characterized in that: The spatial filtering operation in the spatial domain is implemented using a spatial domain filter.

7. The CT image efficient metal artifact removal system based on multi-domain joint filtering according to claim 3 is characterized in that: Among them, softmax(·) is the soft maximum function, d is the number of multiple heads in the channel self-attention mechanism, (·) T Represents matrix transpose.

8. The CT image efficient metal artifact removal system based on multi-domain joint filtering according to claim 2 is characterized in that: The overall-granularity feature extraction module implements overall-fine-granularity feature extraction operations in the following ways: The multi-layer feature maps output by the convolution module H1 after multi-domain filtering are normalized and activated after convolution operations in turn; The second feature separation module performs multi-layer feature map separation on the output of the convolution module H1 and divides it into 5 equal parts, which are recorded as feature maps FG1 to FG5 respectively; The convolution module H2 performs convolution and activation operations on the feature map FG1; The convolution module H3 performs a convolution operation on the feature map FG2; The output of convolution module H2 is multiplied by the output of convolution module H3 to obtain a fine-grained feature map; The feature maps FG3, FG4 and FG5 are subjected to fast Fourier transform to obtain the feature maps FG3', FG4' and FG5' respectively. The conjugate of the feature map FG3' is multiplied by the feature map FG4' and then subjected to inverse Fourier transform, activation operation and fast Fourier transform to obtain the overall selection coefficient C O ; The overall selection coefficient Co is multiplied by the feature map FG5', and an inverse Fourier transform is performed to obtain the overall feature map; After the fine-grained feature map and the overall feature map are connected by the convolution module H2, the feature map is added with the multi-layer feature map output after the multi-domain filtering operation to obtain the overall-fine-grained feature as the output of the overall-grained feature extraction module; The convolution operation performed by the convolution module H1 is 5C / 2 1×1×C convolution operations with a stride of 1; The convolution operation performed by the convolution module H2 is C 3×3 convolution operations with a stride of 1; The convolution operation performed by the convolution module H3 is C 3×3 convolution operations with a stride of 1; The convolution operation performed by the convolution module H4 is C 1×1×C convolution operations with a step size of 1.

9. The CT image efficient metal artifact removal system based on multi-domain joint filtering according to claim 2 is characterized in that: When N=10 and the number of feature layers of the feature map output by the input layer is C=16, the N multi-domain Transformer integrated encoding layers in the encoding subnetwork are defined as E1 to E10, and the N / 2-1 down-converters are defined as P1 to P4; the decoding subnetwork includes N multi-domain Transformer integrated decoding layers defined as S1 to S10, and the N / 2-1 up-converters are defined as D1 to D4; The multi-domain Transformer integrated encoding layer E1 is used to perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the C-layer feature map output by the input layer in sequence; The multi-domain Transformer integrated coding layer E2 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E1; The multi-domain Transformer integrated coding layer E3 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E2; The multi-domain Transformer integrated coding layer E4 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E3; The multi-domain Transformer integrated coding layer E5 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E4; The multi-domain Transformer integrated coding layer E6 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E5; The multi-domain Transformer integrated coding layer E7 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E6; The multi-domain Transformer integrated coding layer E8 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E7; The multi-domain Transformer integrated coding layer E9 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E8; The multi-domain Transformer integrated coding layer E10 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E9; The down-converter P1 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated encoding layer E2; The down-converter P2 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated coding layer E4; The down-converter P3 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated coding layer E6; The down-converter P4 is used to perform a 2x down-sampling operation on the output of the multi-domain Transformer integrated coding layer E8; The resolution of the operation object of the multi-domain Transformer integrated encoding layer E1 and E2 is H×W; The operation object resolution of the multi-domain Transformer integrated coding layer E3 and E4 is H / 2×W / 2; The operation object resolution of the multi-domain Transformer integrated coding layers E5 and E6 is H / 4×W / 4; The operation object resolution of the multi-domain Transformer integrated coding layers E7 and E8 is H / 8×W / 8; The operation object resolution of the multi-domain Transformer integrated coding layer E9 and E10 is H / 16×W / 16; the multi-domain Transformer integrated decoding layer S10 is used to perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated coding layer E10 in sequence; The multi-domain Transformer integrated decoding layer S9 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S10; The up-converter D4 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S9; The fourth feature addition module is used to add the output of the up-converter D4 and the output of the multi-domain Transformer integrated encoding layer E8 and send them to the multi-domain Transformer integrated decoding layer S8 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence; The multi-domain Transformer integrated decoding layer S7 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S8; The up-converter D3 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S7; The third feature addition module is used to add the output of the up-converter D3 and the output of the multi-domain Transformer integrated encoding layer E6 and send them to the multi-domain Transformer integrated decoding layer S6 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence; The multi-domain Transformer integrated decoding layer S5 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S6; The up-converter D2 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S5; The second feature summing module is used to sum the output of the up-converter D2 and the output of the multi-domain Transformer integrated encoding layer E4 and send them to the multi-domain Transformer integrated decoding layer S4 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence; The multi-domain Transformer integrated decoding layer S3 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S4; The up-converter D1 is used to perform a 2x up-sampling operation on the output of the multi-domain Transformer integrated decoding layer S3; The first feature summing module is used to sum the output of the up-converter D1 and the output of the multi-domain Transformer integrated encoding layer E2 and send them to the multi-domain Transformer integrated decoding layer S2 to perform multi-domain filtering operations and overall-fine-grained feature extraction operations in sequence; The multi-domain Transformer integrated decoding layer S1 is used to sequentially perform multi-domain filtering operations and overall-fine-grained feature extraction operations on the output of the multi-domain Transformer integrated decoding layer S2; The operation object resolution size of the multi-domain Transformer integrated decoding layer S1 and S2 is H×W; The operation object resolution size of the multi-domain Transformer integrated decoding layer S3 and S4 is H / 2×W / 2; The operation object resolution size of the multi-domain Transformer integrated decoding layer S5 and S6 is H / 4×W / 4; The operation object resolution of the multi-domain Transformer integrated decoding layers S7 and S8 is H / 8×W / 8; The operation object resolution size of the multi-domain Transformer integrated decoding layer S9 and S10 is H / 16×W / 16; The number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E1 and E2, and the multi-domain Transformer integrated decoding layers S1 and S2 is 16, and the number of multi-heads is 2; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E3 and E4, and the multi-domain Transformer integrated decoding layers S3 and S4 is 32, and the number of multi-heads is 4; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E5 and E6, and the multi-domain Transformer integrated decoding layers S5 and S6 is 64, and the number of multi-heads is 8; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E7 and E8, and the multi-domain Transformer integrated decoding layers S7 and S8 is 128, and the number of multi-heads is 16; the number of feature layers of the operation objects of the multi-domain Transformer integrated encoding layers E9 and E10, and the multi-domain Transformer integrated decoding layers S9 and S10 is 256, and the number of multi-heads is 32.

10. The CT image efficient metal artifact removal system based on multi-domain joint filtering according to claim 1, characterized in that: The expression of the loss function Loss of the metal artifact removal system in the training stage is: Where l2||·|| is L2 regularization, I is the high-definition original image, and Y is the input CT image containing metal artifacts. is the mask of the non-metal region, O is the number of samples, PSNR(·) represents the PSNR loss function, represents the Radon transform.

Images