Method and system for removing artifacts of sparse angle CT (Computed Tomography) image

By removing the network based on sparse angle CT image artifacts based on bulk blocks, the problem of artifacts in sparse angle CT images is solved by using three-dimensional sub-blocks and cyclic consistency perception loss function, and efficient and accurate image processing and diagnostic support are achieved.

CN119006315AActive Publication Date: 2024-11-22SHANGHAI JIAOTONG UNIV

Patent Information

Application Number
CN202411161056.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-22
Estimated Expiration
2044-08-22

AI Technical Summary

Technical Problem

The existence of artifacts in sparse angle CT images leads to a decline in image quality and affects diagnostic accuracy. The existing methods are difficult to balance the removal of artifacts and retain image details, and the processing efficiency is low, making it difficult to adapt to large-scale CT data processing.

Method used

The sparse angle CT image artifact is used to remove the network based on bulk blocks. By dividing the CT image into three-dimensional sub-blocks, the adversarial network and cyclic consistency perceptual loss function are used to remove the artifacts and retain important details, improving the processing efficiency and Image quality.

Benefits of technology

It significantly improves the structural consistency and diagnostic value of images on different sections, improves processing efficiency, and can process large-scale CT data without increasing hardware costs, achieving efficient artifact removal and image reconstruction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006315A_ABST
    Figure CN119006315A_ABST
Patent Text Reader

Abstract

The invention relates to the field of medical image processing, and discloses a method and system for removing artifacts of a sparse angle CT image. According to the method, the CT image is processed by adopting the three-dimensional sub-blocks instead of two-dimensional tomography, three-dimensional information is effectively utilized, the discontinuous phenomenon between slices is avoided, the sub-blocks are sampled at random positions in the whole three-dimensional image in each round during training instead of sending the whole three-dimensional image into a network or artificially cutting the sub-blocks, the pressure of a video memory is relieved, and the training efficiency is improved. And the diversity of training samples can be improved. A body block-based network structure is designed, a residual module guided by body attention is included, and the network expression capability and the artifact removal precision are enhanced. The LPIPS measurement is used to calculate the loop consistency loss, which is more in line with a human visual system and avoids generating an over-smooth image. According to the system, the image detail recovery effect is remarkably improved, the coherence of the image on different sections is enhanced, the processing speed is remarkably improved compared with a sectional processing method, and the system is suitable for rapid processing of large-scale CT data and high-quality image reconstruction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of medical image processing, and in particular to a sparse angle computed tomography (CT) image artifact removal technology. Background Art

[0002] Computed tomography (CT) is a widely used medical imaging technology that plays a vital role in disease diagnosis, surgical planning, and treatment evaluation. However, the high-dose radiation used in traditional CT scanning poses a potential hazard to patients. In order to reduce the radiation dose, sparse angle CT technology has gradually attracted attention in recent years. This technology reduces the radiation dose by reducing the projection angle, but it also brings about the problem of image quality degradation, which is mainly manifested in the generation of artifacts.

[0003] Artifacts in sparse angle CT images not only affect the visual quality of the images, but more importantly, may lead to incorrect medical diagnoses. Traditional artifact removal methods are mainly based on image processing techniques such as filtering and iterative reconstruction algorithms. However, these methods often find it difficult to strike a good balance between reducing artifacts and preserving image details. With the development of deep learning technology, neural network-based methods have shown great potential in the field of image processing and have gradually been applied to the task of artifact removal in CT images.

[0004] At present, most deep learning-based CT image artifact removal methods are for processing two-dimensional tomographic images. Although this method can achieve good results in some cases, there are still some problems. For example, CT images are essentially three-dimensional data. Processing only two-dimensional tomography will ignore the spatial correlation between adjacent layers, resulting in the loss of important three-dimensional structural information. For example, since each tomography is processed independently, the reconstructed three-dimensional image may be discontinuous or inconsistent in different sections (such as sagittal and coronal planes), affecting the accuracy of diagnosis. For example, processing each tomography separately requires frequent data input and output, which is inefficient when processing a large number of CT images. It is difficult for two-dimensional processing methods to effectively capture complex three-dimensional artifact patterns, resulting in inconsistent processing effects on CT images at different locations or under different scanning parameters. For example, traditional loss functions (such as mean square error) often result in the generated images being too smooth and lacking in realism, which does not conform to the perceptual characteristics of the human visual system.

[0005] These problems have severely restricted the widespread application of sparse angle CT technology in clinical practice. Therefore, there is an urgent need for a new artifact removal method that can fully utilize the three-dimensional information of CT images, improve spatial consistency, enhance processing efficiency, stabilize image quality and improve perceived quality. This method must not only be able to effectively remove artifacts, but also generate images that are more visually natural and in line with doctors' reading habits while retaining key diagnostic information. At the same time, considering the real-time requirements of the medical field, this method also needs to have the ability to process large-scale CT data to meet the growing clinical needs. Summary of the invention

[0006] The purpose of this application is to provide a method and system for removing artifacts from sparse angle CT images to solve the problems raised in the above background technology.

[0007] The present application discloses a method for removing artifacts from sparse angle CT images, comprising the following steps:

[0008] Acquire a sparse angle CT image and a full angle CT image, and preprocess them to obtain a preprocessed sparse angle CT image and a full angle CT image;

[0009] Dividing the preprocessed sparse angle CT image and full angle CT image into a number of three-dimensional sub-blocks for training; and constructing a volume-based sparse angle CT image artifact removal network, the network comprising two pairs of generators and discriminators, one pair for transforming the three-dimensional sub-blocks for training from the sparse angle image domain to the full angle image domain, and the other pair for performing the reverse process;

[0010] The network is trained using the training three-dimensional sub-blocks, and during the training process, the network is optimized using a loss function including adversarial loss, cycle consistency loss, and reconstruction loss to obtain a trained network;

[0011] After preprocessing the sparse angle CT image to be processed, the image is divided into a number of three-dimensional sub-blocks to be processed that are the same size as the three-dimensional sub-blocks in order of spatial position, and the three-dimensional sub-blocks to be processed are input into the trained network in turn for processing to obtain three-dimensional sub-blocks after artifacts are removed; and the three-dimensional sub-blocks after artifacts are removed are merged to form a CT image after artifacts are removed.

[0012] In a preferred example, the generator of the block-based sparse angle CT image artifact removal network includes an encoder module, a dilated spatial convolutional pooling pyramid module, a multi-body attention-guided residual module, and a decoder module connected in series; wherein:

[0013] The encoder module is composed of a plurality of three-dimensional convolutional layers connected in series, and an instance normalization layer and a ReLU activation function are sequentially added after each of the three-dimensional convolutional layers to extract features of the three-dimensional sub-blocks for training;

[0014] The atrous spatial convolution pooling pyramid module receives the output of the encoder module, and includes a plurality of atrous convolution branches with different atrous rates and a global average pooling branch, and the outputs of the branches are spliced ​​along the channel direction to capture multi-scale features;

[0015] The body attention guided residual module includes a body attention weighted branch, a convolution branch and a residual connection branch, wherein the body attention weighted branch performs pooling on the four dimensions of channel, depth, height and width of the input features respectively, and is used to capture the cross-dimensional correlation between different dimensions after being processed by a convolution layer, an instance normalization layer and a Sigmoid activation function, and multiply the input features; the convolution branch includes multiple three-dimensional convolution layers; the residual connection branch directly transfers the input features;

[0016] The decoder module is composed of a plurality of three-dimensional transposed convolutional layers connected in series, and an instance normalization layer and an activation function are sequentially added after each of the three-dimensional transposed convolutional layers to reconstruct the processed features into an output image of the same size as the input.

[0017] In a preferred example, the encoder module consists of three serially connected three-dimensional convolutional layers, each of which is sequentially connected to an instance normalization layer and a ReLU activation function, wherein the convolution kernel size of the first three-dimensional convolutional layer is 7×7×7, with a step size of 1, and the convolution kernel size of the second and third three-dimensional convolutional layers is 3×3×3, with a step size of 2.

[0018] In a preferred example, the atrous spatial convolution pooling pyramid module includes a 1×1×1 convolution branch, three 3×3×3 atrous convolution branches with different atrous rates and a global average pooling branch, and the outputs of each branch are spliced ​​along the channel direction.

[0019] In a preferred embodiment, the body attention guided residual module includes:

[0020] - The body attention weighted branch pools the four dimensions of channel, depth, height and width of the input features, and then captures the cross-dimensional correlation between each three dimensions through the convolution layer, instance normalization layer and Sigmoid activation function, and then multiplies the input features;

[0021] - Convolutional branch, consisting of two 3×3×3 3D convolutional layers connected in series with a stride of 1;

[0022] -Residual connection branch, used to directly transfer input features.

[0023] In a preferred example, the decoder module consists of three serially connected three-dimensional transposed convolutional layers, wherein the convolution kernel size of the first and second three-dimensional transposed convolutional layers is 3×3×3, with a step size of 2, and each layer is sequentially connected to an instance normalization layer and a ReLU activation function, and the convolution kernel size of the third three-dimensional transposed convolutional layer is 7×7×7, with a step size of 1, and a Tanh activation function is added.

[0024] In a preferred example, the cycle consistency loss in the loss function is calculated using a perceptual metric method to better conform to the characteristics of the human visual system.

[0025] In a preferred example, the perceptual metric method is a learned perceptual patch similarity (LPIPS) metric, which is used to calculate the perceptual difference between the generated image and the original image, and to avoid over-smoothing of the generated image by capturing high-level features.

[0026] In a preferred example, the cycle consistency loss includes:

[0027] After the sparse angle CT image is transformed into the full angle CT image domain by the first generator A, it is transformed back into the sparse angle CT image domain by the second generator B, and the LPIPS metric between the result and the original sparse angle CT image is obtained;

[0028] After the full-angle CT image is transformed into the sparse angle CT image domain through the second generator B, it is then transformed back to the full-angle CT image domain through the first generator A, and the LPIPS metric between the obtained result and the original full-angle CT image is obtained.

[0029] In a preferred example, the calculation formula of the cycle consistency loss is:

[0030] L cycle =E[||B(A(s))-s|| LPIPS ]+E[||A(B(f))-f|| LPIPS ] (5)

[0031] L cycle represents cycle consistency loss;

[0032] E[·] represents the expectation operation;

[0033] ||·|| LPIPS represents the distance calculated using the LPIPS (Learning Perceptual Patch Similarity) metric;

[0034] s represents sparse angle CT image;

[0035] f represents the full-angle CT image;

[0036] A(·) represents the first generator that transforms the sparse angle CT image into the full angle CT image domain;

[0037] B(·) represents the second generator that transforms the full-angle CT image into the sparse-angle CT image domain;

[0038] B(A(s)) represents the result of transforming the sparse angular CT image s into the full-angle CT image domain through generator A and then transforming it back into the sparse angular CT image domain through generator B;

[0039] A(B(f)) represents the result of transforming the full-angle CT image f into the sparse angle CT image domain through generator B, and then transforming it back to the full-angle CT image domain through generator A.

[0040] In a preferred example, the loss function also includes a reconstruction loss, which is used to calculate the perceptual difference between the generated image and the true image, and guide the network to optimize the image details.

[0041] In a preferred example, the calculation formula of the reconstruction loss is:

[0042] L rec =E[||A(s)-f|| LPIPS ]+E[||B(f)-s|| LPIPS ] (6)

[0043] in,

[0044] L rec represents reconstruction losses;

[0045] E[·] represents the expectation operation;

[0046] ||·|| LPIPS represents the distance calculated using the LPIPS (Learning Perceptual Patch Similarity) metric;

[0047] s represents sparse angle CT image;

[0048] f represents the full-angle CT image;

[0049] A(·) represents the generator that transforms the sparse angle CT image to the full angle CT image domain;

[0050] B(·) represents the generator that transforms the full-angle CT image into the sparse-angle CT image domain;

[0051] A(s) represents the result of transforming the sparse angular CT image s to the full-angle CT image domain through the generator A;

[0052] B(f) represents the result of transforming the full-angle CT image f into the sparse angular CT image domain through the generator B.

[0053] In a preferred example, the overall loss of the loss function is the weighted sum of adversarial loss, cycle consistency loss and reconstruction loss, and the calculation formula is:

[0054] L total =L GAN (A,D F )+L GAN (B,D S ) +λ 1 L cycle +λ 2 L rec (7)

[0055] in,

[0056] L total Indicates overall loss;

[0057] L GAN (A,D F ) represents the generator A and the discriminator D F The adversarial loss between them is used to optimize the image conversion from sparse angles to full angles;

[0058] L GAN (B,D S ) represents the generator B and the discriminator D S The adversarial loss between them is used to optimize the image conversion from full angle to sparse angle;

[0059] L cycle represents cycle consistency loss;

[0060] L rec represents reconstruction losses;

[0061] λ 1 and λ 2 is the weight coefficient, which is used to balance the contribution of different loss terms;

[0062] A represents the generator that transforms the sparse angle CT image into the full angle CT image domain;

[0063] B represents the generator that transforms the full-angle CT image into the sparse-angle CT image domain;

[0064] D F The discriminator representing the full-angle image domain is used to distinguish the real full-angle CT image from the image generated by generator A;

[0065] D S The discriminator represents the sparse angular image domain, which is used to distinguish between real sparse angular CT images and images generated by generator B.

[0066] The present application also discloses a device for removing artifacts from sparse angle CT images, comprising:

[0067] A data preprocessing module is used to obtain a sparse angle CT image and a full angle CT image, and preprocess them to obtain a preprocessed sparse angle CT image and a full angle CT image;

[0068] A network construction module is used to divide the preprocessed sparse angle CT image and full angle CT image into a number of training three-dimensional sub-blocks; and to construct a volume-based sparse angle CT image artifact removal network, the network comprising two pairs of generators and discriminators, one pair of which is used to transform the training three-dimensional sub-blocks from the sparse angle image domain to the full angle image domain, and the other pair is used to perform the reverse process;

[0069] A network training module, used to train the network using the training three-dimensional sub-blocks, and during the training process, the network is optimized using a loss function including adversarial loss, cycle consistency loss and reconstruction loss to obtain a trained network;

[0070] An image processing module, used for pre-processing the sparse angle CT image to be processed, dividing it into a number of to-be-processed three-dimensional sub-blocks of the same size as the three-dimensional sub-blocks in order of spatial position, and sequentially inputting the to-be-processed three-dimensional sub-blocks into the trained network for processing to obtain the three-dimensional sub-blocks after removing artifacts;

[0071] The image synthesis module is used to merge the three-dimensional sub-blocks after the artifacts are removed to form a CT image after the artifacts are removed.

[0072] The implementation methods of this application have the following technical effects:

[0073] This application proposes an innovative method for removing artifacts from sparse angle CT images, the core of which is to make full use of the three-dimensional information of CT images and advanced deep learning technology. The method mainly contains three key points, each of which brings significant technical effects.

[0074] First, the present application uses three-dimensional sub-blocks rather than two-dimensional slices to perform artifact removal on sparse angle CT images. This innovation makes full use of the three-dimensional information of CT images. Traditional methods usually treat CT images as a series of independent two-dimensional slices for processing, ignoring the spatial correlation between slices. However, the present application effectively captures the structural information of CT images in all directions by processing three-dimensional sub-blocks containing multiple adjacent slices. The technical effect brought about by this method is significant: the processed images show better structural coherence in the cross-section, sagittal and coronal planes, greatly improving the restoration of image details. Especially in the sagittal and coronal planes, this method effectively avoids the discontinuity phenomenon common in traditional methods, making the reconstructed three-dimensional images more realistic and reliable. In addition, by considering the information of multiple adjacent slices at the same time, this method can more accurately distinguish between the real structure and artifacts in the image, thereby achieving more accurate artifact removal.

[0075] Secondly, this application specially designs a block-based network structure for the task of artifact removal in sparse angle CT images. The core of this network lies in its volume attention-guided residual module, which can effectively capture the cross-dimensional correlation between different dimensions and significantly enhance the network's expressive power. Specifically, this module processes the channel, depth, height, and width dimensions through four branches, respectively, capturing the complex interactions between them. The technical effects brought about by this design are multifaceted: first, it greatly improves the network's understanding and processing capabilities of three-dimensional structural information, enabling the network to more accurately identify and remove complex artifact patterns; secondly, the network's multi-branch structure and residual connection not only enhance the feature extraction capability, but also make the training process more stable and efficient by allowing the direct transfer of gradients; finally, this design enables the network to show stronger adaptability when processing CT images of different parts or different scanning parameters.

[0076] Finally, this application uses the LPIPS metric when calculating the cycle consistency loss and proposes the concept of cycle consistency perceptual loss. This loss function is more in line with the characteristics of the human visual system and effectively avoids the generation of overly smoothed images. Traditional pixel-level loss functions often lead to the loss of image details, while the LPIPS metric can better retain the texture and detail information of the image by considering the similarity of high-level features. The technical effects brought about by this innovation are significant: first, the processed images are more natural and realistic visually, with richer details, which is crucial for medical diagnosis; second, this method achieves a better balance between removing artifacts and retaining important anatomical details, improving the clinical practicality of the processing results; finally, because it is more in line with human visual perception, the images processed by this method are easier for doctors to accept and interpret. In addition, this application significantly optimizes the processing speed by processing in units of three-dimensional sub-blocks. The two-dimensional processing method requires frequent loading and processing of each tomographic image, while this application reduces the frequency of data loading by processing three-dimensional sub-blocks containing multiple adjacent slices. Especially in large-scale CT data processing scenarios, this improvement greatly improves processing efficiency. At the same time, this application takes into account the pressure of processing three-dimensional images on video memory, and processes three-dimensional sub-blocks instead of the entire three-dimensional image, making it possible to process larger-sized three-dimensional data under the same hardware conditions, effectively alleviating the video memory pressure in high-resolution CT image processing. In addition, the strategy of sampling three-dimensional sub-blocks during training in this application is to sample sub-blocks at random positions in the entire three-dimensional image each round, rather than artificially dividing sub-blocks when making data sets, which can increase the diversity of training samples and enable the network to learn more information. In summary, especially in large-scale CT data processing scenarios, these improvements have greatly improved processing efficiency and processing capabilities, making it possible to achieve efficient CT image artifact removal without increasing hardware costs.

[0077] In summary, these three key points not only bring significant technical progress individually, but the synergy between them produces a comprehensive effect that goes beyond the individual technical points. The 3D sub-block processing method provides the network with rich spatial information, and the specially designed network structure fully utilizes this information through its volume attention-guided residual module to accurately capture the correlation between different dimensions. This structure complements the cycle consistency perception loss, which together ensures that subtle anatomical details are preserved while removing artifacts. In particular, the LPIPS metric takes into account human visual perception when evaluating image quality, and combined with the 3D sub-block processing method, it maintains a high degree of perceptual consistency throughout the 3D space.

[0078] The effect of this synergy is significant: for a CT image containing 150 slices, the processing time is reduced from 18 seconds (U-former is used as an example) to only 3 seconds, which is a 6-fold increase in efficiency, while achieving unprecedented image quality improvement. This combination of high efficiency and high quality makes this method of great application value in clinical and research scenarios where a large number of sparse angle CT images need to be processed quickly. More notably, the advantages of this method will expand exponentially as the amount of data to be processed and the number of slices increase. This is because the three-dimensional processing method reduces the need to switch between different dimensions, the efficient design of the network structure reduces the computational complexity, and the use of perceptual loss ensures that excellent image quality can be maintained even in high-speed processing. This scalability makes this method an ideal choice for large-scale CT image data processing, bringing revolutionary progress to the field of medical imaging.

[0079] A large number of technical features are recorded in the specification of this application, which are distributed in various technical solutions. If all possible combinations of technical features of this application (i.e., technical solutions) are to be listed, the specification will be too long. In order to avoid this problem, the various technical features disclosed in the above-mentioned invention content of this application, the various technical features disclosed in the various embodiments and examples below, and the various technical features disclosed in the accompanying drawings can be freely combined with each other to form various new technical solutions (these technical solutions are all deemed to have been recorded in this specification), unless the combination of such technical features is technically infeasible. For example, in one example, feature A+B+C is disclosed, and in another example, feature A+B+D+E is disclosed, and features C and D are equivalent technical means that play the same role. Technically, only one can be used, and it is impossible to use them at the same time. Feature E can be combined with feature C technically. Then, the solution of A+B+C+D should not be deemed to have been recorded because it is technically infeasible, and the solution of A+B+C+E should be deemed to have been recorded. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 It is a schematic diagram of the overall process of the method for removing artifacts from sparse angle CT images according to the present application.

[0081] Figure 2 It is a structural diagram of a generator of a volume block-based sparse angle CT image artifact removal network in a method for removing artifacts from sparse angle CT images according to the first embodiment of the present application.

[0082] Figure 3 It is a schematic diagram of the training process and loss function composition of the block-based sparse angle CT image artifact removal network in the method for sparse angle CT image artifact removal according to the first embodiment of the present application.

[0083] Figure 4 It is a schematic diagram of the effect of the method for removing artifacts from sparse angle CT images according to the first embodiment of the present application.

[0084] Figure 5 It is a flowchart of a method for removing artifacts from sparse angle CT images according to the first embodiment of the present application.

[0085] Figure 6 It is a schematic structural diagram of a system for removing artifacts from sparse angle CT images according to the second embodiment of the present application. DETAILED DESCRIPTION

[0086] In the following description, many technical details are provided to help readers better understand the present application. However, those skilled in the art can understand that the technical solution claimed in the present application can be implemented even without these technical details and various changes and modifications based on the following embodiments.

[0087] Description of some concepts:

[0088] Sparse view: Sparse view refers to a technique that reduces radiation dose by reducing the number of projection angles in computed tomography (CT) imaging. Compared with traditional full-angle scanning, sparse view imaging acquires less projection data, so it is easy to produce artifacts or noise in the image. This technology is often used in scenarios where it is necessary to reduce patient exposure to X-ray radiation, such as imaging children or patients who need frequent scans. The number of sparse view projection angles is usually less than one-third of the number of full-angle projection angles. When the number of angles is less than 60, it is an extremely sparse view.

[0089] Full view: Full view refers to the use of a full range of projection angles for scanning in CT imaging, which usually covers a larger number of projection angles to provide more comprehensive image data and higher image quality. The data generated by full-angle scanning is more complete, so it can reduce the appearance of artifacts and provide clearer images, which is suitable for medical diagnosis that requires high precision and detailed anatomical information.

[0090] Artifacts: False images or erroneous information in CT images due to incomplete data or problems with the reconstruction algorithm, which may interfere with medical diagnosis.

[0091] Three-dimensional sub-block: The CT image is divided into three-dimensional blocks containing multiple adjacent slices for processing to make full use of the three-dimensional information of the image and improve the effect of artifact removal and image quality.

[0092] Two-dimensional tomography: Traditional CT image processing methods treat CT images as independent two-dimensional slices and may ignore the spatial correlation between slices.

[0093] Body-attention-guided residual module: A module that introduces attention mechanism and residual connection into the network structure can capture cross-dimensional correlations between different dimensions and enhance the network's expressiveness and training stability.

[0094] LPIPS metric: Learned Perceptual Image PatchSimilarity, a deep learning-based image similarity metric that is more in line with the human visual system and avoids generating overly smooth images.

[0095] Cycle consistency loss: In deep learning, the difference between an image and two opposite transformations is measured to ensure the reversibility of the transformation process and the consistency of the image content.

[0096] Video memory pressure: refers to the usage of the graphics card memory (VRAM) during image processing. By dividing the 3D image into sub-blocks for processing, the video memory pressure can be reduced, allowing the graphics card to process larger data sizes.

[0097] Deep learning: An artificial intelligence technology that simulates the structure and function of the human brain's neural network to perform data processing and pattern recognition. It is widely used in fields such as image processing and natural language processing.

[0098] CT (Computed Tomography): Computer tomography generates three-dimensional images of internal structures by processing X-ray projections from multiple angles with a computer. It is widely used in medical diagnosis and industrial testing.

[0099] Generative Adversarial Network (GAN): A deep learning model architecture that consists of two adversarial neural networks, a generator and a discriminator. In this method, two pairs of GANs are used to achieve bidirectional conversion between sparse angle CT images and full angle CT images.

[0100] The following is a brief description of some of the innovative features of this application:

[0101] First, this application has a wide range of application fields and scenarios in medical image processing, especially in the removal of artifacts and improvement of image quality of CT images. For example:

[0102] In emergency medicine, the application of this technology is particularly significant. Rapid assessment of trauma patients requires rapid CT scanning, and the use of sparse angle CT can speed up the scanning, but it may produce artifacts. This technology removes artifacts and ensures that the image quality is high enough to support emergency medical decision-making, helping doctors quickly identify internal injuries and develop treatment plans, thereby improving the efficiency and effectiveness of treatment.

[0103] Some medical institutions may use older CT equipment, which may not produce images of the same quality as the latest equipment. Through this technology, the relatively low-quality images produced by old equipment can be improved to a level close to that of new equipment, thereby extending the service life of old equipment and saving medical resources. This is especially important for hospitals and clinics with limited budgets, which can improve the quality of medical services without replacing equipment.

[0104] Reducing radiation dose is particularly important for some special patient groups, such as children, pregnant women, or patients with chronic diseases who need frequent CT examinations. This technology can provide high-quality images based on low-dose scans, which is particularly suitable for these patient groups. By reducing radiation exposure, it protects patient health while ensuring diagnostic accuracy.

[0105] CT scanning is one of the important tools in tumor detection and monitoring. Using the technical solution of the present application, high-quality CT images can be obtained while reducing radiation dose, which is helpful for early detection and follow-up monitoring of tumors and ensures timely and accurate treatment.

[0106] CT imaging plays a key role in the diagnosis and evaluation of cardiovascular diseases, such as coronary artery disease, aneurysms, and abnormal cardiac structures. The technical solution of this application can provide clear images of the heart and blood vessels, helping doctors assess the condition and formulate treatment plans, improving the accuracy of diagnosis and treatment effects.

[0107] Low-dose CT scans are widely used for early screening of lung cancer. By applying the technical solution of the present application, the image quality can be improved while reducing the radiation dose, thereby more effectively screening lung diseases, which is particularly important in the early detection of lung cancer.

[0108] In complex surgical operations, accurate preoperative planning and intraoperative navigation are crucial. High-quality CT images can help doctors understand the patient's anatomical structure in detail and develop the best surgical plan. Using the technical solution of this application, clearer and more accurate CT images can be obtained to assist doctors in preoperative planning and intraoperative navigation, reduce surgical risks, and improve surgical success rates.

[0109] The rise of telemedicine has enabled patients in remote areas to enjoy high-quality medical services. The transmission and remote diagnosis of CT images are important components of telemedicine. The technical solution of this application can effectively remove artifacts in sparse angle CT images, improve image quality, and ensure the accuracy and reliability of remote diagnosis.

[0110] Medical researchers and educators need a large number of high-quality CT images for research and teaching. The technical solution of this application can provide clear and accurate image data to support the work of researchers and educators, thereby promoting the development of medical research and improving the quality of teaching.

[0111] Pediatric patients are highly sensitive to radiation, so the radiation dose needs to be minimized. The technical solution of this application can provide high-quality CT images under low-dose scanning, protect children's health, and ensure the accuracy of diagnosis.

[0112] In emergency trauma assessment, CT scans are a key tool for quickly assessing internal injuries. High-quality images help doctors make quick diagnosis and treatment decisions, improving the success rate of treating trauma patients.

[0113] The above-mentioned application fields and scenarios exemplarily demonstrate the wide and important application of the technical solution of the present application in actual medical image processing, providing important support for improving the quality of diagnosis and treatment.

[0114] According to the technical concept of this application, a block-based sparse angle CT image artifact removal network is proposed. The overall process is shown in Figure 1 The network is trained using sparse angular CT images and full-angle CT images, and adopts a composite loss function consisting of adversarial loss, cycle consistency-aware loss, and reconstruction loss.

[0115] Considering that CT images are essentially three-dimensional, there are content similarities and structural continuity between adjacent layers, which help the network distinguish structural details and artifacts in the image. Therefore, the present application is no longer limited to the processing of a single fault, but divides the CT image into three-dimensional sub-blocks and inputs them into the network for processing. This method can effectively retain the information in the three-dimensional structure of the CT image, thereby helping the network to more accurately distinguish image details and artifacts.

[0116] The artifact removal network consists of two pairs of generators and discriminators with the same structure but non-shared weights. This design is conducive to ensuring the reversibility of the image transformation process and the consistency of the content before and after the transformation, while also ensuring the image fidelity and the stability of the training process. Among them, one pair of generators and discriminators is responsible for transforming the image from the sparse angular image domain (S) to the full angular image domain (F), and the other pair performs the opposite process.

[0117] The structure of the generator is as follows Figure 2 As shown in Figure 1, it contains four key modules: encoder module, dilated spatial convolutional pooling pyramid module, volume attention guided residual module and decoder module. Figure 2 (c) uses dilated convolutions with different dilation rates to capture multi-scale image features, so that the network has receptive fields of different sizes, thereby improving feature extraction capabilities and promoting the network's comprehensive understanding of the image. Figure 2 (B) contains four branches that capture the dimensional interaction weights between each combination of three dimensions: channel, depth, height, and width, and perform an average operation. In addition to the attention weighted operation, the input tensor is also passed through a convolution branch for feature extraction and directly passed through a residual connection. This multi-branch structure not only improves the expressiveness of the network, but also improves the stability of training through the direct transfer of gradients.

[0118] The network is optimized using a composite loss function consisting of adversarial loss, cycle consistency perceptual loss, and reconstruction loss. In theory, a CT image should be restored to its original state after two opposite transformations. Taking a sparse angular CT image sub-block s as an example, s is first transformed from the S domain to the F domain through generator A to obtain A(s), and then A(s) is transformed back to the S domain through generator B to obtain B(A(s)). The cycle consistency perceptual loss is used to measure the difference between B(A(s)) and the original image s to ensure the reversibility of the transformation process and the consistency of the image content. In addition, the intermediate result A(s) is compared with the corresponding full-angle CT image label to calculate the reconstruction loss. All these loss calculations use the learning perceptual image patch similarity (LPIPS) to measure the difference between images.

[0119] The calculation formulas for each part of the loss function are as follows:

[0120] L GAN (A,D F )=E[log D F (f)]+E[log(1-D F (A(s)))] (3)

[0121] L GAN (B,D S )=E[log D S (s)]+E[log(1-D S (B(f)))] (4)

[0122] L cycle =E[||B(A(s))-s|| LPIPS ]+E[||A(B(f))-f|| LPIPS ] (5)

[0123] L rec =E[||A(s)-f|| LPIPS ]+E[||B(f)-s|| LPIPS ] (6)

[0124] In these formulas, L GAN (A,D F ) and L GAN (B,D S )) constitutes the adversarial loss, A and B are the generators of S→F and F→S respectively, and D F and D S is the corresponding discriminator. s and f are samples from the S and F domains, respectively. L cycle Calculate the cycle consistency aware loss, L rec Calculate the reconstruction loss,||·|| LPIPS Represents the LPIPS metric.

[0125] L total =L GAN (A,D F )+L GAN (B,D S ) +λ 1 L cycle +λ 2 L rec (7)

[0126] During the network training process, each iteration will sample a fixed-size image sub-block at a random position as input. After the training is completed, the sparse angle CT image to be processed is divided into sub-blocks of the same size as the sampling size during training according to the spatial position order, and then input into the generator A for processing in turn. Finally, the processed sub-blocks are merged to form the final optimization result.

[0127] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0128] The first embodiment of the present application relates to a method for removing artifacts from sparse angle CT images, the process of which is as follows: Figure 5 As shown, the method comprises the following steps:

[0129] Step 10: Acquire a sparse angle CT image and a full angle CT image, and preprocess them to obtain a preprocessed sparse angle CT image and a full angle CT image;

[0130] Step 20: Divide the preprocessed sparse angle CT image and full angle CT image into a number of training three-dimensional sub-blocks; and construct a block-based sparse angle CT image artifact removal network, the network comprising two pairs of generators and discriminators, one pair is used to transform the training three-dimensional sub-blocks from the sparse angle image domain to the full angle image domain, and the other pair is used to perform the reverse process;

[0131] Step 30: using the training three-dimensional sub-block to train the network, during the training process, optimizing the network using adversarial loss, cycle consistency loss and reconstruction loss to obtain a trained network;

[0132] Step 40: After preprocessing the sparse angle CT image to be processed, divide it into a number of three-dimensional sub-blocks to be processed with the same size as the three-dimensional sub-blocks in order of spatial position, input the three-dimensional sub-blocks to be processed into the trained network in turn for processing to obtain three-dimensional sub-blocks after artifacts are removed; and merge the three-dimensional sub-blocks after artifacts are removed to form a CT image after artifacts are removed.

[0133] Each step is further described below.

[0134] In step 10, first, a sparse angle CT image and a full angle CT image are acquired using a CT scanning device. The sparse angle CT image reduces the radiation dose by reducing the projection angle, thereby reducing the potential harm to the patient. However, this method may cause artifacts in the image. The full angle CT image is acquired using a traditional full angle scanning method, which has higher image quality and integrity.

[0135] Next, the acquired sparse angle CT images and full angle CT images are preprocessed. The preprocessing steps include resampling, size clipping, and grayscale value normalization. Resampling is used to adjust the spatial resolution of the CT image to ensure that different images have the same resolution. This step is crucial because it ensures that the spatial features of the image can be processed and analyzed consistently in subsequent processing. Size clipping is to clip the CT image to a specific size to meet the input requirements of the network. Usually, the size of the CT image may be too large or irregular. By clipping, the image can be adjusted to a uniform size, thereby improving the efficiency and effect of the processing. Grayscale value normalization is to standardize the grayscale value range of the image, usually adjusted to [-1,1] or [0,1]. The grayscale value of the CT image represents the density of different tissues. By normalization, the grayscale values ​​of different images can be adjusted to the same range. This step improves the operability and stability of the image data in the network and avoids training instability caused by different grayscale value ranges.

[0136] For example, assume that the CT images processed in this step are scanned images of walnuts. The sparse angle CT image may be acquired through 40 projection angles, while the full angle CT image is acquired through 1200 projection angles. The size of the original image is (512,512,512), and this resolution is maintained during the resampling process. Then, the image is cropped to a size of (360,360,150) to ensure that all images are of the same size. Next, grayscale normalization is performed to adjust the grayscale value to the range of [-1,1].

[0137] Through these preprocessing steps, the consistency and adaptability of sparse angle and full angle CT images in subsequent processing and analysis are ensured, providing a solid foundation for subsequent artifact removal and image reconstruction. These technical means are combined to form a complete preprocessing process aimed at improving the processing efficiency and quality of CT images.

[0138] In step 20, first, the preprocessed sparse angle CT image and full angle CT image are divided into a number of 3D sub-blocks for training. This process involves dividing the CT image into small blocks according to spatial position, each of which contains multiple adjacent slices, thereby forming a 3D data block. This division method ensures that each sub-block not only contains the information of a single slice, but also captures the spatial correlation between adjacent slices, providing rich 3D structural information for subsequent network training.

[0139] Next, a volume-based sparse angle CT image artifact removal network is constructed. The core of the network is two pairs of generators and discriminators, which are designed to achieve bidirectional conversion between image domains. Specifically, one pair of generators and discriminators is responsible for converting the training 3D sub-blocks from the sparse angle image domain to the full angle image domain, and the other pair performs the opposite process, that is, converting the 3D sub-blocks from the full angle image domain back to the sparse angle image domain. The role of the generator is to generate images in the target domain, while the discriminator is used to distinguish the generated images from the real images.

[0140] This bidirectional conversion design enables the network to learn and understand the essential differences between sparse angle and full angle CT images, thereby gradually optimizing the image conversion effect during the training process. By learning the features of these two image domains at the same time, the network can more effectively remove artifacts and retain important details in the image.

[0141] In summary, this step provides a technical basis for removing artifacts and improving image quality by dividing the CT image into 3D sub-blocks and utilizing a bidirectional generative adversarial network to achieve the conversion between sparse angle and full angle image domains.

[0142] In general, step 20 demonstrates the innovative thinking of this method in data processing and network architecture design. By combining 3D data processing and advanced GAN architecture, this method not only improves the effect of artifact removal, but also enhances the learning ability and generalization of the network, providing a new paradigm in the field of CT image reconstruction.

[0143] In step 30, first, the constructed network is trained using the three-dimensional sub-blocks for training divided in step 20. During the training process, these three-dimensional sub-blocks are used as inputs and input into the network in sequence. In order to optimize the network performance and ensure that it can effectively remove artifacts and retain image details, three main loss functions are used in the training process: adversarial loss, cycle consistency loss, and reconstruction loss.

[0144] Adversarial loss comes from the training idea of ​​Generative Adversarial Network (GAN), which improves the quality of generated images through adversarial training of generator and discriminator. Specifically, the generator tries to generate realistic CT images, while the discriminator tries to distinguish the generated images from the real ones. This process is continuously iterated, making the images generated by the generator more and more realistic, thereby improving the network's ability to remove artifacts.

[0145] The cycle consistency loss ensures that the conversion of images between the sparse angle and full angle image domains is reversible, that is, after a sparse angle CT image is converted to a full angle CT image through a generator, and then converted back to a sparse angle CT image through another generator, the resulting image should be as consistent as possible with the original sparse angle CT image. Similarly, after a full angle CT image is converted to a sparse angle CT image through a generator, and then converted back to a full angle CT image, the resulting image should be as consistent as possible with the original full angle CT image. This loss function ensures the stability and consistency of the conversion process by measuring the conversion difference between the two domains.

[0146] The reconstruction loss is used to measure the similarity between the generated image and the target image in terms of overall structure. It ensures that the network retains the details and structural information of the original image as much as possible while generating high-quality images. The reconstruction loss usually uses advanced similarity measurement methods rather than simple pixel-level differences, which can better preserve the detailed features of the image.

[0147] By comprehensively applying these three loss functions, the network is optimized and its ability to remove artifacts and reconstruct images is gradually improved. After multiple iterations of training, a trained network is finally obtained, which can generate high-quality artifact-free CT images when new sparse angle CT images are input.

[0148] It should be pointed out that in this step, the use of training 3D sub-blocks for network training is an innovation. This method not only utilizes the 3D characteristics of CT images, but also cleverly solves the problem of computing resources for large-scale 3D data training. By randomly sampling 3D sub-blocks instead of using the entire CT volume, this method achieves the following goals: Increase the diversity of training samples: Random sampling can generate a large number of different 3D sub-blocks, greatly increasing the diversity of training data and helping to improve the generalization ability of the network. Balance computational efficiency and learning effect: Using sub-blocks instead of the entire CT volume significantly reduces the amount of computation for each iteration, allowing effective training even with limited computing resources. Implicit data enhancement: Different sub-blocks may contain artifacts and anatomical structures to varying degrees. This diversity itself is an implicit data enhancement that helps the network learn more robust features.

[0149] Secondly, the method adopts a multi-component composite loss function, which is another key innovation of the training process. This loss function contains three main parts: adversarial loss, cycle consistency loss and reconstruction loss. Specifically, adversarial loss: This is derived from the idea of ​​generative adversarial network (GAN). Through adversarial training of generator and discriminator, the network is able to generate images that are more realistic and more consistent with the distribution of the target domain. In CT image reconstruction, this means that the generated full-angle CT images should not only be free of artifacts, but also maintain the texture and detail characteristics of the real CT images. Cycle consistency loss: This is a unique innovation of the method. By ensuring that the transformation of images between the two domains is reversible (sparse angle → full angle → sparse angle, and the reverse process), the network is forced to learn to preserve the essential characteristics of the image. This not only improves the accuracy of reconstruction, but also helps to preserve those subtle but important structural details that may be mistakenly removed during the artifact removal process. Reconstruction loss: This ensures that the generated image is similar to the target image in terms of overall structure. It is worth noting that the method may adopt advanced similarity metrics (such as perceptual loss) instead of simple pixel-level differences, which is more in line with the characteristics of the human visual system.

[0150] The combination of these three losses forms a multi-objective optimization problem, where each loss targets a different aspect of the reconstruction process. This complex optimization strategy requires the network to find a balance between multiple constraints to produce high-quality, realistic, and detail-preserving reconstructions.

[0151] In step 40, first, the sparse angle CT image to be processed is preprocessed, which includes adjusting the spatial resolution, size cropping and gray value normalization of the image to ensure the consistency and adaptability of the image. The preprocessed image will be segmented into a number of to-be-processed three-dimensional sub-blocks of the same size as the training three-dimensional sub-blocks in order of spatial position. This segmentation process divides the entire CT image into multiple small three-dimensional blocks, each block contains multiple adjacent slices, and retains the three-dimensional structural information of the image.

[0152] Next, these 3D sub-blocks to be processed are sequentially input into the trained network for processing. The network will perform artifact removal on each input 3D sub-block to generate an artifact-free 3D sub-block. This step utilizes the previously trained network model to ensure that each sub-block retains its important structural information while removing artifacts.

[0153] Finally, all the artifact-free 3D sub-blocks are merged to form a complete artifact-free CT image. This merging process requires the processed sub-blocks to be reassembled according to their original spatial positions to restore a complete 3D CT image. This method ensures the coherence and integrity of the entire CT image after processing, providing technical support for the final image reconstruction.

[0154] Step 40 can solve the challenge of large-scale 3D data processing while ensuring the high quality and consistency of the processing results. The innovation of this step is reflected in the following aspects:

[0155] First, the sparse angle CT image to be processed is divided into three-dimensional sub-blocks in order of spatial position. This processing method reflects a deep understanding of the 3D data structure. Unlike random sampling in the training phase, ordered segmentation is used here to ensure that the entire CT volume is processed completely and in an orderly manner. This method has several significant advantages: Maintaining spatial continuity: Ordered segmentation ensures that the spatial relationship between adjacent sub-blocks is maintained, which is crucial for maintaining the coherence of the overall image. Adapting to large-scale data: Through block processing, this method cleverly overcomes GPU memory limitations, allowing even large CT data sets to be processed efficiently. Parallel processing potential: This block strategy provides the possibility for parallel computing, which can significantly improve processing speed.

[0156] Secondly, the 3D sub-blocks to be processed are input into the trained network for processing in turn. This process reflects the flexibility and adaptability of the method: Consistent processing: Each sub-block is processed by the same network, ensuring the consistency of the entire CT volume processing. Local-global balance: When processing each local sub-block, the network actually processes it based on the knowledge learned in a larger spatial context, which helps to maintain the consistency of the global structure while removing local artifacts. Dynamic adaptation: Although the network is trained on fixed-size sub-blocks, this processing method allows the network to dynamically adapt to CT data of different sizes and shapes.

[0157] Finally, the artifact-free 3D sub-blocks are merged to form the final artifact-free CT image.

[0158] In general, according to the above process of this embodiment, first, the method uses three-dimensional sub-blocks as processing units instead of traditional two-dimensional slices. This method deeply recognizes that CT images are essentially three-dimensional data, and there is rich spatial correlation information between adjacent slices. By processing multiple adjacent slices at the same time, the network can capture complex three-dimensional structural features, which is crucial for accurately distinguishing real anatomical structures from artifacts. This three-dimensional processing strategy not only improves the accuracy of artifact removal, but also significantly improves the coherence of reconstructed images on different slices, solving the common problem of fault discontinuity in traditional methods.

[0159] Secondly, this method designs a unique bidirectional conversion mechanism, which includes two pairs of generators and discriminators. This design is inspired by the field of image style transfer, but its application in CT image processing is innovative. By simultaneously learning the forward mapping from sparse angles to full angles and the reverse mapping from full angles to sparse angles, the network can more comprehensively understand the relationship between the two image domains. This bidirectional learning not only enhances the robustness of the network, but also lays the foundation for subsequent cycle consistency constraints.

[0160] The composite loss function used in the method is another highlight. In particular, the perception-based cycle consistency loss is introduced, and the LPIPS (Learned Perceptual Image Patch Similarity) metric is used to evaluate image similarity. This method goes beyond the traditional pixel-level comparison and is more in line with the perceptual characteristics of the human visual system. By considering the similarity of high-level features, the method is able to remove artifacts while retaining subtle texture details, avoiding the problem of over-smoothing, which is crucial for medical diagnosis.

[0161] Finally, the method adopts a clever sliding window strategy in practical application. By segmenting large CT datasets into overlapping 3D sub-blocks, and then processing and reassembling them block by block, the method effectively balances computational efficiency and processing quality. This strategy not only solves the GPU memory limitation problem, but also ensures the continuity of processing results through smooth transition of overlapping areas, providing a feasible solution for efficient processing of large-scale CT data.

[0162] This approach forms a unique and powerful framework by fusing 3D processing, bidirectional learning, perceptual loss, and an efficient implementation strategy, opening up new avenues for high-quality reconstruction of sparse angular CT images.

[0163] Volume-based Artifact Removal Network for Sparse Angular CT Images

[0164] The specific structure of the block-based sparse angular CT image artifact removal network is further described in detail below.

[0165] Optionally, the generator of the volume-based sparse angular CT image artifact removal network includes an encoder module, a dilated spatial convolution pooling pyramid module, a plurality of volume attention-guided residual modules and a decoder module connected in series in sequence; wherein: the encoder module is composed of a plurality of three-dimensional convolution layers connected in series, and an instance normalization layer and a ReLU activation function are sequentially added after each of the three-dimensional convolution layers to extract the features of the training three-dimensional sub-blocks; the dilated spatial convolution pooling pyramid module receives the output of the encoder module, and includes a plurality of dilated convolution branches with different dilation rates and a global average pooling branch, and the outputs of the branches are spliced ​​along the channel direction to capture multi-scale features; the volume attention The force-guided residual module includes a body attention weighted branch, a convolution branch and a residual connection branch, wherein the body attention weighted branch pools the four dimensions of channel, depth, height and width of the input features respectively, and is used to capture the cross-dimensional correlation between different dimensions after being processed by a convolution layer, an instance normalization layer and a Sigmoid activation function, and multiply the input features; the convolution branch includes multiple three-dimensional convolution layers; the residual connection branch directly transfers the input features; the decoder module is composed of multiple three-dimensional transposed convolution layers in series, and an instance normalization layer and an activation function are added in sequence after each of the three-dimensional transposed convolution layers to reconstruct the processed features into an output image with the same size as the input.

[0166] Optionally, the encoder module consists of three serially connected three-dimensional convolutional layers, each of which is sequentially connected to an instance normalization layer and a ReLU activation function, wherein the convolution kernel size of the first three-dimensional convolutional layer is 7×7×7, with a step size of 1, and the convolution kernel sizes of the second and third three-dimensional convolutional layers are 3×3×3, with a step size of 2.

[0167] Optionally, the atrous spatial convolution pooling pyramid module includes a 1×1×1 convolution branch, three 3×3×3 atrous convolution branches with different atrous rates and a global average pooling branch, and the outputs of each branch are spliced ​​along the channel direction.

[0168] Optionally, the body attention guided residual module includes: a body attention weighted branch, which pools the four dimensions of channel, depth, height and width of the input features, captures the cross-dimensional correlation between each three dimensions through a convolution layer, an instance normalization layer and a Sigmoid activation function, and then multiplies the input features; a convolution branch, including two series-connected 3×3×3 three-dimensional convolution layers with a step size of 1; and a residual connection branch, which is used to directly transfer the input features.

[0169] Optionally, the decoder module consists of three serially connected three-dimensional transposed convolutional layers, wherein the convolution kernel size of the first and second three-dimensional transposed convolutional layers is 3×3×3, with a step size of 2, and each layer is sequentially connected to an instance normalization layer and a ReLU activation function, and the convolution kernel size of the third three-dimensional transposed convolutional layer is 7×7×7, with a step size of 1, and a Tanh activation function is added.

[0170] In other words, the generator of the volume-based sparse angle CT image artifact removal network includes an encoder module, a dilated spatial convolution pooling pyramid module, a residual module guided by multiple volume attention, and a decoder module connected in series. The encoder module is composed of multiple three-dimensional convolutional layers connected in series, and an instance normalization layer and a ReLU activation function are added in sequence after each three-dimensional convolutional layer to extract the features of the three-dimensional sub-blocks for training. Through such a design, the encoder module can effectively capture and extract the key feature information of the CT image.

[0171] The dilated spatial convolutional pooling pyramid module receives the output of the encoder module and contains multiple dilated convolution branches with different dilation rates and a global average pooling branch. The outputs of each branch are concatenated along the channel direction to capture multi-scale features. The dilated convolution branches with different dilation rates can capture feature information of different scales, while the global average pooling branch provides global context information.

[0172] The residual module guided by body attention includes body attention weighted branch, convolution branch and residual connection branch. Among them, the body attention weighted branch pools the four dimensions of channel, depth, height and width of the input features respectively, and then processes them through convolution layer, instance normalization layer and Sigmoid activation function to capture the cross-dimensional correlation between different dimensions and multiply them with the input features. The convolution branch includes multiple three-dimensional convolution layers for further processing feature information. The residual connection branch directly transmits the input features. This design can retain the original feature information and enhance the stability of the network.

[0173] The decoder module is composed of multiple 3D transposed convolutional layers in series, and each 3D transposed convolutional layer is followed by an instance normalization layer and an activation function to reconstruct the processed features into an output image of the same size as the input. The design of the decoder module enables the network to effectively reconstruct the extracted feature information back to the original size of the CT image.

[0174] In addition, the encoder module consists of three serially connected 3D convolutional layers, each of which is followed by an instance normalization layer and a ReLU activation function. The convolution kernel size of the first 3D convolutional layer is 7×7×7 with a stride of 1, and the convolution kernel size of the second and third 3D convolutional layers is 3×3×3 with a stride of 2. The dilated spatial convolutional pooling pyramid module includes a 1×1×1 convolution branch, three 3×3×3 dilated convolution branches with different dilation rates, and a global average pooling branch, and the outputs of each branch are concatenated along the channel direction. The volume attention-guided residual module includes: a volume attention weighted branch, which pools the four dimensions of the input features, namely, channel, depth, height, and width, and then captures the cross-dimensional correlation between each three dimensions through a convolution layer, an instance normalization layer, and a sigmoid activation function, and then multiplies the input features; a convolution branch, which includes two serially connected 3×3×3 3D convolutional layers with a stride of 1; and a residual connection branch, which is used to directly transfer the input features. The decoder module consists of three serially connected 3D transposed convolutional layers, where the convolution kernel size of the first and second 3D transposed convolutional layers is 3×3×3, with a stride of 2. Each layer is followed by an instance normalization layer and a ReLU activation function. The convolution kernel size of the third 3D transposed convolutional layer is 7×7×7, with a stride of 1, and a Tanh activation function is added. Through these detailed module designs and configurations, the entire network can efficiently process and reconstruct CT images.

[0175] The above network structure is specifically designed to handle the problem of artifact removal in sparse angle CT images to adapt to the unique needs of CT image processing. This network structure demonstrates a deep understanding of CT image characteristics and cutting-edge deep learning technology. It can not only effectively process 3D data, but also enhance the network's ability to capture multi-scale features and complex spatial relationships through innovative module design. This carefully designed structure is likely to better preserve important anatomical details while removing CT image artifacts, providing a powerful and flexible solution for high-quality CT image reconstruction.

[0176] In this embodiment, the encoder module uses three serially connected three-dimensional convolutional layers, and the design of each layer has its unique purpose and function.

[0177] The first 3D convolutional layer uses a larger 7×7×7 convolution kernel, which is a key innovation. This large convolution kernel can capture a wider range of spatial information in the initial stage, which is crucial for understanding large-scale structures in CT images. The stride of 1 ensures that not too much spatial detail is lost in the first layer. This design allows the network to obtain rich contextual information at the beginning, laying a solid foundation for subsequent feature extraction.

[0178] The second and third 3D convolutional layers use smaller 3×3×3 convolutional kernels, which is based on an important discovery in deep learning: stacking multiple small convolutional kernels can achieve the same receptive field as large convolutional kernels, while greatly reducing the number of parameters and improving computational efficiency. The stride of these two layers is set to 2, which achieves downsampling of feature maps. This progressive downsampling strategy not only reduces the spatial dimension of the feature map, but also gradually increases the number of channels, enabling the network to learn more abstract and advanced feature representations.

[0179] Each convolutional layer is followed by an instance normalization layer and a ReLU activation function, a combination that is widely used in modern deep learning networks for several reasons: The instance normalization layer normalizes the features of each instance (in this case, each CT image). This operation is particularly effective for processing medical images, because different CT scans may have different brightness and contrast. Instance normalization can reduce the impact of such variations, allowing the network to focus more on learning the structural features of the image rather than intensity variations. The ReLU activation function introduces nonlinearity, which is crucial for enhancing the expressive power of the network. It allows the network to learn complex nonlinear mappings, which is particularly important when dealing with complex artifact patterns in CT images. In addition, the simple form of ReLU makes gradient calculations efficient, which helps to speed up the network training process. The design of this three-layer structure embodies a balanced idea: the large convolution kernels in the first layer capture broad spatial information, while the small convolution kernels and downsampling in the next two layers gradually extract more abstract features. This progressive feature extraction strategy is well suited to processing multi-scale information in CT images, from large-scale anatomical structures to small-scale texture details.

[0180] In general, this encoder design demonstrates a deep understanding of 3D medical image processing. It can not only effectively extract key features from CT images, but also achieve a good balance between computational efficiency and feature extraction capabilities through reasonable parameter settings and structural arrangements. This design lays a solid foundation for subsequent artifact removal tasks and is expected to significantly improve the quality of reconstructed CT images.

[0181] The dilated spatial convolutional pooling pyramid module contains five parallel branches, each with its own specific role and advantages. This multi-branch structure allows the network to capture information from different spatial scales and receptive fields simultaneously, which is crucial for processing complex structures and artifact patterns in CT images.

[0182] The 1×1×1 convolution branch is a clever design in this module. This small-sized convolution is mainly used for cross-channel information integration and dimensionality reduction. In CT image processing, it can effectively fuse features from different channels, extract more refined representations, and reduce the amount of computation. This branch can capture local information at the pixel level and provide important details for subsequent feature fusion.

[0183] The core innovation of this module is the three 3×3×3 dilated convolution branches with different dilation rates. The uniqueness of dilated convolution (also known as expanded convolution) is that it can significantly expand the receptive field without increasing the number of parameters and computational complexity. By using different dilation rates, the network is able to focus on features of different scales at the same time. For example, a smaller dilation rate may focus on capturing detailed textures, while a larger dilation rate helps understand large-scale structures. This design is particularly suitable for CT images, because CT images often contain both large-scale anatomical structures and subtle lesion features.

[0184] The introduction of the global average pooling branch reflects the consideration of global contextual information. By averaging the entire feature map, this branch can capture global statistical information. In CT image processing, this global information can help the network understand the overall anatomical structure and large-scale artifact patterns, thereby providing important contextual guidance when extracting local features.

[0185] Finally, the outputs of each branch are concatenated along the channel direction. This operation retains the unique features extracted by each branch, allowing subsequent network layers to learn how to best utilize this multi-scale information. In the task of artifact removal in CT images, this fusion can help the network consider local details, medium-scale structures, and global context at the same time, thereby making more accurate judgments.

[0186] In general, this dilated spatial convolutional pooling pyramid module can not only effectively capture multi-scale features, but also strike a balance between the breadth of feature extraction and computational efficiency through clever design. This structure performs well in dealing with complex CT image artifacts, and can focus on large-scale structures and tiny details at the same time, providing strong support for high-quality CT image reconstruction.

[0187] The volume attention-guided residual module is designed to handle the 3D characteristics of CT images. This module contains three key branches, each with its specific functions and technical considerations.

[0188] Regarding the volume attention weighted branch, the four dimensions (channel, depth, height, and width) of the input features are first pooled separately. This sub-dimensional pooling allows the network to consider the feature distribution of each dimension independently. Subsequently, through the series processing of the convolution layer, the instance normalization layer, and the sigmoid activation function, the network is able to learn to capture the cross-dimensional correlation between each three dimensions. The technical significance of this design is that it enables the network to understand and utilize the complex three-dimensional spatial relationships in CT images. For example, the network can learn that the association of certain features in the depth and height dimensions may be more important than in the width dimension. Finally, the obtained attention weights are multiplied by the input features to achieve adaptive adjustment of the input features.

[0189] The convolution branch uses two 3×3×3 3D convolutional layers in series with a stride of 1. This design allows the network to perform local feature extraction and transformation. Using two convolutional layers instead of one larger convolutional layer is consistent with an important finding in deep learning: stacking multiple small convolutional kernels can achieve the same receptive field as a large convolutional kernel while reducing the number of parameters and improving computational efficiency. A stride of 1 ensures that spatial resolution is maintained, which is important for preserving details in CT images.

[0190] The residual connection branch directly passes the input features. From a technical point of view, the residual connection has several important functions: first, it alleviates the gradient vanishing problem in deep networks, making it easier to train deep networks; second, it allows the network to learn identity mappings, which is beneficial in some cases (for example, when the input is already high-quality features); finally, it provides a shortcut for the network so that information can flow more easily between different layers.

[0191] The outputs of these three branches are finally merged to form the output of the module. This design allows the network to flexibly decide how to best utilize the attention mechanism, convolutional transformation, and original features. In the processing of CT images, this structure can help the network better handle complex three-dimensional structures while maintaining sensitivity to details, which is critical for high-quality artifact removal and image reconstruction.

[0192] The structure of the decoder module is carefully tuned to the characteristics of 3D data. The module consists of three 3D transposed convolutional layers connected in series, and the design of each layer has its own specific technical considerations.

[0193] The first two 3D transposed convolutional layers use a 3×3×3 kernel and a stride of 2. This configuration has several technical advantages: First, the 3×3×3 kernel size can effectively capture local spatial information while keeping the number of parameters small. This is based on an important discovery in deep learning: stacking multiple small kernels can achieve the same receptive field as a large kernel while reducing computational complexity. Second, a stride of 2 means that each operation doubles the spatial dimension of the feature map. This progressive upsampling strategy helps to gradually restore spatial details and avoids artifacts that may be caused by direct large upsampling.

[0194] The instance normalization layer and ReLU activation function connected after each transposed convolution layer also have their specific roles. Instance normalization helps stabilize feature distribution and reduce internal covariate shift, which is particularly effective for processing CT images that may have intensity changes. The ReLU activation function introduces nonlinearity, enhances the network's expressiveness, and facilitates the propagation of gradients.

[0195] The third transposed convolution layer uses a larger 7×7×7 convolution kernel and a stride of 1. This design has its own unique technical considerations: large-sized convolution kernels can capture a wider range of spatial information in the final stage, which helps to restore more detailed image details. The stride of 1 ensures that the last layer does not change the spatial size of the feature map, but focuses on fine-tuning the features.

[0196] The last layer uses the Tanh activation function instead of ReLU. This choice also has technical reasons: the output range of the Tanh function is [-1, 1], which matches the CT image data normalized to the range of [-1, 1] during preprocessing. Tanh has a large gradient near zero, which is good for handling subtle changes in the image and helps to recover fine image details. In addition, Tanh is a bounded function, which can help control the range of the output and avoid producing too large values.

[0197] The design of this three-layer structure embodies a balanced idea: the first two layers gradually increase the spatial dimension of the feature map through smaller convolution kernels and a step size of 2, while the last layer uses a large convolution kernel and a step size of 1 to fine-tune the features. This progressive reconstruction strategy is very suitable for processing multi-scale information in CT images, and can effectively reconstruct everything from large-scale structures to small-scale texture details.

[0198] It should be pointed out that the above description is a preferred embodiment of the present application, which is exemplary and not restrictive. In addition to the specific structures and steps described, the present application may also adopt a variety of other implementation methods. For example, the encoder module may adopt convolutional layers of different numbers and sizes, not limited to three three-dimensional convolutional layers connected in series; the number of residual modules guided by body attention can be adjusted according to actual needs, and the branch structure in the module can also be adjusted according to actual needs, such as increasing or decreasing the number of convolutional layers, or using other types of activation functions. In addition, the configuration of the decoder module can also be optimized according to different application scenarios, such as using transposed convolution kernels of different sizes. Therefore, the present application is not limited to the above-mentioned specific implementation methods, but also includes all equivalent transformations and improvements made according to the technical concept of the present application.

[0199] Loss Function

[0200] The loss function is further explained in detail below.

[0201] Optionally, the cycle consistency loss in the loss function is calculated using a perceptual metric method to better conform to the characteristics of the human visual system.

[0202] Optionally, the perceptual metric method is a learned perceptual patch similarity (LPIPS) metric, which is used to calculate the perceptual difference between the generated image and the original image, and to avoid over-smoothing of the generated image by capturing high-level features.

[0203] Optionally, the cycle consistency loss includes:

[0204] After the sparse angle CT image is transformed into the full angle CT image domain by the first generator A, it is transformed back into the sparse angle CT image domain by the second generator B, and the LPIPS metric between the result and the original sparse angle CT image is obtained;

[0205] After the full-angle CT image is transformed into the sparse angle CT image domain through the second generator B, it is then transformed back to the full-angle CT image domain through the first generator A, and the LPIPS metric between the obtained result and the original full-angle CT image is obtained.

[0206] Specifically, the cycle consistency loss in the loss function is calculated using a perceptual metric method to better conform to the characteristics of the human visual system. Specifically, this perceptual metric method is the learned perceptual patch similarity (LPIPS) metric. The LPIPS metric is used to calculate the perceptual difference between the generated image and the original image, and to avoid over-smoothing the generated image by capturing high-level features.

[0207] The calculation process of the cycle consistency loss is as follows: First, the sparse angle CT image is transformed into the full angle CT image domain through the first generator A, and then transformed back to the sparse angle CT image domain through the second generator B. At this time, the obtained image result is measured by LPIPS with the original sparse angle CT image to calculate the perceptual difference between them. Similarly, the full angle CT image is transformed into the sparse angle CT image domain through the second generator B, and then transformed back to the full angle CT image domain through the first generator A. The final image result is measured by LPIPS with the original full angle CT image to calculate the perceptual difference between them.

[0208] In this way, the cycle consistency loss ensures that the image maintains high-quality perceptual consistency during the translation between the two domains, thereby improving the overall effect of image processing and reconstruction.

[0209] It can be understood that in this embodiment, a perceptual metric method is adopted, specifically a Learning Perceptual Patch Similarity (LPIPS) metric is used to better conform to the characteristics of the human visual system.

[0210] Traditional loss functions, such as mean squared error (MSE) or structural similarity (SSIM), often compare at the pixel level. However, these methods may not fully capture the differences in image quality perceived by the human visual system. The LPIPS metric was introduced to address this problem.

[0211] LPIPS is a perceptual similarity measure based on deep learning. It uses a pre-trained neural network (usually a network trained on large-scale image classification tasks) to extract high-level features of images. These high-level features are closer to the human visual system's understanding of image content, including texture, shape, and semantic information. By comparing these high-level features rather than raw pixel values, LPIPS is able to better assess the perceptual similarity of images.

[0212] There are several important technical advantages of using LPIPS in CT image processing, such as:

[0213] Avoid over-smoothing: Traditional pixel-level loss functions often cause the generated images to be too smooth and lose important texture details. LPIPS can better preserve these details by focusing on high-level features. Preserve structural information: LPIPS can capture the structural information of the image, which is especially important for preserving complex anatomical structures in CT images. Align medical diagnosis requirements: In medical image processing, preserving key diagnostic information is more important than maintaining accurate pixel-level matching. The characteristics of LPIPS are more in line with this requirement.

[0214] The calculation process of cycle consistency loss reflects the innovation of this method: 1) Sparse angle to full angle cycle: The sparse angle CT image is converted to a full angle image through generator A, and then converted back to a sparse angle image through generator B. Finally, the LPIPS metric between this result and the original sparse angle image is calculated. 2) Full angle to sparse angle cycle: Similarly, the full angle CT image is converted to a sparse angle image through generator B, and then converted back to a full angle image through generator A. Then the LPIPS metric between this result and the original full angle image is calculated.

[0215] This bidirectional loop design ensures the consistency and reversibility of the conversion process. By using the LPIPS metric, the network is trained to maintain consistency not only at the pixel level, but also in terms of high-level features and perceptual quality. This is particularly important for preserving key diagnostic information in CT images.

[0216] In general, this LPIPS-based cycle consistency loss design reflects a deep understanding of the special needs of medical image processing and provides an optimization target for high-quality CT image reconstruction that is more in line with human visual perception and medical diagnosis needs.

[0217] It should be pointed out that the above is a preferred embodiment of the present application, which is exemplary and not restrictive. In addition to the specific structures and steps described, the present application may also adopt a variety of other implementation methods. For example, the calculation method of the cycle consistency loss is not limited to the learned perceptual image block similarity (LPIPS), and other perceptual measurement methods may also be used, such as the structural similarity index (SSIM) or other image similarity measurement methods based on deep learning. In addition, the specific implementation of the cycle consistency loss may also be different, for example, the network performance may be optimized by adding additional loss terms. Therefore, the present application is not limited to the above-mentioned specific implementation methods, but also includes all equivalent transformations and improvements made according to the technical concept of the present application.

[0218] Optionally, the calculation formula of the cycle consistency loss is:

[0219] L cycle =E[||B(A(s))-s|| LPIPS ]+E[||A(B(f))-f|| LPIPS ] (5)

[0220] L cycle represents cycle consistency loss;

[0221] E[·] represents the expectation operation;

[0222] ||·|| LPlPS represents the distance calculated using the LPIPS (Learning Perceptual Patch Similarity) metric;

[0223] s represents sparse angle CT image;

[0224] f represents the full-angle CT image;

[0225] A(·) represents the first generator that transforms the sparse angle CT image into the full angle CT image domain;

[0226] B(·) represents the second generator that transforms the full-angle CT image into the sparse-angle CT image domain;

[0227] B(A(s)) represents the result of transforming the sparse angular CT image s into the full-angle CT image domain through generator A and then transforming it back into the sparse angular CT image domain through generator B;

[0228] A(B(f)) represents the result of transforming the full-angle CT image f into the sparse angle CT image domain through generator B, and then transforming it back to the full-angle CT image domain through generator A.

[0229] The above formula describes the calculation method of cycle consistency loss, which is based on the innovative application of the learned perceptual patch similarity (LPIPS) metric. This loss function is designed to ensure that images remain consistent when translated between two domains, while taking into account the characteristics of human visual perception.

[0230] There are several key advantages of using the LPIPS metric instead of traditional pixel-level metrics such as mean squared error: 1) Perceptual similarity: LPIPS is able to capture image similarity that is closer to human visual perception, which is particularly important for medical images. 2) Preserving structural information: It can better preserve the structure and texture information of the image and avoid over-smoothing. 3) High-level feature comparison: LPIPS extracts high-level features through pre-trained neural networks, which are more reflective of the semantic content of the image.

[0231] This cycle consistency loss ensures that the transformation of images between the two domains is consistent and reversible. It encourages the network to learn to preserve the essential features of the image without losing important diagnostic information even in the process of removing artifacts. This is particularly important for CT image reconstruction, which requires a balance between removing artifacts and preserving details.

[0232] In general, the design of this loss function reflects a deep understanding of the special needs of CT image processing, combines advanced deep learning techniques with expertise in medical image processing, and provides a powerful optimization objective for high-quality CT image reconstruction.

[0233] Optionally, the loss function also includes a reconstruction loss, which is used to calculate the perceptual difference between the generated image and the true image, and guide the network to optimize the image details.

[0234] It should be noted that the loss function also includes reconstruction loss, which is used to calculate the perceptual difference between the generated image and the true image, thereby guiding the network to optimize the image details. Specifically, the reconstruction loss evaluates the quality of the image by comparing the difference between the generated image and the true image. The calculation of the reconstruction loss is based on the perceptual metric method, that is, by measuring the similarity between the generated image and the true image in high-level features, not just the pixel-level difference. The perceptual metric method can capture the texture, shape and other complex feature information of the image, which enables the network to retain and restore more image details during the optimization process. By introducing the reconstruction loss, the network can be continuously adjusted and improved during the training process to generate higher quality and closer to the real image.

[0235] Specifically, the reconstruction loss is introduced to further improve the quality of generated images, especially the accuracy in details.

[0236] The core idea of ​​reconstruction loss is to directly compare the difference between the generated image and the true value image (i.e., the target image). In the context of CT image reconstruction, this means comparing the difference between the image after artifact removal and the ideal full-angle CT image. This comparison is not just a simple pixel-level comparison, but a perceptual difference measurement method is used.

[0237] There are several important technical considerations for using perceptual differences instead of traditional pixel-level differences (such as mean square error): 1) Preserving structural information: Perceptual metrics focus more on the structure and texture information of the image rather than just the exact matching of pixel values. This is critical for preserving key anatomical structures and lesion features in CT images. 2) Avoiding over-smoothing: Traditional pixel-level loss functions often result in overly smoothed images and loss of important details. Perceptual metrics can better preserve these diagnostically valuable details. 3) Compatible with the human visual system: Perceptual metrics are closer to the human visual system's judgment of image quality, which is especially important for medical images because these images are ultimately interpreted by doctors. 4) Comparison of high-level features: Many perceptual metrics (such as the aforementioned LPIPS) use pre-trained neural networks to extract high-level features. These features better reflect the semantic content of the image rather than just low-level pixel information.

[0238] By introducing the reconstruction loss, the network is trained not only to remove artifacts, but also to reconstruct results that are as similar as possible to the true value image. This process can be seen as a supervised learning, in which the true value image provides a clear target.

[0239] The addition of reconstruction loss has several important effects on network training: Detail optimization: It guides the network to pay more attention to the details of the image, ensuring that important anatomical structures and potential lesion features are accurately reconstructed. Balanced denoising and fidelity: Reconstruction loss helps find a balance between removing artifacts (mainly handled by cycle consistency loss) and retaining real information. Accelerated convergence: By providing a more direct optimization target, reconstruction loss can help the network converge to the ideal solution faster. Improve generalization ability: By comparing multiple real samples, the network can learn more general image reconstruction rules instead of relying solely on cycle consistency.

[0240] In general, the introduction of reconstruction loss reflects a comprehensive consideration of the CT image reconstruction task. It not only focuses on the removal of artifacts, but also emphasizes the overall quality and detail accuracy of the reconstructed image. This multi-objective optimization strategy helps to generate high-quality CT images that are clear and artifact-free while retaining key diagnostic information.

[0241] Optionally, the calculation formula of the reconstruction loss is:

[0242] L rec =E[||A(s)-f|| LPIPS ]+E[||B(f)-s|| LPIPS ] (6)

[0243] in,

[0244] L rec represents reconstruction losses;

[0245] E[·] represents the expectation operation;

[0246] ||·|| LPIPS represents the distance calculated using the LPIPS (Learning Perceptual Patch Similarity) metric;

[0247] s represents sparse angle CT image;

[0248] f represents the full-angle CT image;

[0249] A(·) represents the generator that transforms the sparse angle CT image to the full angle CT image domain;

[0250] B(·) represents the generator that transforms the full-angle CT image into the sparse-angle CT image domain;

[0251] A(s) represents the result of transforming the sparse angular CT image s to the full-angle CT image domain through the generator A;

[0252] B(f) represents the result of transforming the full-angle CT image f into the sparse angular CT image domain through the generator B.

[0253] The above formula describes the calculation method of the reconstruction loss, which is based on the innovative application of the learned perceptual patch similarity (LPIPS) metric. The reconstruction loss is designed to directly evaluate the perceptual similarity between the generated image and the target image, thereby guiding the network to produce high-quality reconstruction results.

[0254] Using the LPIPS metric instead of traditional pixel-level metrics such as mean squared error has several important advantages: Perceptual similarity: LPIPS is able to capture image similarities that are closer to human visual perception, which is particularly important for quality assessment of medical images. Preserving structural information: It better preserves the structure and texture information of the image and avoids generating overly smoothed results. High-level feature comparison: LPIPS extracts high-level features through pre-trained neural networks, which are more reflective of the semantic content of the image. Strong adaptability: LPIPS can capture subtle differences that are difficult for the human eye to quantify but can be perceived.

[0255] The design of this reconstruction loss has several key roles: Direct optimization objective: It provides a direct optimization objective for the network, that is, the generated image should be as close to the real image as possible. Bidirectional consistency: By simultaneously considering the conversion from sparse angles to full angles and from full angles to sparse angles, the bidirectional consistency of the conversion process is ensured. Detail preservation: The use of LPIPS helps to preserve important image details, which is crucial for the identification of tiny structures and potential lesions in CT images. Balanced denoising and fidelity: This loss function helps the network find a balance between removing artifacts and retaining real information.

[0256] By calculating the expected values ​​of these two parts, we get the reconstruction loss L rec This loss function is used to guide the network to be closer to the real image in the process of generating images, thereby optimizing image details and improving the quality of generated images.

[0257] In general, the design of this reconstruction loss reflects a deep understanding of the CT image reconstruction task. It not only focuses on the removal of artifacts, but also emphasizes the overall quality and perceptual similarity of the reconstructed image. This method helps to generate high-quality CT images that are clear and artifact-free while retaining key diagnostic information, providing a powerful optimization target for medical image processing.

[0258] Optionally, the overall loss of the loss function is a weighted sum of adversarial loss, cycle consistency loss and reconstruction loss, and the calculation formula is:

[0259] L total =L GAN (A,D F )+L GAN (B,D S ) +λ 1 L cycle +λ2 L rec (7)

[0260] in,

[0261] L total Indicates overall loss;

[0262] L GAN (A,D F ) represents the generator A and the discriminator D F The adversarial loss between them is used to optimize the image conversion from sparse angles to full angles;

[0263] L GAN (B,D S ) represents the generator B and the discriminator D S The adversarial loss between them is used to optimize the image conversion from full angle to sparse angle;

[0264] L cycle represents cycle consistency loss;

[0265] L rec represents reconstruction losses;

[0266] λ 1 and λ 2 is the weight coefficient, which is used to balance the contribution of different loss terms;

[0267] A represents the generator that transforms the sparse angle CT image into the full angle CT image domain;

[0268] B represents the generator that transforms the full-angle CT image into the sparse-angle CT image domain;

[0269] D F The discriminator representing the full-angle image domain is used to distinguish the real full-angle CT image from the image generated by generator A;

[0270] D S The discriminator represents the sparse angular image domain, which is used to distinguish between real sparse angular CT images and images generated by generator B.

[0271] The above formula describes the overall loss function used in network training, which is a weighted combination of multiple loss terms. The design of this composite loss function reflects a comprehensive consideration of the CT image reconstruction task and aims to optimize network performance from multiple perspectives.

[0272] Among them, L total Represents the overall loss. GAN (A,D F ) represents the generator A and the discriminator D FThe adversarial loss between is used to optimize the image conversion from sparse angles to full angles. Specifically, the generator A generates full-angle CT images, while the discriminator D F It is used to distinguish the images generated by generator A from the real full-angle CT images. The adversarial loss improves the performance of the generator through this adversarial mechanism, making the images it generates more realistic.

[0273] Likewise, L GAN (B,D S ) represents the generator B and the discriminator D S The adversarial loss between is used to optimize the image conversion from full angle to sparse angle. Generator B generates sparse angle CT images, and discriminator D S It is used to distinguish the images generated by generator B from the real sparse angle CT images. This adversarial mechanism also improves the performance of generator B, making the sparse angle CT images it generates more realistic.

[0274] In addition, L cycle Represents the cycle consistency loss, which is used to ensure that the image remains consistent during the conversion process between the two domains. The cycle consistency loss ensures the reversibility and stability of the conversion process by measuring the difference between the image in the two conversion processes.

[0275] L rec Represents the reconstruction loss, which is used to measure the perceptual difference between the generated image and the true image. The reconstruction loss guides the network to retain more details in the image generation process by calculating the high-level feature similarity between the generated image and the real image.

[0276] Weight coefficient λ 1 and λ 2 Used to balance the contributions of different loss terms. These coefficients adjust the relative importance of adversarial loss, cycle consistency loss, and reconstruction loss in the overall loss. By setting these weight coefficients appropriately, it is possible to ensure that the network simultaneously optimizes the authenticity, consistency, and detail preservation of the generated images during training.

[0277] Generator A and Generator B are used to convert sparse angle CT images to full angle CT image domain and convert full angle CT images to sparse angle CT image domain respectively. Discriminator D F The discriminator D is used to distinguish the real full-angle CT image from the image generated by the generator A. S Used to distinguish between real sparse angle CT images and images generated by generator B. By combining these different loss functions and network components, the overall network can more effectively remove artifacts and generate high-quality CT images.

[0278] The design of this composite loss function has several key advantages: Multi-aspect optimization: consider image authenticity, conversion consistency and reconstruction accuracy at the same time. Balancing different goals: find a balance between removing artifacts, preserving details and generating realistic images by adjusting weights. Perceptual quality improvement: focus on the image quality perceived by the human visual system using the LPIPS metric. Bidirectional learning: optimize in two directions to ensure the bidirectional consistency and reliability of the conversion. Strong adaptability: can adapt to different CT image reconstruction requirements by adjusting weights.

[0279] This overall loss function reflects a deep understanding and comprehensive consideration of the CT image reconstruction task, integrating multiple advanced deep learning concepts to generate high-quality, artifact-free CT images that retain key diagnostic information.

[0280] It should be pointed out that the above is a preferred embodiment of the present application, which is exemplary and not restrictive. In addition to the specific structures and steps described, the present application can also adopt a variety of other implementation methods. For example, the calculation method of the cycle consistency loss is not limited to the learned perceptual image block similarity (LPIPS), and the structural similarity index (SSIM) or other image similarity measurement methods based on deep learning can also be used. The calculation of the reconstruction loss can also use other perceptual measurement methods instead of LPIPS. In addition, the generator and discriminator structures of the adversarial loss can be adjusted according to the specific application, such as using different network architectures or optimization algorithms. The weight coefficients (λ1 and λ2) of the overall loss can also be adjusted according to different training requirements to balance the contribution of each loss item. Therefore, the present application is not limited to the above-mentioned specific implementation methods, but also includes all equivalent transformations and improvements made according to the technical concept of the present application.

[0281] Combine the following Figure 2 , Figure 3 and Figure 4 , this embodiment is further explained.

[0282] Figure 2 The generator structure of the volume-based sparse angular CT image artifact removal network is shown. The structure presents the entire processing flow from the input CT image with artifacts to the output optimization result from left to right. The architecture of the generator mainly consists of four parts: encoder, atrous spatial convolutional pooling pyramid module, volume attention-guided residual module and decoder.

[0283] The encoder is located on the left side of the structure and contains multiple 3D convolutional layers. Each convolutional layer is followed by instance normalization and ReLU activation function, which helps to extract multi-level features of the input image. After the encoder, the dilated spatial convolutional pooling pyramid module (marked as c in the figure) plays an important role. This module contains multiple parallel convolution branches, each branch uses a different dilation rate, which enables the network to effectively capture multi-scale features and enhance the recognition ability of artifacts of different sizes.

[0284] The core of the network is composed of 6 series-connected residual modules guided by body attention (marked as B in the figure). Each module contains a complex body attention mechanism that can capture the correlation between the four dimensions of channel (C), depth (D), height (H) and width (W). In addition, the module also contains convolution branches for further feature extraction, and residual connections to directly transfer input features. This design not only enhances the expressiveness of the network, but also helps to stabilize the propagation of gradients.

[0285] The decoder is located on the right side of the structure and consists of multiple 3D transposed convolutional layers. The role of these layers is to reconstruct the processed features into the final output image. It is worth noting that there are also multiple jump connections between the encoder and the decoder. This design helps to preserve the details of the original image and ensure the high quality of the output image.

[0286] The design of this network structure embodies several key innovations: first, by using 3D operations to process volume blocks, the three-dimensional information of CT images is fully utilized; second, the volume attention mechanism can capture the complex correlations between different dimensions, significantly enhancing the network's expressive power; third, the introduction of the atrous convolutional pyramid module enables the network to effectively capture multi-scale features; finally, the use of residual structures and jump connections not only facilitates the network training process, but also better preserves image details.

[0287] Overall, the network structure is designed to process 3D CT images more effectively, improve the effect of artifact removal, and ensure the high quality of the output image. This method fully considers the three-dimensional characteristics of CT images and the complexity of artifacts, and provides a powerful solution for artifact removal of sparse angle CT images.

[0288] Figure 3 The training process and loss function composition of the volume-based sparse angle CT image artifact removal network are shown. The figure clearly shows how the network processes sparse angle CT images and full angle CT images at the same time, as well as the various loss functions applied during the training process.

[0289] In the upper part of the figure, we can see the processing flow of the sparse angular CT image s. First, s is processed by generator A, which converts it from the sparse angular image domain (S) to the full angular image domain (F), resulting in A(s). Then, A(s) is converted back to the sparse angular image domain through generator B, resulting in B(A(s)). This process forms a complete loop for calculating the cycle consistency loss L cycle_s At the same time, A(s) is compared with the real full-angle CT image f to calculate the reconstruction loss L rec_s In addition, A(s) is also fed into the discriminator D F , used to calculate the adversarial loss L GAN (A,D F ).

[0290] The lower part of the figure shows a similar process for the full-angle CT image f. f is first converted to the sparse angular image domain by generator B, obtaining B(f), and then converted back to the full-angle image domain by generator A, obtaining A(B(f)). This process is used to calculate another cycle consistency loss L rec_f B(f) is compared with the real sparse angular CT image s to calculate the reconstruction loss L rec_f Similarly, B(f) is fed into the discriminator D s , used to calculate the adversarial loss L GAN (B,D S )).

[0291] This complex training process and diverse loss function designs reflect several key features of the network: first, the bidirectional conversion (S→F and F→S) ensures that the network can learn the essential relationship between sparse angle and full angle CT images; second, the cycle consistency loss ensures the reversibility of the conversion process, which helps to preserve the important features of the image; third, the reconstruction loss ensures the similarity between the generated image and the real image; finally, the adversarial loss encourages the generator to produce more realistic images, improving the quality of the artifact removal effect.

[0292] In general, this carefully designed combination of training process and loss function enables the network to comprehensively learn the characteristics of sparse angle CT images and effectively remove artifacts while maintaining the authenticity and details of the image. This method not only considers the image transformation in a single direction, but also ensures the consistency and reliability of the transformation through a bidirectional cycle, providing strong support for the high-quality reconstruction of sparse angle CT images.

[0293] Figure 4The comparison of the effect of the method of this application and several other advanced methods on artifact removal of sparse angle CT images is shown. This figure contains six columns of images, from left to right, they are full-angle CT images (as a reference standard), original sparse angle CT images, U-former method processing results, PTNet3D method processing results, Pix2Pix3D method processing results, and the processing results of the method of this application. The images are divided into two rows, the upper row shows the cross-sectional images, and the lower row shows the sagittal images. This multi-angle display helps to comprehensively evaluate the performance of each method.

[0294] In the cross-sectional image, it can be clearly seen that there are obvious streak artifacts in the original sparse angle CT image, which seriously affects the image quality and the recognition of details. In contrast, the image processed by the method of the present application almost completely removes these artifacts, and the image quality is close to that of the full-angle CT image. Especially in the area marked by the yellow box in the figure, the method of the present application shows excellent detail retention ability, and the structural clarity is significantly better than other methods.

[0295] The comparison of sagittal images further highlights the advantages of the method of the present application. In the original sparse angle CT image, we can observe severe blurring and streak-like artifacts, which almost make the image lose continuity in the vertical direction. Other methods have improved this situation to a certain extent, but there are still varying degrees of blurring and structural breaks. The sagittal images processed by the method of the present application show excellent continuity and clarity, especially in the area marked by the red box, the recovery effect of fine structure is very close to the full-angle CT image.

[0296] Through such visual comparison, we can intuitively see that the method of this application performs well in removing artifacts, preserving details and maintaining image continuity. Especially when dealing with complex three-dimensional structures, this method can maintain high-quality image reconstruction on different sections, which is of great significance for improving the application value of CT images in clinical diagnosis. In general, Figure 4 This strongly demonstrates the superiority of the method in the application in the task of removing artifacts from sparse angle CT images. It not only has significant effects on cross-sections, but also shows obvious advantages in maintaining the three-dimensional consistency of images.

[0297] The above embodiments have the following technical effects:

[0298] The above embodiment proposes an innovative method for removing artifacts from sparse angle CT images, the core of which is to make full use of the three-dimensional information of CT images and advanced deep learning technology. The method mainly includes three key points, each of which brings significant technical effects.

[0299] First, the above embodiment uses three-dimensional sub-blocks rather than two-dimensional slices as units to perform artifact removal on sparse angle CT images. This innovation makes full use of the three-dimensional information of CT images. Traditional methods usually treat CT images as a series of independent two-dimensional slices for processing, ignoring the spatial correlation between slices. The above embodiment effectively captures the structural information of CT images in all directions by processing three-dimensional sub-blocks containing multiple adjacent slices. The technical effect brought about by this method is significant: the processed images show better structural coherence in the cross-section, sagittal and coronal planes, greatly improving the restoration effect of image details. Especially in the sagittal and coronal planes, this method effectively avoids the discontinuity phenomenon common in traditional methods, making the reconstructed three-dimensional images more realistic and reliable. In addition, by considering the information of multiple adjacent slices at the same time, this method can more accurately distinguish between the real structure and artifacts in the image, thereby achieving more accurate artifact removal.

[0300] Secondly, the above embodiment specially designs a block-based network structure for the artifact removal task of sparse angle CT images. The core of this network lies in its volume attention-guided residual module, which can effectively capture the cross-dimensional correlation between different dimensions and significantly enhance the network's expressive power. Specifically, this module processes the channel, depth, height, and width dimensions through four branches, respectively, to capture the complex interactions between them. The technical effects brought about by this design are multifaceted: first, it greatly improves the network's understanding and processing capabilities of three-dimensional structural information, allowing the network to more accurately identify and remove complex artifact patterns; secondly, the network's multi-branch structure and residual connection not only enhance the feature extraction capability, but also make the training process more stable and efficient by allowing the direct transfer of gradients; finally, this design enables the network to show stronger adaptability when processing CT images of different parts or different scanning parameters.

[0301] Finally, the above embodiment uses the LPIPS metric when calculating the cycle consistency loss, and proposes the concept of cycle consistency perceptual loss. This loss function is more in line with the characteristics of the human visual system and effectively avoids the generation of overly smoothed images. Traditional pixel-level loss functions often lead to the loss of image details, while the LPIPS metric can better retain the texture and detail information of the image by considering the similarity of high-level features. The technical effects brought about by this innovation are significant: first, the processed images are more natural and realistic visually, with richer details, which is crucial for medical diagnosis; second, this method achieves a better balance between removing artifacts and retaining important anatomical details, improving the clinical practicality of the processing results; finally, because it is more in line with human visual perception, the images processed by this method are easier for doctors to accept and interpret.

[0302] In addition, the present application significantly optimizes the processing speed by processing in units of three-dimensional sub-blocks. The two-dimensional processing method requires frequent loading and processing of each tomographic image, while the present application reduces the frequency of data loading by processing three-dimensional sub-blocks containing multiple adjacent slices. Especially in large-scale CT data processing scenarios, this improvement greatly improves the processing efficiency. At the same time, the present application takes into account the pressure of processing three-dimensional images on the video memory, and processes three-dimensional sub-blocks instead of the entire three-dimensional image, making it possible to process larger-sized three-dimensional data under the same hardware conditions, effectively alleviating the video memory pressure in high-resolution CT image processing. In addition, the strategy of sampling three-dimensional sub-blocks during training in the present application is to sample sub-blocks at random positions in the entire three-dimensional image each round, rather than artificially cutting sub-blocks when making data sets, which can improve the diversity of training samples and enable the network to learn more information. In summary, especially in large-scale CT data processing scenarios, these improvements have greatly improved processing efficiency and processing power, so that efficient CT image artifact removal can be achieved without increasing hardware costs.

[0303] In summary, these three key points not only bring significant technical progress individually, but the synergy between them produces a comprehensive effect that goes beyond the individual technical points. The 3D sub-block processing method provides the network with rich spatial information, and the specially designed network structure fully utilizes this information through its volume attention-guided residual module to accurately capture the correlation between different dimensions. This structure complements the cycle consistency perception loss, which together ensures that subtle anatomical details are preserved while removing artifacts. In particular, the LPIPS metric takes into account human visual perception when evaluating image quality, and combined with the 3D sub-block processing method, it maintains a high degree of perceptual consistency throughout the 3D space.

[0304] The effect of this synergy is significant: for a CT image containing 150 slices, the processing time is reduced from 18 seconds (U-former is used as an example) to only 3 seconds, which is a 6-fold increase in efficiency, while achieving unprecedented image quality improvement. This combination of high efficiency and high quality makes this method of great application value in clinical and research scenarios where a large number of sparse angle CT images need to be processed quickly. More notably, the advantages of this method will expand exponentially as the amount of data to be processed and the number of slices increase. This is because the three-dimensional processing method reduces the need to switch between different dimensions, the efficient design of the network structure reduces the computational complexity, and the use of perceptual loss ensures that excellent image quality can be maintained even in high-speed processing. This scalability makes this method an ideal choice for large-scale CT image data processing, bringing revolutionary progress to the field of medical imaging.

[0305] In order to better understand the technical solution of the present application, a specific example is provided below for illustration. The details listed in the example are mainly for ease of understanding and are not intended to limit the scope of protection of the present application.

[0306] This example proposes a method for removing artifacts from sparse angle CT images, including the following steps:

[0307] Step 110, acquiring sparse angle CT images and full angle CT images, and preprocessing the data, including resampling, size clipping and gray value normalization.

[0308] Preferably, for step 110, the object in the CT image is a walnut. In this embodiment, the full angle is preferably 1200 projection angles. To show the effectiveness of the method of the present application in extremely sparse angle conditions, the sparse angles are preferably one-fortieth to one-twentieth of the full angle, that is, 30 to 60 projection angles, and specifically can be 40 projection angles, which is one-thirtieth of the full angle projection angles. The original image size is (512, 512, 512), and no resampling is performed. The size is cropped to (360, 360, 150), and the grayscale value is normalized to between [-1, 1].

[0309] Step 120, construct a volume-based sparse angle CT image artifact removal network, the model includes two pairs of generators and discriminators, one pair of generators and discriminators is dedicated to transforming images from sparse angle image domain (S) to full angle image domain (F), and the other pair performs the opposite process. The generator includes an encoder module, a void spatial convolution pooling pyramid module, a volume attention guided residual module and a decoder module; the discriminator consists of 5 downsampling convolution blocks.

[0310] More specifically, step 120 may further include the following sub-steps:

[0311] Step 121: The generator as described in step 120 is composed of an encoder module, a dilated spatial convolutional pooling pyramid module, a residual module guided by 6 individual attentions, and a decoder module in series, and the final output image has the same size as the output image. The structure of each module is as follows:

[0312] The encoder module consists of three 3D convolutional layers in series. The convolution kernel size of the first convolutional layer is 7×7×7 and the stride is 1. The convolution kernel size of the second and third convolutional layers is 3×3×3 and the stride is 2. An instance normalization layer and a ReLU activation function are added after each convolutional layer.

[0313] The dilated spatial convolutional pooling pyramid module consists of a multi-branch structure of a 1×1×1 convolution, three dilated convolutions with different dilation rates, and a global average pooling layer. The convolution kernel size of the three dilated convolutions is 3×3×3, the step size is 1, and the dilation rates are 1, 2, and 4 respectively. The feature layers output by the previous encoder module pass through the above five branches respectively and are spliced ​​along the channel;

[0314] The residual block guided by body attention includes three branches, body attention weighted branch, convolution branch and residual connection branch. The body attention weighted branch pools the four dimensions of channel, body length, body width and body height respectively, and then passes through the convolution layer, instance regularization layer and Sigmoid activation function to capture the cross-dimensional correlation between every three dimensions, and then multiplies the average with the input feature. The convolution branch includes two convolution layers with a convolution kernel size of 3×3×3 and a stride of 1.

[0315] The decoder module consists of three transposed convolutional layers in series. The convolution kernel size in the first and second transposed convolutional layers is 3×3×3, with a step size of 2. An instance regularization layer and a ReLU activation function are added after each layer. The convolution kernel size of the third transposed convolutional layer is 7×7×7, with a step size of 1, followed by Tanh activation.

[0316] Step 122: The discriminator described in step 120 is composed of 5 downsampling feature extraction layers connected in series, and the feature extraction layer includes a convolution layer, an instance normalization layer and a LeakyReLu activation function.

[0317] Step 130: Use the preprocessed data for training, and sample a certain size of image sub-blocks at random positions in each round and input them into the network for training. Preferably, for the step 130, the size of the image sub-block sampled each time is (180, 180, 28), and the training process is as follows: Figure 3 As shown, taking a sparse angular CT image sub-block s as an example, generator A executing S→F transforms s into A(s), and then generator B executing F→S transforms it back to S and obtains the result B(A(s)).

[0318] Step 140: Optimize the model using a loss function consisting of adversarial loss, cycle consistency perception loss and reconstruction loss. The adversarial loss, cycle consistency perception loss and reconstruction loss may further include the following sub-steps:

[0319] Step 141: Define adversarial loss: adversarial loss is used to improve the image generation ability of the generator and the classification ability of the discriminator;

[0320] L GAN (A,D F )=E[log D F (f)]+E[log(1-DF (A(s)))] (3)

[0321] L GAN (B,D S )=E[log D S (s)]+E[log(1-D S (B(f)))] (4)

[0322] Where S is the sparse angle CT image domain, s is a sample in S, F is the full angle CT image domain, and f is a sample in F. A is the generator that transforms from the sparse angle CT image domain to the full angle CT image domain, and B is the generator that transforms from the full angle CT image domain to the sparse angle CT image domain. F D is a discriminator for determining whether an image belongs to the full-angle CT image domain. S A discriminator for determining whether an image belongs to the sparse angular CT image domain.

[0323] E[logD F (f)]: This part is the expected operation, which represents the discriminator D F The probability of judging a sample in the full-angle CT image domain as a full-angle CT image. S (s)] The same applies.

[0324] E[log(1-D F (A(s)))]: This is also the expected operation, indicating the discriminator D F The probability of judging the result after the generator transforms the samples in the sparse angle CT image domain to the full angle CT image domain as a sparse angle CT image. s (B(f)))]The same applies.

[0325] Step 142: Define the cycle consistency perceptual loss. After the sparse angle CT image is transformed into the full angle CT image domain through generator A, it is then transformed back into the sparse angle CT image domain through another generator B. The result obtained should be the same as the original image in theory. The difference between the transformed result and the original image is measured by LPIPS, which is the cycle consistency perceptual loss. The cycle transformation process of the full angle image is the same.

[0326] L cycle =E[||B(A(s))-s|| LPIPS ]+E[||A(B(f))-f|| LPIPS ] (5)

[0327] The first term is the calculation formula for calculating the cycle consistency loss for sparse angle CT images. For sparse angle CT images s, they are first transformed to the full angle CT image domain by the generator sub-network A, and then transformed back to the sparse angle CT image domain by the generator sub-network B. In theory, they should be the same as the original image s. The perceptual loss between them and s is calculated, and the smaller the expectation, the better. The second term is the same.

[0328] Step 143: Define the reconstruction loss, calculate the perceptual loss between the generated image and the true image, and guide the network to optimize the image details;

[0329] L rec =E[||A(s)-f|| LPIPS ]+E[||B(f)-s|| LPIPS ] (6)

[0330] E[||A(s)-f|| PPIPS ]: A(s) is the result of transforming the sparse angle CT image s to the full angle CT image domain, f is the true value image of s in the full angle CT image domain, that is, the result we want to obtain through the transformation. We calculate the perceptual loss between the two, and we hope that the smaller the expectation, the better. E[||B(f)-s|| LPIPS ]The same applies.

[0331] Step 144: Define the overall loss, add the adversarial loss, cycle consistency perception loss and reconstruction loss according to certain weights to obtain the overall loss.

[0332] L total =L GAN (A,D F )+L GAN (B,D s )+λ 1 L cycle +λ 2 L rec (7)

[0333] Preferably, for step 140, the weight coefficient λ 1 , 2 , 3 Both are set to 10, iterated 300 times, and the Adam optimizer is used to optimize the model.

[0334] Step 150: Segment the sparse angle CT image into a number of sub-blocks of the same size as the sampling size during training in order of spatial position, input them into generator A in sequence for processing, and finally merge them into an optimization result.

[0335] Preferably, for the step 150, the sparse angle CT image is divided into a number of sub-blocks of size (180, 180, 28) in order of spatial position, and are sequentially input into generator A for processing, and then the output sub-blocks are filled into corresponding positions. For the case where it cannot be divided evenly, the last extra incomplete sub-block is also required to take a sub-block of size (180, 180, 28) from the end to the front and input it into generator A, and the corresponding part of the output result is taken for filling, and finally the result after artifact removal is obtained.

[0336] The second embodiment of the present application relates to a system for removing artifacts from sparse angle CT images, the structure of which is as follows: Figure 6 As shown, the system for removing artifacts from sparse angle CT images includes:

[0337] A data preprocessing module is used to obtain a sparse angle CT image and a full angle CT image, and preprocess them to obtain a preprocessed sparse angle CT image and a full angle CT image;

[0338] A network construction module is used to divide the preprocessed sparse angle CT image and full angle CT image into a number of training three-dimensional sub-blocks; and to construct a volume-based sparse angle CT image artifact removal network, the network comprising two pairs of generators and discriminators, one pair of which is used to transform the training three-dimensional sub-blocks from the sparse angle image domain to the full angle image domain, and the other pair is used to perform the reverse process;

[0339] A network training module, used to train the network using the training three-dimensional sub-blocks, and during the training process, the network is optimized using a loss function including adversarial loss, cycle consistency loss and reconstruction loss to obtain a trained network;

[0340] An image processing module, used for pre-processing the sparse angle CT image to be processed, dividing it into a number of to-be-processed three-dimensional sub-blocks of the same size as the three-dimensional sub-blocks in order of spatial position, and sequentially inputting the to-be-processed three-dimensional sub-blocks into the trained network for processing to obtain the three-dimensional sub-blocks after removing artifacts;

[0341] The image synthesis module is used to merge the three-dimensional sub-blocks after the artifacts are removed to form a CT image after the artifacts are removed.

[0342] The first implementation manner is a method implementation manner corresponding to the present implementation manner. The technical details in the first implementation manner can be applied to the present implementation manner, and the technical details in the present implementation manner can also be applied to the first implementation manner.

[0343] It should be noted that those skilled in the art should understand that the implementation functions of each module shown in the above-mentioned system for removing artifacts from sparse angle CT images can be understood by referring to the relevant description of the above-mentioned method for removing artifacts from sparse angle CT images. The functions of each module shown in the above-mentioned system for removing artifacts from sparse angle CT images can be implemented by a program (executable instruction) running on a processor, or by a specific logic circuit. If the system for removing artifacts from sparse angle CT images in the embodiment of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, ReadOnly Memory), a disk or an optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.

[0344] Accordingly, an embodiment of the present application further provides a computer storage medium, in which computer executable instructions are stored. When the computer executable instructions are executed by a processor, the various method embodiments of the present application are implemented.

[0345] In addition, the embodiments of the present application also provide a system for removing artifacts from sparse angle CT images, which includes a memory for storing computer executable instructions, and a processor; the processor is used to implement the steps in the above-mentioned method implementations when executing the computer executable instructions in the memory. Among them, the processor can be a central processing unit (Central Processing Unit, referred to as "CPU"), or other general-purpose processors, digital signal processors (Digital Signal Processor, referred to as "DSP"), Application Specific Integrated Circuit (Application Specific Integrated Circuit, referred to as "ASIC"), etc. The aforementioned memory can be a read-only memory (read-only memory, referred to as "ROM"), a random access memory (random access memory, referred to as "RAM"), a flash memory (Flash), a hard disk or a solid-state drive, etc. The steps of the method disclosed in each embodiment of the present application can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0346] All documents mentioned in this application are considered to be included in the disclosure of this application as a whole, so that they can be used as the basis for modification when necessary. In addition, it should be understood that after reading the above disclosure of this application, those skilled in the art can make various changes or modifications to this application, and these equivalent forms also fall within the scope of protection claimed in this application.

Claims

1. A method for removing artifacts from sparse angle CT images, characterized in that: The following steps are involved: Acquire a sparse angle CT image and a full angle CT image, and preprocess them to obtain a preprocessed sparse angle CT image and a full angle CT image; Dividing the preprocessed sparse angle CT image and full angle CT image into a number of three-dimensional sub-blocks for training; and constructing a volume-based sparse angle CT image artifact removal network, the network comprising two pairs of generators and discriminators, one pair for transforming the three-dimensional sub-blocks for training from the sparse angle image domain to the full angle image domain, and the other pair for performing the reverse process; The network is trained using the training three-dimensional sub-blocks, and during the training process, the network is optimized using a loss function including adversarial loss, cycle consistency loss, and reconstruction loss to obtain a trained network; After preprocessing the sparse angle CT image to be processed, the image is divided into a number of three-dimensional sub-blocks to be processed that are the same size as the three-dimensional sub-blocks in order of spatial position, and the three-dimensional sub-blocks to be processed are input into the trained network in turn for processing to obtain three-dimensional sub-blocks after artifacts are removed; and the three-dimensional sub-blocks after artifacts are removed are merged to form a CT image after artifacts are removed.

2. The method according to claim 1, characterized in that: The generator of the block-based sparse angle CT image artifact removal network includes an encoder module, a dilated spatial convolutional pooling pyramid module, a multi-body attention-guided residual module and a decoder module connected in series; wherein: The encoder module is composed of a plurality of three-dimensional convolutional layers connected in series, and an instance normalization layer and a ReLU activation function are sequentially added after each of the three-dimensional convolutional layers to extract features of the three-dimensional sub-blocks for training; The atrous spatial convolution pooling pyramid module receives the output of the encoder module, and includes a plurality of atrous convolution branches with different atrous rates and a global average pooling branch, and the outputs of the branches are spliced ​​along the channel direction to capture multi-scale features; The body attention guided residual module includes a body attention weighted branch, a convolution branch and a residual connection branch, wherein the body attention weighted branch performs pooling on the four dimensions of channel, depth, height and width of the input features respectively, and is used to capture the cross-dimensional correlation between different dimensions after being processed by a convolution layer, an instance normalization layer and a Sigmoid activation function, and multiply the input features; the convolution branch includes multiple three-dimensional convolution layers; the residual connection branch directly transfers the input features; The decoder module is composed of a plurality of three-dimensional transposed convolutional layers connected in series, and an instance normalization layer and an activation function are sequentially added after each of the three-dimensional transposed convolutional layers to reconstruct the processed features into an output image of the same size as the input.

3. The method according to claim 2, characterized in that The dilated spatial convolution pooling pyramid module includes a 1×1×1 convolution branch, three 3×3×3 dilated convolution branches with different dilation rates and a global average pooling branch, and the outputs of each branch are spliced ​​along the channel direction.

4. The method according to claim 2, characterized in that: The body attention guided residual module includes: - The body attention weighted branch pools the four dimensions of channel, depth, height and width of the input features, and then captures the cross-dimensional correlation between each three dimensions through the convolution layer, instance normalization layer and Sigmoid activation function, and then multiplies the input features; - Convolutional branch, consisting of two 3×3×3 3D convolutional layers connected in series with a stride of 1; -Residual connection branch, used to directly transfer input features.

5. The method according to claim 1, characterized in that The cycle consistency loss in the loss function is calculated using a perceptual metric method to better conform to the characteristics of the human visual system, and the perceptual metric method is a learned perceptual image patch similarity (LPIPS) metric, which is used to calculate the perceptual difference between the generated image and the original image, and avoid excessive smoothing of the generated image by capturing high-level features.

6. The method according to claim 5, characterized in that The cycle consistency loss includes: After the sparse angle CT image is transformed into the full angle CT image domain by the first generator A, it is transformed back into the sparse angle CT image domain by the second generator B, and the LPIPS metric between the result and the original sparse angle CT image is obtained; After the full-angle CT image is transformed into the sparse angle CT image domain through the second generator B, it is transformed back to the full-angle CT image domain through the first generator A. The LPIPS metric between the obtained result and the original full-angle CT image is calculated as follows: L cycle =E[||B(A(s))-s|| LPIPS ]+E[||A(B(f))-f|| LPIPS ] (5) L cycle represents cycle consistency loss; E[·] represents the expectation operation; ||·|| LPIPS represents the distance calculated using the LPIPS (Learning Perceptual Patch Similarity) metric; s represents sparse angle CT image; f represents the full-angle CT image; A(·) represents the first generator that transforms the sparse angle CT image into the full angle CT image domain; B(·) represents the second generator that transforms the full-angle CT image into the sparse-angle CT image domain; B(A(s)) represents the result of transforming the sparse angular CT image s into the full-angle CT image domain through generator A and then transforming it back into the sparse angular CT image domain through generator B; A(B(f)) represents the result of transforming the full-angle CT image f into the sparse angle CT image domain through generator B, and then transforming it back to the full-angle CT image domain through generator A.

7. The method according to claim 6, characterized in that The loss function also includes a reconstruction loss, which is used to calculate the perceptual difference between the generated image and the true value image, and guide the network to optimize the image details, wherein the calculation formula of the reconstruction loss is: L rec =E[||A(s)-f|| LPIPS ]+E[||B(f)-s|| LPIPS ] (6) in, L rec represents reconstruction losses; E[·] represents the expectation operation; ||·|| LPIPS represents the distance calculated using the LPIPS (Learning Perceptual Patch Similarity) metric; s represents sparse angle CT image; f represents the full-angle CT image; A(·) represents the generator that transforms the sparse angle CT image to the full angle CT image domain; B(·) represents the generator that transforms the full-angle CT image into the sparse-angle CT image domain; A(s) represents the result of transforming the sparse angular CT image s to the full-angle CT image domain through the generator A; B(f) represents the result of transforming the full-angle CT image f into the sparse angular CT image domain through the generator B.

8. The method according to claim 7, characterized in that The overall loss of the loss function is the weighted sum of adversarial loss, cycle consistency loss and reconstruction loss, and the calculation formula is: L total =L GAN (A,D F )+L GAN (B,D S )+λ1L cycle +λ2L rec (7) in, L total Indicates overall loss; L GAN (A,D F ) represents the generator A and the discriminator D F The adversarial loss between them is used to optimize the image conversion from sparse angles to full angles; L GAN (B,D S ) represents the generator B and the discriminator D S The adversarial loss between them is used to optimize the image conversion from full angle to sparse angle; L cycle represents cycle consistency loss; L rec represents reconstruction losses; λ1 and λ2 are weight coefficients used to balance the contribution of different loss terms; A represents the generator that transforms the sparse angle CT image into the full angle CT image domain; B represents the generator that transforms the full-angle CT image into the sparse-angle CT image domain; D F The discriminator representing the full-angle image domain is used to distinguish the real full-angle CT image from the image generated by the generator A; D S The discriminator represents the sparse angular image domain, which is used to distinguish between real sparse angular CT images and images generated by generator B.

9. A device for removing artifacts from sparse angle CT images, characterized in that: include: A data preprocessing module is used to obtain a sparse angle CT image and a full angle CT image, and preprocess them to obtain a preprocessed sparse angle CT image and a full angle CT image; A network construction module is used to divide the preprocessed sparse angle CT image and full angle CT image into a number of training three-dimensional sub-blocks; and to construct a volume-based sparse angle CT image artifact removal network, the network comprising two pairs of generators and discriminators, one pair of which is used to transform the training three-dimensional sub-blocks from the sparse angle image domain to the full angle image domain, and the other pair is used to perform the reverse process; A network training module, used to train the network using the training three-dimensional sub-blocks, and during the training process, the network is optimized using a loss function including adversarial loss, cycle consistency loss and reconstruction loss to obtain a trained network; An image processing module, used for pre-processing the sparse angle CT image to be processed, dividing it into a number of to-be-processed three-dimensional sub-blocks of the same size as the three-dimensional sub-blocks in order of spatial position, and sequentially inputting the to-be-processed three-dimensional sub-blocks into the trained network for processing to obtain the three-dimensional sub-blocks after removing artifacts; The image synthesis module is used to merge the three-dimensional sub-blocks after the artifacts are removed to form a CT image after the artifacts are removed.

Citation Information

Patent Citations

  • Sparse angle CT artifact removal method

    CN114596378A

  • Thoracic cavity CT image artifact removal method and system based on three-dimensional generative adversarial network

    CN114897726A

  • Generative adversarial network-based artifact suppression method for extreme sparse view CT (Computed Tomography) reconstruction

    CN115239588A

  • Training and generation methods for generating high-energy CT image model, device, and storage medium

    WO2021189383A1

Cited By

  • Tumor image segmentation and processing method and system based on AI

    CN122368507A

  • AI-based tumor image segmentation and processing method and system

    CN122368507B