CBCT artifact correction system and method based on multi-stage reconstruction network

Abstract

The invention belongs to the technical field of medical image processing, and provides a CBCT artifact correction system and method based on a multi-stage reconstruction network. The system comprises an image preprocessing module, a model training module and an image generation module, wherein the image preprocessing module is used for reconstructing projection data acquired by CBCT scanning to obtain a reference CBCT and a sparse angle CBCT; the model training module is used for training a multi-stage mixed attention reconstruction network based on the sparse angle CBCT and the reference CBCT to obtain a sparse angle CBCT artifact correction model; and the image generation module is used for inputting the actual sparse angle CBCT into the sparse angle CBCT artifact correction model to generate a corrected CBCT. Through multi-stage fusion of the image domain and the projection domain, common stripe artifacts and noise in sparse angle reconstruction are effectively reduced, and the definition and clinical availability of CBCT are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a CBCT artifact correction system and method based on a multi-stage reconstruction network. Background Art

[0002] Computed tomography (CT) is an important medical imaging technique that can produce detailed images of internal structures. Conventional CT reconstructs cross-sectional images of the body's interior by acquiring multiple projection data using a rotating X-ray beam. Although CT is very effective in diagnosis, it has a high radiation dose, slow imaging speed, and is sensitive to motion artifacts. Cone-beam computed tomography (CBCT) is a modified form of CT that uses a cone-shaped X-ray beam to acquire data from the entire volume in a single rotation. CBCT not only has fast imaging speed and low radiation dose, but also can produce high-resolution three-dimensional images, so it is widely used in fields such as dentistry and oncology.

[0003] In order to reduce the radiation dose of CBCT and improve the imaging efficiency, technicians in this field have begun to pay attention to sparse angle CBCT imaging technology. However, due to insufficient acquisition angles, sparse angle CBCT imaging often produces artifacts, which seriously affects the image quality and diagnostic accuracy. Therefore, developing a method that can effectively process sparse view projection data and reconstruct high-quality CT images is a task of great clinical value but extremely challenging.

[0004] In recent years, deep learning technology has made significant progress and achievements in the field of sparse angle reconstruction. In the prior art, there are three main types of deep learning-based methods: First, single-domain image to image. Single-domain image to image mainly relies on the features of existing images for reconstruction, which is easily limited by the quality of the input image. When the input low-quality image has obvious artifacts, the reconstruction result may be further deteriorated. In addition, this method lacks full utilization of projection data, which may lead to information loss. Second, single-domain projection to image. Although single-domain projection to image can directly reconstruct images from projection data, under sparse view conditions, the projection data itself does not have enough information, which may cause the reconstructed image to be blurred and inaccurate. At the same time, this method has poor robustness in dealing with noise and artifacts, and it is often difficult to generate high-quality images. Third, dual-domain projection to image. Most of the existing dual-domain projection to image methods use interpolation methods to supplement the missing projections of sparse angle images, which will introduce noise and artifacts in the reconstruction process and cannot provide high-quality input images for subsequent image domain restoration networks.

[0005] In summary, it can be seen that there is an urgent need to provide a method that can more efficiently utilize dual-domain information to significantly improve the quality of sparse angle CBCT. Summary of the invention

[0006] Based on this, the present invention proposes a CBCT artifact correction system and method based on a multi-stage reconstruction network to solve the technical defects of the prior art. The system and method can make full use of the complementary information in the projection domain and the image domain, significantly improve the quality of sparse angle CBCT, and simplify the clinical diagnosis process, helping doctors to diagnose diseases more accurately and provide patients with better treatment options.

[0007] In a first aspect, the present invention provides a CBCT artifact correction system based on a multi-stage reconstruction network, the system comprising: an image preprocessing module, a model training module, and an image generation module, wherein: The image preprocessing module is used to preprocess the projection data acquired by the CBCT scan to obtain the reference CBCT and the sparse angle CBCT; The model training module is used to train a multi-stage hybrid attention reconstruction network based on the sparse angle CBCT and the reference CBCT to obtain a sparse angle CBCT artifact correction model; The image generation module is used to input the actual sparse angle CBCT into the sparse angle CBCT artifact correction model to generate a corrected CBCT.

[0008] In a possible implementation, the model training module includes an image domain restoration module, a projection domain enhancement module, and an image domain denoising module that are connected in sequence; The image domain restoration module includes a first multi-input mixed attention convolutional neural network; the projection domain enhancement module includes a serially connected forward projection module and a second multi-input mixed attention convolutional neural network; the image domain denoising module includes a serially connected FDK reconstruction module and a third multi-input mixed attention convolutional neural network.

[0009] In a possible implementation, the first multi-input mixed attention convolutional neural network, the second multi-input mixed attention convolutional neural network and the third multi-input mixed attention convolutional neural network each include a multi-input encoder, a shallow semantic information fusion attention module and a single output decoder.

[0010] In a possible implementation, the multi-input encoder includes a first convolution block, a second convolution block, a third convolution block, a first residual block, a second residual block, and a third residual block; The first convolution block is connected in series with the first residual block, the second convolution block is connected in series with the second residual block, and the third convolution block is connected in series with the third residual block; one output end of the first residual block is connected to the input end of the second residual block, and the other output end is connected to the shallow semantic information fusion attention module; one output end of the second residual block is connected to the input end of the third residual block, and the other output end is connected to the shallow semantic information fusion attention module; the output end of the third residual block is connected to the shallow semantic information fusion attention module; The first convolution block, the second convolution block and the third convolution block are all composed of a cascade combination of 3×3 convolutions and 1×1 convolutions; the first residual block, the second residual block and the third residual block are all composed of two 3×3 convolution blocks with residual connections and a Relu activation function.

[0011] In a possible implementation, the shallow semantic information fusion attention module includes a first resize module, a second resize module, a third resize module, a first hybrid attention module, a second hybrid attention module and a third hybrid attention module; The first hybrid attention module includes a first information fusion attention module, a first channel attention module and a first regulation module, the first information fusion attention module has three input terminals, and the input terminals of the first channel attention module and the first regulation module are connected to the third input terminal of the first information fusion attention module; The second hybrid attention module includes a second information fusion attention module, a second channel attention module and a second regulation module, the second information fusion attention module has three input terminals, and the input terminals of the second channel attention module and the second regulation module are connected to the third input terminal of the second information fusion attention module; The third hybrid attention module includes a third information fusion attention module, a third channel attention module and a third regulation module, wherein the third information fusion attention module has three input terminals, and the input terminals of the third channel attention module and the third regulation module are connected to the third input terminal of the third information fusion attention module; The output end of the first resize module is respectively connected to the third input end of the first information fusion attention module, the first input end of the second information fusion attention module and the first input end of the third information fusion attention module; the output end of the second resize module is respectively connected to the first input end of the first information fusion attention module, the third input end of the second information fusion attention module and the second input end of the third information fusion attention module; the output end of the third resize module is respectively connected to the second input end of the first information fusion attention module, the second input end of the second information fusion attention module and the third input end of the third information fusion attention module.

[0012] In a possible implementation, the output feature of the shallow semantic information fusion attention module is: , , , in, represents the output features of the first hybrid attention module, represents the output features of the second hybrid attention module, Represents the output features of the third hybrid attention module; represents the output features of the first information fusion attention module; represents the output features of the second information fusion attention module; represents the output features of the third information fusion attention module; represents the output features of the first channel attention module; Represents the output features of the second channel attention module; Represents the output features of the third channel attention module; represents the output features of the first regulatory module; represents the output features of the second regulatory module; represents the output features of the third regulatory module; , , , , , , , , , SoftMax Represents the SoftMax activation function; Represents the output of the first resize module; Represents the output of the second resize module; Represents the output of the third resize module; and Respectively represent upsampling and downsampling; Conv represents convolution; MaxPool Represents the maximum pooling function; AvgPool represents the average pooling function; ReLU Represents the ReLU activation function.

[0013] In a possible implementation, the single output decoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a first residual group, a second residual group, a third residual group, and a multi-scale supervision mechanism; The first residual group is serially connected to the first convolutional layer, the second residual group is serially connected to the second convolutional layer, the third residual group is serially connected to the third convolutional layer, and the output end of the first convolutional layer is connected to the input end of the second residual group, and the output end of the second convolutional layer is connected to the input end of the third residual group.

[0014] In a second aspect, the present invention provides a CBCT artifact correction method based on the CBCT artifact correction system based on the multi-stage reconstruction network described in the first aspect, the method comprising: Preprocessing the projection data acquired by CBCT scanning to obtain reference CBCT and sparse angle CBCT; Based on the sparse angle CBCT and the reference CBCT, a multi-stage hybrid attention reconstruction network is trained to obtain a sparse angle CBCT artifact correction model; The actual sparse angle CBCT is input into the sparse angle CBCT artifact correction model to generate a corrected CBCT.

[0015] In a possible implementation, the multi-stage hybrid attention reconstruction network is trained based on the sparse angle CBCT and the reference CBCT to obtain a sparse angle CBCT artifact correction model, specifically: Inputting the sparse angle CBCT into an image domain restoration module, training a first multi-input mixed attention convolutional neural network, and obtaining a trained first multi-input mixed attention convolutional neural network; Based on the trained first multi-input hybrid attention convolutional neural network, an image domain restoration module preliminarily removes artifacts of the sparse angle CBCT and restores a clean sparse angle CBCT; Inputting the clean sparse angle CBCT into a projection domain enhancement module, and forward-projecting the clean sparse angle CBCT using a forward projection module to obtain a projection image; Using the projection image, training a second multi-input mixed attention convolutional neural network to obtain a trained second multi-input mixed attention convolutional neural network; Performing projection domain enhancement on the projection image based on the trained second multi-input mixed attention convolutional neural network to obtain a projection domain enhanced projection image; Inputting the projection domain enhanced projection image into an image domain denoising module, and reconstructing the projection domain enhanced projection image through an FDK reconstruction module to obtain a reconstructed CBCT; The reconstructed CBCT is used to train a third multi-input mixed attention convolutional neural network to obtain a trained third multi-input mixed attention convolutional neural network, and a sparse angle CBCT artifact correction model is obtained.

[0016] In a possible implementation, the training of the first multi-input mixed attention convolutional neural network, the training of the second multi-input mixed attention convolutional neural network, and the training of the third multi-input mixed attention convolutional neural network are specifically: Inputting the input image into a multi-input encoder, the multi-input encoder extracts features from the input image to generate a first low-size feature map, a second low-size feature map, and a third low-size feature map; Inputting the first low-size feature map into a first convolution block, and then passing through a first residual block to obtain a first intermediate feature map; Inputting the second low-size feature map into a second convolution block, and then inputting the output features of the second convolution block and the first intermediate feature map into a second residual block to obtain a second intermediate feature map; Inputting the third low-size feature map into a third convolution block, and then inputting the output features of the third convolution block and the second intermediate feature map into a third residual block to obtain a third intermediate feature map; Inputting the first intermediate feature map, the second intermediate feature map and the third intermediate feature map into the shallow semantic information fusion attention module respectively to obtain a first enhanced intermediate feature map, a second enhanced intermediate feature map and a third enhanced intermediate feature map; Inputting the third enhanced intermediate feature map into the first residual group for refinement, and then passing through the first convolution layer to obtain a first large-size feature map; Inputting the first large-size feature map and the second enhanced intermediate feature map into a second residual group for refinement, and then passing through a second convolutional layer to obtain a second large-size feature map; Inputting the second largest size feature map and the first enhanced intermediate feature map into the third residual group for refinement, and then passing through the third convolution layer to obtain an output image; The multi-scale supervision mechanism supervises the output features of each stage step by step based on the reference CBCT. When the multi-scale supervision mechanism stabilizes at a lower value after multiple iterations and no longer decreases significantly, it means that the network training is completed.

[0017] The present invention constructs a multi-stage hybrid attention reconstruction network to better deal with the problem of missing information in sparse angle CBCT reconstruction. The network not only reconstructs in the image domain, but also introduces a collaborative optimization mechanism between the image domain and the projection domain in multiple stages, integrating superficial and semantic information, thereby achieving remarkable results in restoring global structure and detail texture. Specifically, the multi-stage hybrid attention reconstruction network of the present invention is divided into three main modules: the image domain restoration module uses sparse projection data to obtain a preliminary reconstructed image through a convolutional neural network; the projection domain enhancement module uses a multi-scale input strategy and a hybrid attention mechanism to extract supplementary information from the projection domain to further improve image quality; the image domain denoising module combines a multi-scale frequency domain supervision mechanism to effectively remove residual noise and artifacts, and improves the contrast and clarity of the reconstructed image. Compared with the prior art, the beneficial effects of the present invention are: (1) The present invention effectively reduces the streak artifacts and noise commonly seen in sparse angle reconstruction through multi-stage fusion of the image domain and the projection domain, thereby improving the clarity and clinical usability of CBCT.

[0018] (2) The present invention adopts multi-scale input and frequency domain supervision strategies, so that both low-frequency and high-frequency information are optimized, further enhancing the structural details and texture expression.

[0019] (3) The present invention integrates a shallow semantic information fusion attention module, which can efficiently filter and fuse key information between the image domain and the projection domain, effectively improving the reconstruction accuracy of sparse data and ensuring the reliability and integrity of the image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 A structural block diagram of a CBCT artifact correction system based on a multi-stage reconstruction network provided in an embodiment of the present invention; Figure 2A schematic diagram of the structure and workflow of a multi-input mixed attention convolutional neural network provided in an embodiment of the present invention; Figure 3 The structure and workflow diagram of the shallow semantic information fusion attention module provided by the embodiment of the present invention; Figure 4 The structure and workflow diagram of the hybrid attention module provided by the embodiment of the present invention; Figure 5 A schematic diagram of a flow chart of a CBCT artifact correction method based on a multi-stage reconstruction network provided in an embodiment of the present invention; Figure 6 A comparison diagram of the sparse angle CBCT / extreme sparse angle CBCT, reference CBCT and corrected CBCT provided in an embodiment of the present invention; Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to better understand the technical solution of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0023] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0025] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0026] See also Figure 1 , is a structural block diagram of a CBCT artifact correction system based on a multi-stage reconstruction network provided by an embodiment of the present invention. Figure 1 As shown, the system includes: an image preprocessing module 101, a model training module 102, and an image generation module 103. Among them, The image preprocessing module 101 is used to preprocess the projection data acquired by CBCT scanning to obtain reference CBCT and sparse angle CBCT. Specifically: The projection data acquired by the CBCT scan are reconstructed at 600 and 150 angles respectively to obtain a reference CBCT and a sparse angle CBCT; the sparse angle CBCT and the reference CBCT are paired one by one to obtain an image data pair.

[0027] Optionally, the sparse angle CBCT may be replaced by an extreme sparse angle CBCT, which is obtained by reconstructing projection data acquired by CBCT scanning at 40 angles.

[0028] The model training module 102 is used to train a multi-stage hybrid attention reconstruction network based on the sparse angle CBCT and the reference CBCT to obtain a sparse angle CBCT artifact correction model.

[0029] The image generation module 103 is used to input the actual sparse angle CBCT into the sparse angle CBCT artifact correction model to generate a corrected CBCT.

[0030] Specifically, the model training module 102 includes an image domain restoration module, a projection domain enhancement module and an image domain denoising module which are connected in sequence. The image domain restoration module is used to preliminarily remove artifacts of the sparse angle CBCT and restore a clean sparse angle CBCT.

[0031] The image domain restoration module includes a first multi-input mixed attention convolutional neural network.

[0032] The projection domain enhancement module is used to forward project the clean sparse angle CBCT to obtain a projection image; fit the projection image and the sparse angle CBCT to eliminate the difference in details between the projection image and the sparse angle CBCT as much as possible; The projection domain enhancement module includes a forward projection module and a second multi-input mixed attention convolutional neural network connected in series.

[0033] The forward projection module is used to forward project the clean sparse angle CBCT through a forward projection algorithm to obtain a projection image.

[0034] The image domain denoising module is used to remove errors generated during image domain restoration, projection domain enhancement and reconstruction.

[0035] The image domain denoising module includes a serially connected FDK reconstruction module and a third multi-input mixed attention convolutional neural network.

[0036] The FDK reconstruction module is used to reconstruct the output image of the image domain denoising module into a reconstructed CBCT by using the FDK algorithm.

[0037] Furthermore, the first multi-input mixed attention convolutional neural network, the second multi-input mixed attention convolutional neural network and the third multi-input mixed attention convolutional neural network have the same network structure, and are all used to combine information from different input sources, using a mixed attention mechanism to focus on important features and suppress irrelevant noise, ultimately improving the effect of CBCT reconstruction.

[0038] See also Figure 2 , is a schematic diagram of the structure and workflow of a multi-input hybrid attention convolutional neural network provided by an embodiment of the present invention. Figure 2 As shown, the first multi-input mixed attention convolutional neural network, the second multi-input mixed attention convolutional neural network and the third multi-input mixed attention convolutional neural network all include a multi-input encoder, a shallow semantic information fusion attention module and a single output decoder. The multi-input encoder is used to process multiple downsampled CBCT (or projection images) and extract richer feature representations by processing these images in parallel.

[0039] Preferably, in this embodiment, the multi-input encoder performs three-layer downsampling on the input image, extracts feature images of three different scales, and inputs the three different scale feature images into the input ends of the corresponding scales of the shallow semantic information fusion attention module respectively to obtain multi-scale context feature representation.

[0040] The multi-input encoder includes 3 convolution blocks and 3 residual blocks. The convolution block is composed of a cascade of 3×3 convolutions and 1×1 convolutions, and is used to extract shallow features of the input image, and combine the multi-scale features to input the shallow semantic information fusion attention module to the input end of the corresponding scale; The residual block consists of two 3×3 convolution blocks connected by residuals and a Relu activation function, which is used to prevent the gradient from disappearing during network training and integrate shallow features and multi-scale features.

[0041] Specifically, in this embodiment, the multi-input encoder includes a first convolution block, a second convolution block, a third convolution block, a first residual block, a second residual block and a third residual block. The first convolution block is connected in series with the first residual block, the second convolution block is connected in series with the second residual block, and the third convolution block is connected in series with the third residual block; one output end of the first residual block is connected to the input end of the second residual block, and the other output end is connected to the shallow semantic information fusion attention module; one output end of the second residual block is connected to the input end of the third residual block, and the other output end is connected to the shallow semantic information fusion attention module; the output end of the third residual block is connected to the shallow semantic information fusion attention module.

[0042] See also Figure 3 , which is a structure and workflow diagram of the shallow semantic information fusion attention module provided by an embodiment of the present invention; see Figure 4 , which is a structure and workflow diagram of the hybrid attention module provided by an embodiment of the present invention. Figure 3-Figure 4 As shown, the shallow semantic information fusion attention module includes a first resize module, a second resize module, a third resize module, a first hybrid attention module, a second hybrid attention module and a third hybrid attention module.

[0043] The first hybrid attention module includes a first information fusion attention module, a first channel attention module and a first regulation module. The first information fusion attention module has three input ends. The input ends of the first channel attention module and the first regulation module are connected to the third input end of the first information fusion attention module, so that the first channel attention module, the first regulation module and the third input end of the first information fusion attention module are connected in parallel.

[0044] The second hybrid attention module includes a second information fusion attention module, a second channel attention module and a second regulation module. The second information fusion attention module has three input ends. The input ends of the second channel attention module and the second regulation module are connected to the third input end of the second information fusion attention module, so that the second channel attention module, the second regulation module and the third input end of the second information fusion attention module are connected in parallel.

[0045] The third hybrid attention module includes a third information fusion attention module, a third channel attention module and a third regulation module. The third information fusion attention module has three input ends. The input ends of the third channel attention module and the third regulation module are connected to the third input end of the third information fusion attention module, so that the third channel attention module, the third regulation module and the third input end of the third information fusion attention module are connected in parallel.

[0046] The information fusion attention module is used to explore the anatomical structure connection between shallow superficial information and deep semantic information from the feature space, and establish a long-distance dependency relationship between the two, which helps to capture fine anatomical details and structures.

[0047] The channel attention module is used to compress the information of the feature channel, thereby covering the information of the entire image, which is conducive to capturing the global features of the image.

[0048] Input ends of the first resizing module, the second resizing module and the third resizing module are connected to output ends of the first residual block, the second residual block and the third residual block respectively.

[0049] The output end of the first resize module is respectively connected to the third input end of the first information fusion attention module, the first input end of the second information fusion attention module and the first input end of the third information fusion attention module; the output end of the second resize module is respectively connected to the first input end of the first information fusion attention module, the third input end of the second information fusion attention module and the second input end of the third information fusion attention module; the output end of the third resize module is respectively connected to the second input end of the first information fusion attention module, the second input end of the second information fusion attention module and the third input end of the third information fusion attention module.

[0050] The output features of the first information fusion attention module for: , The output features of the second information fusion attention module for: , The output features of the third information fusion attention module for: , in, SoftMax Represents the SoftMax activation function; Indicates m The output of the resize module, m =1,2,3; and represent upsampling and downsampling respectively.

[0051] The output features of the first channel attention module for: , The output features of the second channel attention module for: , The output features of the third channel attention module for: , in, Indicates m The output of the resize module, m =1,2,3; Conv represents convolution; MaxPool Represents the maximum pooling function; AvgPool Represents the average pooling function.

[0052] It should be noted that the channel attention module of this embodiment does not process the results of the maximum pooling operation and the average pooling operation serially, but uses parallel processing to multiply the two to enhance the specificity of information expression between different channels and reduce the impact of irrelevant global information on network training.

[0053] The first regulation module, the second regulation module and the third regulation module are all composed of a series of 3×3 convolution and ReLU activation functions.

[0054] The control module is used to initialize a learnable parameter so that the shallow semantic information fusion attention module has the ability to adaptively perceive local or global features to avoid conflicts between the information fusion attention module and the channel attention module during the feature extraction process.

[0055] The output characteristics of the first control module for: , The output characteristics of the second control module for: , The output characteristics of the third control module for: , in, Indicates m The output of the resize module, m =1,2,3; Conv represents convolution; ReLU Represents the ReLU activation function.

[0056] The output features of the first hybrid attention module for: , The output features of the second hybrid attention module for: , The output features of the third hybrid attention module for: , The single output decoder is used to integrate the scale features to generate the corrected CBCT.

[0057] The single-output decoder consists of three convolutional layers, three residual groups, and a multi-scale supervision mechanism.

[0058] Specifically, in this embodiment, the single output decoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a first residual group, a second residual group and a third residual group. The first residual group is serially connected to the first convolutional layer, the second residual group is serially connected to the second convolutional layer, the third residual group is serially connected to the third convolutional layer, and the output end of the first convolutional layer is connected to the input end of the second residual group, and the output end of the second convolutional layer is connected to the input end of the third residual group.

[0059] The convolution layer is used to preliminarily integrate scale features and perform upsampling operations on low-scale feature maps.

[0060] The residual group is used to further extract and enhance features, thereby improving the representation capability of the image.

[0061] The residual group is composed of four residual blocks connected in series, each of which contains two convolutional layers and a jump connection to retain the original information of the input features and reduce the risk of gradient vanishing. By connecting four residual blocks in series, the single output decoder can gradually refine features and capture image information of different scales to achieve a more sophisticated artifact correction effect.

[0062] The multi-scale supervision mechanism is used to supervise the output of the single-output decoder at different scales, so that the single-output decoder can take into account both global and local features during the training process, thereby generating high-quality corrected CBCT. This multi-scale supervision strategy not only improves image detail restoration and artifact removal, but also enhances the robustness and generalization ability of the model.

[0063] Based on the above embodiments, this embodiment provides a CBCT artifact correction method based on a multi-stage reconstruction network. Figure 5 , is a flow chart of a CBCT artifact correction method based on a multi-stage reconstruction network provided by an embodiment of the present invention. Figure 5 As shown, the method comprises the following steps: Step S1, preprocessing the projection data acquired by CBCT scanning to obtain reference CBCT and sparse angle CBCT. Specifically: The projection data acquired by the CBCT scan are reconstructed at 600 and 150 angles respectively to obtain a reference CBCT and a sparse angle CBCT; the sparse angle CBCT and the reference CBCT are paired one by one to obtain an image data pair.

[0064] Step S2, training a multi-stage hybrid attention reconstruction network based on the sparse angle CBCT and the reference CBCT to obtain a sparse angle CBCT artifact correction model. Specifically: Step S21, inputting the sparse angle CBCT into an image domain restoration module, training a first multi-input mixed attention convolutional neural network, and obtaining a trained first multi-input mixed attention convolutional neural network; Step S22, based on the trained first multi-input hybrid attention convolutional neural network, the image domain restoration module preliminarily removes artifacts of the sparse angle CBCT to restore a clean sparse angle CBCT; Step S23, inputting the clean sparse angle CBCT into the projection domain enhancement module, and forward projecting the clean sparse angle CBCT using the forward projection module to obtain a projection image; Step S24, using the projection image to train a second multi-input mixed attention convolutional neural network to obtain a trained second multi-input mixed attention convolutional neural network; Step S25, performing projection domain enhancement on the projection image based on the trained second multi-input mixed attention convolutional neural network to obtain a projection domain enhanced projection image; Step S26, inputting the projection domain enhanced projection image into the image domain denoising module, and reconstructing the projection domain enhanced projection image through the FDK reconstruction module to obtain a reconstructed CBCT; Step S27, using the reconstructed CBCT, training a third multi-input mixed attention convolutional neural network to obtain a trained third multi-input mixed attention convolutional neural network, and obtaining a sparse angle CBCT artifact correction model.

[0065] See also Figure 2 , the training of the first multi-input mixed attention convolutional neural network, the training of the second multi-input mixed attention convolutional neural network, and the training of the third multi-input mixed attention convolutional neural network are specifically: Inputting the input image into a multi-input encoder, the multi-input encoder extracts features from the input image to generate a first low-size feature map, a second low-size feature map, and a third low-size feature map; Inputting the first low-size feature map into a first convolution block, and then passing through a first residual block to obtain a first intermediate feature map; Inputting the second low-size feature map into a second convolution block, and then inputting the output features of the second convolution block and the first intermediate feature map into a second residual block to obtain a second intermediate feature map; Inputting the third low-size feature map into a third convolution block, and then inputting the output features of the third convolution block and the second intermediate feature map into a third residual block to obtain a third intermediate feature map; Inputting the first intermediate feature map, the second intermediate feature map and the third intermediate feature map into the shallow semantic information fusion attention module respectively to obtain a first enhanced intermediate feature map, a second enhanced intermediate feature map and a third enhanced intermediate feature map; Inputting the third enhanced intermediate feature map into the first residual group for refinement, and then passing through the first convolution layer to obtain a first large-size feature map; Inputting the first large-size feature map and the second enhanced intermediate feature map into a second residual group for refinement, and then passing through a second convolutional layer to obtain a second large-size feature map; The second largest size feature map and the first enhanced intermediate feature map are input into the third residual group for refinement, and then passed through the third convolution layer to obtain an output image.

[0066] During the training process, the multi-scale supervision mechanism supervises the output of each stage step by step based on the reference CBCT, so that the model can effectively learn the ability of artifact correction at each level. When the multi-scale supervision mechanism stabilizes at a lower value after multiple iterations and no longer decreases significantly, it means that the network training is completed.

[0067] It should be noted that when the first multi-input mixed attention convolutional neural network is trained, the input image is a sparse angle CBCT; when the second multi-input mixed attention convolutional neural network is trained, the input image is a projection image; when the third multi-input mixed attention convolutional neural network is trained, the input image is a reconstructed CBCT.

[0068] S3: Inputting the actual sparse angle CBCT into the sparse angle CBCT artifact correction model to generate a corrected CBCT.

[0069] See also Figure 6 , which is a comparison diagram of the sparse angle CBCT / extreme sparse CBCT, the reference CBCT, and the corrected CBCT provided in the embodiment of the present invention. Figure 6 As shown, it can be clearly seen that the images generated by the system and method provided by the embodiments of the present disclosure are very similar to the reference CBCT, which proves the effectiveness of the system and method provided by the embodiments of the present disclosure.

[0070] Corresponding to the above embodiment, an embodiment of the present invention further provides an electronic device.

[0071] See also Figure 7 , is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 7 As shown, the electronic device 700 may include: a processor 701, a memory 702 and a communication unit 703. These components communicate via one or more buses. Those skilled in the art will appreciate that the electronic device structure shown in the figure does not limit the embodiments of the present invention. It may be a bus structure or a star structure, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0072] The communication unit 703 is used to establish a communication channel so that the electronic device can communicate with other devices.

[0073] The processor 701 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It runs or executes software programs and / or modules stored in the memory 702, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 701 can include only a central processing unit (CPU). In an embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0074] The memory 702 is used to store the execution instructions of the processor 701. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0075] When the execution instructions in the memory 702 are executed by the processor 701 , the electronic device 700 is enabled to execute part or all of the steps in the above method embodiments.

[0076] Corresponding to the above embodiment, the embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium may store a program, wherein when the program is running, the device where the computer-readable storage medium is located may be controlled to execute some or all of the steps in the above method embodiment. In a specific implementation, the computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0077] Corresponding to the above embodiment, an embodiment of the present invention further provides a computer program product, which includes executable instructions. When the executable instructions are executed on a computer, the computer executes part or all of the steps in the above method embodiment.

[0078] In the embodiments of the present invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. A and B may be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c may be single or multiple.

[0079] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0080] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0081] In several embodiments provided by the present invention, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0082] The above description is only a specific implementation of the present invention. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention.

Claims

1. A CBCT artifact correction system based on a multi-stage reconstruction network, characterized in that: The system includes: an image preprocessing module, a model training module, and an image generation module, wherein: The image preprocessing module is used to preprocess the projection data acquired by the CBCT scan to obtain the reference CBCT and the sparse angle CBCT; The model training module is used to train a multi-stage hybrid attention reconstruction network based on the sparse angle CBCT and the reference CBCT to obtain a sparse angle CBCT artifact correction model; The image generation module is used to input the actual sparse angle CBCT into the sparse angle CBCT artifact correction model to generate a corrected CBCT.

2. The CBCT artifact correction system based on the multi-stage reconstruction network according to claim 1, characterized in that: The model training module includes an image domain restoration module, a projection domain enhancement module and an image domain denoising module connected in sequence; The image domain restoration module includes a first multi-input mixed attention convolutional neural network; the projection domain enhancement module includes a serially connected forward projection module and a second multi-input mixed attention convolutional neural network; the image domain denoising module includes a serially connected FDK reconstruction module and a third multi-input mixed attention convolutional neural network.

3. The CBCT artifact correction system based on the multi-stage reconstruction network according to claim 2, characterized in that: The first multi-input mixed attention convolutional neural network, the second multi-input mixed attention convolutional neural network and the third multi-input mixed attention convolutional neural network each include a multi-input encoder, a shallow semantic information fusion attention module and a single output decoder.

4. The CBCT artifact correction system based on the multi-stage reconstruction network according to claim 3, characterized in that: The multi-input encoder includes a first convolution block, a second convolution block, a third convolution block, a first residual block, a second residual block and a third residual block; The first convolution block is connected in series with the first residual block, the second convolution block is connected in series with the second residual block, and the third convolution block is connected in series with the third residual block; one output end of the first residual block is connected to the input end of the second residual block, and the other output end is connected to the shallow semantic information fusion attention module; One output end of the second residual block is connected to the input end of the third residual block, and the other output end is connected to the shallow semantic information fusion attention module; The output end of the third residual block is connected to the shallow semantic information fusion attention module; The first convolution block, the second convolution block and the third convolution block are all composed of a cascade combination of 3×3 convolutions and 1×1 convolutions; the first residual block, the second residual block and the third residual block are all composed of two 3×3 convolution blocks with residual connections and a Relu activation function.

5. The CBCT artifact correction system based on the multi-stage reconstruction network according to claim 3, characterized in that: The shallow semantic information fusion attention module includes a first resize module, a second resize module, a third resize module, a first hybrid attention module, a second hybrid attention module and a third hybrid attention module; The first hybrid attention module includes a first information fusion attention module, a first channel attention module and a first regulation module, the first information fusion attention module has three input terminals, and the input terminals of the first channel attention module and the first regulation module are connected to the third input terminal of the first information fusion attention module; The second hybrid attention module includes a second information fusion attention module, a second channel attention module and a second regulation module, the second information fusion attention module has three input terminals, and the input terminals of the second channel attention module and the second regulation module are connected to the third input terminal of the second information fusion attention module; The third hybrid attention module includes a third information fusion attention module, a third channel attention module and a third regulation module, wherein the third information fusion attention module has three input terminals, and the input terminals of the third channel attention module and the third regulation module are connected to the third input terminal of the third information fusion attention module; The output end of the first resize module is respectively connected to the third input end of the first information fusion attention module, the first input end of the second information fusion attention module and the first input end of the third information fusion attention module; the output end of the second resize module is respectively connected to the first input end of the first information fusion attention module, the third input end of the second information fusion attention module and the second input end of the third information fusion attention module; the output end of the third resize module is respectively connected to the second input end of the first information fusion attention module, the second input end of the second information fusion attention module and the third input end of the third information fusion attention module.

6. The CBCT artifact correction system based on the multi-stage reconstruction network according to claim 5, characterized in that: The output features of the shallow semantic information fusion attention module are: ， ， ， in, represents the output features of the first hybrid attention module, represents the output features of the second hybrid attention module, Represents the output features of the third hybrid attention module; represents the output features of the first information fusion attention module; represents the output features of the second information fusion attention module; represents the output features of the third information fusion attention module; represents the output features of the first channel attention module; Represents the output features of the second channel attention module; Represents the output features of the third channel attention module; represents the output features of the first regulatory module; represents the output features of the second regulatory module; represents the output features of the third regulatory module; ， ， ， ， ， ， ， ， ， SoftMax Represents the SoftMax activation function; Represents the output of the first resize module; Represents the output of the second resize module; Represents the output of the third resize module; and Respectively represent upsampling and downsampling; Conv represents convolution; MaxPool Represents the maximum pooling function; AvgPool represents the average pooling function; ReLU Represents the ReLU activation function.

7. The CBCT artifact correction system based on a multi-stage reconstruction network according to claim 4, characterized in that: The single output decoder comprises a first convolutional layer, a second convolutional layer, a third convolutional layer, a first residual group, a second residual group, a third residual group and a multi-scale supervision mechanism; The first residual group is serially connected to the first convolutional layer, the second residual group is serially connected to the second convolutional layer, the third residual group is serially connected to the third convolutional layer, and the output end of the first convolutional layer is connected to the input end of the second residual group, and the output end of the second convolutional layer is connected to the input end of the third residual group.

8. A CBCT artifact correction method based on the CBCT artifact correction system based on a multi-stage reconstruction network according to any one of claims 1 to 7, characterized in that: The method comprises: Preprocessing the projection data acquired by CBCT scanning to obtain reference CBCT and sparse angle CBCT; Based on the sparse angle CBCT and the reference CBCT, a multi-stage hybrid attention reconstruction network is trained to obtain a sparse angle CBCT artifact correction model; The actual sparse angle CBCT is input into the sparse angle CBCT artifact correction model to generate a corrected CBCT.

9. The CBCT artifact correction method according to claim 8, characterized in that: The multi-stage hybrid attention reconstruction network is trained based on the sparse angle CBCT and the reference CBCT to obtain a sparse angle CBCT artifact correction model, specifically: Inputting the sparse angle CBCT into an image domain restoration module, training a first multi-input mixed attention convolutional neural network, and obtaining a trained first multi-input mixed attention convolutional neural network; Based on the trained first multi-input hybrid attention convolutional neural network, an image domain restoration module preliminarily removes artifacts of the sparse angle CBCT and restores a clean sparse angle CBCT; Inputting the clean sparse angle CBCT into a projection domain enhancement module, and forward-projecting the clean sparse angle CBCT using a forward projection module to obtain a projection image; Using the projection image, training a second multi-input mixed attention convolutional neural network to obtain a trained second multi-input mixed attention convolutional neural network; Performing projection domain enhancement on the projection image based on the trained second multi-input mixed attention convolutional neural network to obtain a projection domain enhanced projection image; Inputting the projection domain enhanced projection image into an image domain denoising module, and reconstructing the projection domain enhanced projection image through an FDK reconstruction module to obtain a reconstructed CBCT; The reconstructed CBCT is used to train a third multi-input mixed attention convolutional neural network to obtain a trained third multi-input mixed attention convolutional neural network, and a sparse angle CBCT artifact correction model is obtained.

10. The CBCT artifact correction method according to claim 9, characterized in that: The training of the first multi-input mixed attention convolutional neural network, the training of the second multi-input mixed attention convolutional neural network, and the training of the third multi-input mixed attention convolutional neural network are specifically: Inputting the input image into a multi-input encoder, the multi-input encoder extracts features from the input image to generate a first low-size feature map, a second low-size feature map, and a third low-size feature map; Inputting the first low-size feature map into a first convolution block, and then passing through a first residual block to obtain a first intermediate feature map; Inputting the second low-size feature map into a second convolution block, and then inputting the output features of the second convolution block and the first intermediate feature map into a second residual block to obtain a second intermediate feature map; Inputting the third low-size feature map into a third convolution block, and then inputting the output features of the third convolution block and the second intermediate feature map into a third residual block to obtain a third intermediate feature map; Inputting the first intermediate feature map, the second intermediate feature map and the third intermediate feature map into the shallow semantic information fusion attention module respectively to obtain a first enhanced intermediate feature map, a second enhanced intermediate feature map and a third enhanced intermediate feature map; Inputting the third enhanced intermediate feature map into the first residual group for refinement, and then passing through the first convolution layer to obtain a first large-size feature map; Inputting the first large-size feature map and the second enhanced intermediate feature map into a second residual group for refinement, and then passing through a second convolutional layer to obtain a second large-size feature map; Inputting the second largest size feature map and the first enhanced intermediate feature map into the third residual group for refinement, and then passing through the third convolution layer to obtain an output image; The multi-scale supervision mechanism supervises the output features of each stage step by step based on the reference CBCT. When the multi-scale supervision mechanism stabilizes at a lower value after multiple iterations and no longer decreases significantly, it means that the network training is completed.

Images