A four-dimensional computed tomography hybrid imaging method

Abstract

The application provides a four-dimensional computer tomography hybrid imaging method, which can be applied to the technical field of medical imaging and other technologies requiring X-ray CT imaging. The method comprises the following steps: performing image reconstruction on projections of a target object obtained through four-dimensional computer tomography by using a filtered back projection algorithm to obtain a first reconstructed image; performing image reconstruction on the first reconstructed image based on an error minimization algorithm by using a trained hybrid imaging neural network to obtain a second reconstructed image, and performing elastic registration and motion compensation between the second reconstructed images to obtain a third reconstructed image; and performing iterative image reconstruction based on bone weighting on the third reconstructed image to obtain a four-dimensional computer tomography hybrid imaging result of the target object. The application also provides a four-dimensional computer tomography hybrid imaging device, an electronic device and a storage medium.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical imaging technology, in particular to a four-dimensional computed tomography hybrid imaging method, an electronic device and a storage medium. BACKGROUND

[0002] Computed Tomography (CT), including Cone Beam CT (CBCT), is an important tool in clinical applications and preclinical studies. It is widely used in clinical diagnosis, image-guided radiotherapy, and preclinical in vivo animal imaging scenarios. However, respiratory motion during imaging will cause motion blur and strip artifacts in chest and abdominal CT images. Four-dimensional CT (4DCT) technology can effectively improve the above problems: it can classify X-ray projections (hereinafter referred to as projections) containing organ motion information into different phases, and then reconstruct images for each phase. This can effectively reduce the artifacts caused by organ motion and provide high-quality time-dependent dynamic image sequences. Traditional 4DCT imaging usually uses a slow gantry rotation or repeated multi-turn scanning method to provide sufficient projection data for each respiratory phase. This method can generate high-quality 4DCT images, but at the cost of significantly prolonged scanning time and increased radiation dose. If the same scanning protocol as conventional static three-dimensional CT (3DCT) imaging is used to obtain 4DCT images, when using the standard filtered back-projection algorithm for image reconstruction, the image quality will be affected and severe strip artifacts will be produced due to insufficient sampling of projection data available for each phase.

[0003] In recent years, deep learning technology has become a breakthrough method in the field of low-dose CT image reconstruction. On clinical CT datasets, deep learning-based reconstruction methods have shown superior image quality to traditional algorithms such as compressed sensing, motion compensation, etc. However, deep learning-based reconstruction methods usually require a large amount of dataset for training the neural network. In addition, in preclinical studies, the imaging parameters differ greatly between different micro-CT instruments, such as spatial resolution and X-ray energy used, making it difficult to construct a large-scale and universal preclinical animal dataset, often requiring the construction of customized datasets for specific instruments; further, the acquisition of large quantities of customized datasets is difficult, and the data size is much smaller than the data volume in clinical scenarios. When the training data is limited, the stability and generalization of deep learning methods on small-scale datasets will be greatly limited, and the reconstruction effect will be greatly compromised. SUMMARY

[0004] In view of the above problems, the present application provides a four-dimensional computed tomography hybrid imaging method, an electronic device and a storage medium, which at least solve one of the problems of the prior art.

[0005] According to a first aspect of the present application, a four-dimensional computed tomography hybrid imaging method is provided, comprising:

[0006] Image reconstruction is performed on projections of a target object acquired by four-dimensional computed tomography using a filtered back-projection algorithm to obtain a first reconstructed image;

[0007] Image reconstruction is performed on the first reconstructed image based on an error minimization algorithm using a trained hybrid imaging neural network to obtain a second reconstructed image, and elastic registration and motion compensation are performed between the second reconstructed images to obtain a third reconstructed image, wherein the trained hybrid imaging neural network is obtained by phase classification and image reconstruction based on the filtered back-projection algorithm on four-dimensional computed tomography projection samples to obtain reference image samples and input image samples having a paired relationship, and the hybrid imaging neural network is trained using the reference image samples and the input image samples;

[0008] Iterative image reconstruction based on bone weighting is performed on the third reconstructed image to obtain a four-dimensional computed tomography hybrid imaging result of the target object.

[0009] According to an embodiment of the present application, the phase classification and image reconstruction based on the filtered back-projection algorithm on four-dimensional computed tomography projection samples to obtain reference image samples and input image samples having a paired relationship comprises:

[0010] Phase classification is performed on four-dimensional computed tomography projection samples of an experimental organism based on physiological information of the experimental organism to obtain phase classified image samples;

[0011] Image reconstruction based on the filtered back-projection algorithm is performed on the phase classified projection image samples to obtain reference image samples;

[0012] Four-dimensional computed tomography projection samples corresponding to the reference image samples are down-sampled to obtain down-sampled projection samples;

[0013] Image reconstruction based on the filtered back-projection algorithm is performed on the down-sampled image samples to obtain input image samples having strip artifacts.

[0014] According to an embodiment of the present application, the training of the hybrid imaging neural network using the reference image samples and the input image samples comprises:

[0015] The hybrid imaging neural network is constructed using a three-dimensional convolution layer and a plurality of dense convolution modules having a stacking function;

[0016] The hybrid imaging neural network is trained after normalization and preprocessing of the reference image sample and the input image sample with the strip artifact, and a trained hybrid imaging neural network with strip artifact elimination capability is obtained.

[0017] According to the embodiment of the present application, the image reconstruction of the projection of the target object obtained by the four-dimensional computed tomography by using the filtered back projection algorithm to obtain the first reconstructed image includes:

[0018] The four-dimensional computed tomography projection of the target object is acquired by using the computed tomography imaging device.

[0019] The four-dimensional computed tomography projection of the target object is phase classified according to the physiological information of the target object to obtain the phase classified image.

[0020] The image reconstruction of the phase classified projection by using the filtered back projection algorithm obtains the first reconstructed image with strip artifacts.

[0021] According to the embodiment of the present application, the image reconstruction of the first reconstructed image based on the error minimization algorithm by using the trained hybrid imaging neural network to obtain the second reconstructed image includes:

[0022] The strip artifact of the first reconstructed image is eliminated by using the trained hybrid imaging neural network to obtain the artifact-free image.

[0023] The artifact-free image is forward projected to the angle corresponding to each respiratory phase in the four-dimensional computed tomography projection by using the forward projection algorithm, and the difference processing is performed with the true projection data of the first reconstructed image to obtain the difference projection of each respiratory phase.

[0024] The image reconstruction of the difference projection by using the filtered back projection algorithm obtains the difference image of each respiratory phase, and the difference image and the artifact-free image are added to obtain the second reconstructed image.

[0025] According to the embodiment of the present application, the elastic registration and motion compensation between the second reconstructed images are performed to obtain the third reconstructed image, which includes:

[0026] The elastic registration of the second reconstructed image is performed by using the elastic registration algorithm to obtain the deformation information between the images corresponding to each respiratory phase.

[0027] The deformation processing of the second reconstructed image is performed by using the deformation information, and the motion compensation is performed by superimposing all the deformed second reconstructed images on the anatomical structure of the second reconstructed image of the target object at the preset respiratory phase to obtain the third reconstructed image.

[0028] According to an embodiment of the present application, the above-mentioned iterative image reconstruction based on bone weighting of the third reconstructed image to obtain the four-dimensional computed tomography hybrid imaging result of the target object comprises:

[0029] The bone structure corresponding to each respiratory phase in the four-dimensional computed tomography image is segmented from the third reconstructed image to obtain a bone image mask, and the bone image mask is forward projected into the projection corresponding to each phase to obtain a bone projection mask;

[0030] The third reconstructed image is iteratively reconstructed based on bone weighting according to the bone projection mask to obtain the four-dimensional computed tomography hybrid imaging result of the target object.

[0031] According to an embodiment of the present application, the above-mentioned iterative image reconstruction based on bone weighting of the third reconstructed image according to the bone projection mask to obtain the four-dimensional computed tomography hybrid imaging result of the target object comprises:

[0032] According to the bone projection mask, the third reconstructed image is taken as prior information and an initial value, and the weight of the projection pixel of the bone part in the image reconstruction process is reduced to eliminate the strip artifacts caused by the bone structure;

[0033] The bone structure reconstructed without weighting is combined with other parts reconstructed by bone weighting to obtain the four-dimensional computed tomography hybrid imaging result of the target object.

[0034] The second aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above-mentioned method.

[0035] The third aspect of the present application also provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the steps of the above-mentioned method.

[0036] The four-dimensional computed tomography hybrid imaging method provided by the present application uses the trained hybrid imaging neural network to provide prior information for 4DCT image reconstruction in a deep learning manner, and suppresses the strip artifacts caused by the bone structure through the iterative reconstruction method of bone weighting to complete high-quality image reconstruction. The four-dimensional computed tomography hybrid imaging method provided by the present application can improve the accuracy of image reconstruction in the projection angle sparse scenario, especially in the low-dose 4DCT scenario, effectively reduce the imaging time and radiation dose of 4DCT, and provide an effective reconstruction method for fast low-dose 4DCT, which has high clinical and preclinical application value. BRIEF DESCRIPTION OF DRAWINGS

[0037] The above content of the present application and other purposes, features and advantages will be more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:

[0038] Figure 1 is an application scenario diagram of a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present application;

[0039] Figure 2 is a flowchart of a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present application;

[0040] Figure 3 is a schematic diagram of a preclinical small animal CBCT imaging platform according to an embodiment of the present application;

[0041] Figure 4 is a process schematic diagram of a low-dose four-dimensional cone-beam computed tomography imaging method based on deep learning according to an embodiment of the present application;

[0042] Figure 5 is a hybrid imaging neural network structure schematic diagram according to an embodiment of the present application;

[0043] Figure 6 is a comparative result schematic diagram of a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present application;

[0044] Figure 7 is a structural block diagram of a four-dimensional computed tomography hybrid imaging device according to an embodiment of the present application;

[0045] Figure 8 is a block diagram of an electronic device suitable for implementing a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It should be understood, however, that the description is merely exemplary and is not intended to limit the scope of the present application. In the following detailed description of the embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it would be apparent to those skilled in the art that the embodiments of the present application can be practiced without these specific details. In other instances, well-known structures and techniques have been omitted in order to avoid obscuring the concepts of the present application.

[0047] The terms used herein are merely used to describe specific embodiments and are not intended to limit the present application. The terms "include", "comprise" and the like as used herein indicate the presence of the described features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0048] All terms used herein, including technical and scientific terms, have the meanings as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of the specification, and should not be interpreted in an idealized or overly formal way.

[0049] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should generally be interpreted that the meaning of the expression is the same as that of "at least one of A, or B, or C, etc.", unless otherwise defined. In other words, the expression "at least one of A, B, and C, etc." should be interpreted as "at least one of A, or B, or C, etc."

[0050] To solve the technical problem that the existing deep learning-based 4DCT imaging method has poor imaging quality on a small-scale data set (for example, a CT image set of a target object obtained under low-dose 4DCT conditions), the present application provides a four-dimensional computer tomography hybrid imaging method, an electronic device, and a storage medium, which complete high-quality 4DCT imaging under the condition of a small-scale data set by means of deep learning and bone weighting.

[0051] Figure 1 is an application scenario diagram of the four-dimensional computer tomography hybrid imaging method according to the embodiment of the present application.

[0052] As Figure 1 shown, the application scenario 100 according to the embodiment can include medical imaging and other scenarios requiring X-ray CT imaging. The network 104 is used to provide a communication link medium between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0053] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0054] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc.

[0055] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0056] It should be noted that the four-dimensional computed tomography (CT) hybrid imaging method provided in this embodiment of the invention can generally be executed by server 105. Correspondingly, the four-dimensional computed tomography (CT) hybrid imaging device provided in this embodiment of the invention can generally be located in server 105. The four-dimensional computed tomography (CT) hybrid imaging method provided in this embodiment of the invention can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the four-dimensional computed tomography (CT) hybrid imaging device provided in this embodiment of the invention can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0057] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0058] The following will be based on Figure 1 The described scene, through Figures 2-6 The four-dimensional computed tomography hybrid imaging method of the disclosed embodiments is described in detail.

[0059] Figure 2 This is a flowchart of a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present invention.

[0060] like Figure 2 As shown, the four-dimensional computed tomography hybrid imaging method of this embodiment includes operations S210 to S230.

[0061] In operation S210, taking four-dimensional cone-beam computed tomography (4DCBCT) as an example, the cone-beam filter back projection algorithm is used to reconstruct the image of the target object obtained by 4DCBCT to obtain the first reconstructed image.

[0062] The application of the above-mentioned cone beam filtered back-projection algorithm (for example, the Feldkamp-Davis-Kress algorithm, referred to as the FDK algorithm) in 4DCBCT image reconstruction is to reconstruct three-dimensional body data with a time sequence through weighted, filtered and back-projection operations on three-dimensional dynamic projection data.

[0063] The FDK algorithm is essentially a two-dimensional filtered back-projection algorithm (Filtered BackProjection, referred to as the FBP algorithm) extended to three-dimensional cone beam scanning, and its process includes: (1) weighting processing: applying a cone angle related weight factor to the projection data to compensate for geometric distortion; (2) filtering: performing convolution on the projection data using a ramp filter or the like; (3) back-projection reconstruction: reversely stacking the corrected projection data along the ray path into a three-dimensional space grid. Four-dimensional reconstruction extension: 4DCBCT acquires dynamic data through multiple cone beam scans in a time sequence. When reconstructing, the FDK algorithm needs to be independently applied to three-dimensional projection data at each time point, and finally four-dimensional space-time body data is synthesized.

[0064] The above-mentioned first reconstructed image is the starting point of 4DCBCT image reconstruction, and the first reconstructed image obtained in the FDK reconstruction process is an image of each respiratory phase with bar-shaped artifacts.

[0065] In operation S220, the first reconstructed image is subjected to image reconstruction based on an error minimization algorithm by using the trained hybrid imaging neural network, to obtain a second reconstructed image, and the second reconstructed images are mutually subjected to elastic registration and motion compensation, to obtain a third reconstructed image, wherein the trained hybrid imaging neural network is obtained by phase classification and image reconstruction based on the cone beam filtered back-projection algorithm on four-dimensional cone beam computed tomography projection samples, to obtain reference image samples and input image samples with a pairing relationship, and the hybrid imaging neural network is trained by using the reference image samples and the input image samples.

[0066] The image reconstruction based on the error minimization algorithm on the first reconstructed image by using the trained hybrid imaging neural network can provide prior knowledge for subsequent image reconstruction. Especially in the case of low-dose 4DCBCT, the hybrid imaging neural network based on deep learning provides prior knowledge, which can provide four-dimensional feature information of 4DCBCT projection for subsequent image reconstruction, and improves the imaging quality.

[0067] Elastic registration and motion compensation can reduce image blurring caused by motion and improve the quality of the reconstructed image.

[0068] In operation S230, the third reconstructed image is subjected to iterative image reconstruction based on bone weighting, to obtain the result of four-dimensional cone beam computed tomography hybrid imaging of the target object.

[0069] The four-dimensional computer tomography hybrid imaging method provided by the application uses the trained hybrid imaging neural network to provide prior information for 4DCBCT image reconstruction in a deep learning manner, and suppresses the strip artifacts caused by bone structures through bone weighted iterative reconstruction, thereby completing high-quality image reconstruction. The four-dimensional computer tomography hybrid imaging method provided by the application can improve the accuracy of image reconstruction under sparse projection angle scenarios, especially low-dose 4DCBCT scenarios, effectively reduce the imaging time and radiation dose of 4DCBCT, provide an effective reconstruction method for low-dose 4DCBCT, and has high clinical and preclinical application value.

[0070] The four-dimensional computer tomography hybrid imaging method provided by the application will be further described in detail below through specific embodiments.

[0071] To solve the technical problem that the performance of the deep learning CT image reconstruction method is limited on a small-scale data set, the application provides an innovative low-dose 4DCBCT hybrid imaging method based on deep learning. The method is centered on deep learning and combines several data fidelity algorithms, and can perform high-quality low-dose 4DCBCT image reconstruction on a small-scale data set. Taking preclinical animal imaging as an example, the method is introduced in three parts: (1) constructing a data set, building a neural network and training; (2) preliminary image reconstruction based on deep learning; (3) iterative reconstruction based on bone weighting.

[0072] According to the embodiment of the application, the phase classification of the four-dimensional cone beam computer tomography projection sample and the image reconstruction based on the cone beam filtered back projection algorithm are as follows: the four-dimensional cone beam computer tomography projection sample of the experimental organism is phase classified according to the physiological information of the experimental organism, and the phase classified image sample is obtained; the reference image sample is obtained by image reconstruction based on the cone beam filtered back projection algorithm on the phase classified projection image sample; the four-dimensional cone beam computer tomography projection sample corresponding to the reference image sample is down-sampled to obtain the down-sampled projection sample; and the input image sample with strip artifacts is obtained by image reconstruction based on the cone beam filtered back projection algorithm on the down-sampled image sample.

[0073] Before training the hybrid imaging neural network, first construct the training dataset: use the preclinical CBCT imaging platform to create a customized small animal lung 4D CBCT dataset. In order to obtain high-quality 4D CBCT images, the method of prolonging the imaging time is used to scan the small animal lung, and a total of 9000 projections are obtained for each scan, which takes 450 seconds. After the scan is completed, all the projections are classified according to the physiological information, and the FDK algorithm is used to reconstruct the images of each respiratory phase to obtain high-quality 4D CBCT images without bar artifacts, which are used as reference images. In order to obtain paired low-quality 4D CBCT images, the 9000 projections obtained by long-time scanning are down-sampled in number to 80 projections for each phase, and the FDK algorithm is used again to reconstruct the images to obtain low-quality 4D CBCT images with bar artifacts, which are used as input images paired with the reference images. The reference images and the input images are decomposed into smaller three-dimensional image blocks, which will be used to train the neural network with the ability to eliminate bar artifacts.

[0074] According to an embodiment of the present application, training the hybrid imaging neural network with the reference image samples and the input image samples comprises: constructing the hybrid imaging neural network with a three-dimensional convolutional layer and a plurality of dense convolutional modules with stacking functions; training the hybrid imaging neural network after normalizing and preprocessing the reference image samples and the input image samples with bar artifacts to obtain the trained hybrid imaging neural network with the ability to eliminate bar artifacts.

[0075] After obtaining the training dataset and dividing the training dataset into reference Figure 1 samples and input image samples, the neural network is built and trained: in the low-dose 4D CBCT hybrid imaging method based on deep learning, the neural network uses three-dimensional image data as input and output. The neural network is composed of a three-dimensional convolutional layer and a plurality of DenseBlock modules, and different DenseBlock modules are connected to each other through down-sampling, up-sampling and jump connection. The DenseBlock module is a dense convolutional module, which includes four three-dimensional convolutional layers and four stacking operations. In order to train the neural network, first normalize and preprocess the reference and input image blocks in the lung 4D CBCT dataset created in the above. Then use these three-dimensional image data to train the neural network, so that the neural network has the ability to eliminate bar artifacts in the three-dimensional image data.

[0076] According to the embodiment of the present application, the image reconstruction of the projection of the target object acquired by the four-dimensional cone-beam computed tomography using the cone-beam filtered back-projection algorithm includes: acquiring the four-dimensional cone-beam computed tomography projection of the target object by using the preclinical cone-beam computed tomography imaging device; performing phase classification on the four-dimensional cone-beam computed tomography projection of the target object according to the physiological information of the target object to obtain the phase classified image; and performing image reconstruction on the phase classified projection using the cone-beam filtered back-projection algorithm to obtain the first reconstructed image with the strip artifact.

[0077] During the image acquisition and FDK reconstruction process: low-dose 4D CBCT is acquired using a preclinical small animal CBCT imaging platform, and a total of 900 projections are acquired within 45 seconds. After the scan is completed, all projections are classified according to the amplitude of the respiratory phase. Then, the FDK algorithm is used for image reconstruction to obtain images of each respiratory phase with strip artifacts. These images will serve as the starting point for the image reconstruction process.

[0078] The preclinical micro-CT system combined with phase classification can achieve micron-level spatial resolution in live imaging while reducing a large amount of radiation dose.

[0079] After phase classification and FDK algorithm reconstruction, although the strip artifact is retained, it provides a structured error input for subsequent hybrid imaging neural network repair.

[0080] Respiratory phase classification by physiological information can effectively reduce the motion artifact existing in the image.

[0081] According to the embodiment of the present application, the image reconstruction of the first reconstructed image based on the minimum maximum error algorithm using the trained hybrid imaging neural network to obtain the second reconstructed image includes: eliminating the strip artifact of the first reconstructed image using the trained hybrid imaging neural network to obtain an artifact-free image; performing forward projection of the artifact-free image to the angle corresponding to each respiratory phase in the four-dimensional cone-beam computed tomography projection using the forward projection algorithm, and then performing difference processing on the artifact-free image and the true projection data of the first reconstructed image to obtain the difference projection of each respiratory phase; performing image reconstruction on the difference projection using the cone-beam filtered back-projection algorithm to obtain the difference image of each respiratory phase, and adding the difference image and the artifact-free image to obtain the second reconstructed image.

[0082] In the image reconstruction process based on the deep learning MKB algorithm: after obtaining the 4D CBCT image with bar-shaped artifacts reconstructed by the FDK algorithm, the hybrid imaging neural network trained in the image acquisition and FDK reconstruction process is used to preliminarily remove the bar-shaped artifacts from the 4D CBCT image reconstructed by the FDK algorithm; next, the artifact-free 4D CBCT image output by the hybrid imaging neural network is forward projected to the corresponding angle, and the difference value projection is obtained by subtracting the true projection data; finally, the difference value projection belonging to each respiratory phase is reconstructed by using the FDK algorithm to obtain the difference value image of each respiratory phase, and the difference value image is added to the artifact-free 4D CBCT image output by the neural network to obtain the corrected 4D CBCT image. The above operation is called the deep learning-based McKinnon-Bates algorithm, referred to as the deep learning MKB algorithm.

[0083] The hybrid neural network preferentially repairs the bar-shaped artifacts, combines the forward projection difference compensation, improves the structural similarity, and retains the bone microstructure details.

[0084] The forward projection algorithm is matched with the phase classification, and the respiratory phase registration error is effectively controlled.

[0085] The difference value projection local reconstruction strategy can greatly reduce the iterative calculation amount, reduce the hardware cost, and improve the imaging efficiency.

[0086] According to the embodiment of the present application, the elastic registration and motion compensation between the second reconstructed images are performed to obtain the third reconstructed image, including: using an elastic registration algorithm to perform elastic registration on the second reconstructed images to obtain deformation information between the images corresponding to each respiratory phase; performing deformation processing on the second reconstructed images by using the deformation information, and performing motion compensation by superimposing all the deformed second reconstructed images on the anatomical structure of the second reconstructed image of the target object at the preset respiratory phase to obtain the third reconstructed image.

[0087] In the elastic registration and motion compensation process: in order to make full use of the information in other respiratory phases, elastic registration is performed between the images of each phase obtained by using the deep learning MKB algorithm to obtain the deformation information between the images of each phase. According to the deformation information, other phases can be deformed and superimposed on the anatomical structure of a certain specific phase. The above operation is called MKB-motion compensation image reconstruction. The image obtained in this process will be used as the initial value of the subsequent bone-weighted iterative image reconstruction process.

[0088] The elastic registration algorithm can capture more detailed deformation information between respiratory phases; by deforming and superimposing on the reference phase, the displacement of the anatomical structure caused by respiratory motion is eliminated; and the elastic registration is complementary to the previous neural network reconstruction stage: the elastic registration solves the motion artifacts, and the difference value projection iteratively corrects the anatomical structure of the image.

[0089] According to the embodiment of the present application, the above-mentioned iterative image reconstruction based on bone weighting of the third reconstructed image to obtain the four-dimensional cone-beam computed tomography hybrid imaging result of the target object includes: segmenting the bone structure corresponding to each respiratory phase in the four-dimensional cone-beam computed tomography image from the third reconstructed image to obtain a bone image mask, and forward projecting the bone image mask into the projection corresponding to each phase to obtain a bone projection mask; and performing iterative image reconstruction based on bone weighting of the third reconstructed image according to the bone projection mask to obtain the four-dimensional computed tomography hybrid imaging result of the target object.

[0090] According to the embodiment of the present application, the above-mentioned iterative image reconstruction based on bone weighting of the third reconstructed image according to the bone projection mask to obtain the four-dimensional computed tomography hybrid imaging result of the target object includes: using the third reconstructed image as prior information and an initial value according to the bone projection mask, and reducing the weight of the projection pixels of the bone part in the image reconstruction process to eliminate the strip artifacts caused by the bone structure; and combining the bone structure reconstructed without weighting with other parts reconstructed by bone weighting to obtain the four-dimensional computed tomography hybrid imaging result of the target object.

[0091] The iterative reconstruction process based on bone weighting includes a bone structure segmentation operation and a bone-weighted image reconstruction operation.

[0092] The bone structure segmentation operation: first, according to the low-dose 4DCBCT acquisition and the result obtained by the MKB-motion compensation image reconstruction in the preliminary image reconstruction based on deep learning, the main bone structure in each phase image is segmented to obtain a bone image mask. The bone image mask is forward projected into the projection corresponding to each phase to obtain a bone projection mask.

[0093] The bone-weighted image reconstruction operation: using the result obtained by the MKB-motion compensation image reconstruction in the preliminary image reconstruction based on deep learning as prior information and an initial value in the low-dose 4DCBCT acquisition, the compressed sensing image reconstruction constrained by prior information is performed. In the process of image reconstruction, according to the bone projection mask, the weight of the projection pixels of the bone part in the image reconstruction process is reduced to avoid the strip artifacts caused by the bone structure. At the same time, the image reconstruction without weighting is also performed to reconstruct the correct bone structure. The final result of the image reconstruction is represented as: the combination of the bone structure obtained by the unweighted reconstruction and the other parts obtained by the bone-weighted reconstruction can effectively suppress the strip artifacts of the other parts caused by the bone structure without losing the reconstruction accuracy of the bone structure itself.

[0094] Through the above steps, the low-dose four-dimensional computed tomography hybrid imaging method based on deep learning can be applied to low-dose 4DCBCT imaging applications. Combined with deep learning and bone weighting iterative reconstruction, high-quality image reconstruction results can be obtained based on a neural network trained by a small-scale data set.

[0095] The specific embodiments are described below with reference to the accompanying drawings and in conjunction with specific embodiments. Figures 3-6 The above four-dimensional computed tomography hybrid imaging method provided by the present application is further described in detail.

[0096] The above four-dimensional computed tomography hybrid imaging method provided by the present application can complete high-quality imaging under the condition of low-dose 4DCBCT or small-scale image data set. The low-dose 4DCBCT refers to a radiation dose that can be reduced to not more than the level of conventional 3DCBCT.

[0097] Figure 3 is a schematic diagram of a preclinical small animal CBCT imaging platform according to an embodiment of the present application.

[0098] Figure 4 is a process schematic diagram of a low-dose four-dimensional computed tomography imaging method based on deep learning according to an embodiment of the present application.

[0099] Figure 5 is a schematic diagram of a hybrid imaging neural network structure according to an embodiment of the present application.

[0100] Figure 6 is a schematic diagram of a comparison result of a 4DCBCT hybrid imaging method according to an embodiment of the present application.

[0101] The specific embodiments are described below with reference to the accompanying drawings and in conjunction with specific embodiments. This time, low-dose 4DCBCT imaging of mouse lung is taken as an example. The 4DCBCT imaging collects X-ray projections at a frame rate of 20 frames per second, and 900 projections are collected at equal intervals on a 360° circumference, taking 45 seconds. The imaging time and dose are comparable to conventional 3DCBCT.

[0102] As shown in Figure 3 , the imaging platform used by the present application mainly consists of an X-ray tube, a stage, and a flat panel detector. The X-ray tube and the flat panel detector are fixed, and the projections distributed on the 360° circumference are obtained by rotating the stage. The mouse is fixed vertically after anesthesia and placed on the rotating stage to obtain the projection. According to the breathing amplitude information of the mouse in the projection, the projection is classified into the corresponding breathing phase.

[0103] As shown in Figure 4 , the above 4DCBCT hybrid imaging method provided by the present application includes the training process of the hybrid imaging neural network and the actual application process of the 4DCBCT hybrid imaging method.

[0104] In the training process of the hybrid imaging neural network, the dataset is collected and the neural network is trained: this specific embodiment performs long-time scanning on 13 mice to construct the dataset. The long-time scanning acquires 9000 projections within 450 seconds, and after phase classification and FDK reconstruction, high-quality lung 4D CBCT images are obtained. The projections of the long-time scanning are down-sampled to contain 80-90 projections per phase, and FDK image reconstruction is performed again to obtain low-quality lung 4D CBCT images with bar-shaped artifacts paired with the high-quality images. The paired high-low quality lung 4D CBCT images are used to train the neural network shown in the figure, so that the neural network has the ability to eliminate bar-shaped artifacts, wherein Figure 5 Figure 5 (1) in the figure is a structural schematic diagram of the neural network; Figure 5 (2) in the figure is a structural schematic diagram of the DenseBlock module used in the neural network, the input feature map channel number is m, and the output feature map channel number is n; Figure 5 (3) in the figure is a legend used.

[0105] In the actual application process of the 4D CBCT hybrid imaging method, the projections of the current low-dose 4D CBCT are acquired, and the preliminary image reconstruction based on deep learning is performed: the low-dose 4D CBCT acquires 900 projections within 45 seconds. After acquiring the projections, all the projections are classified into several respiratory phases. The preliminary reconstruction is performed on all the respiratory phases using the deep learning MKB algorithm to obtain the primary images eliminating part of the bar-shaped artifacts. Elastic registration is performed between the primary images of all phases, and MKB-motion compensation reconstruction is performed according to the registration results.

[0106] In the iterative reconstruction of bone weighting: according to the MKB-motion compensation image reconstruction results, the bone structure is segmented, and the segmented bone structure is mapped to the corresponding position of the projection by forward projection to obtain the position of the bone structure in the projection. Next, the image reconstruction results after MKB-motion compensation reconstruction are used as prior information and initial value to perform iterative reconstruction, and the weight of the bone part projection pixels is reduced in the reconstruction process. Finally, the results without residual bar-shaped artifacts are obtained.

[0107] Figure 6 (1) in the figure: reference image; Figure 6 (2) in the figure: the result directly predicted by the neural network, the arrow indicates the structure of the wrong prediction; Figure 6 (3) in the figure: the reconstruction result of the method without using bone weighting, the arrow indicates the bar-shaped artifact caused by the bone; Figure 6 ​(4) The reconstruction result using the bone-weighted method of this invention. The method provided by this invention can successfully suppress additional bar artifacts caused by bones while maintaining the accuracy of image reconstruction, and significantly improve the quality of image reconstruction.

[0108] In summary, the deep learning-based low-dose four-dimensional computed tomography imaging method provided by this invention can provide high-quality low-dose 4DCBCT images even with limited dataset size.

[0109] Based on the above-described four-dimensional computed tomography hybrid imaging method, this invention also provides a four-dimensional computed tomography hybrid imaging device. The following will be combined with... Figure 7 The device is described in detail.

[0110] Figure 7 This is a structural block diagram of a four-dimensional computed tomography hybrid imaging device according to an embodiment of the present invention.

[0111] like Figure 7 As shown, the four-dimensional computed tomography hybrid imaging device 700 of this embodiment includes an algorithm image reconstruction module 710, a neural network image reconstruction module 720, and a bone-weighted image reconstruction module 730.

[0112] The algorithm image reconstruction module 710 is used to reconstruct the image of the target object obtained by four-dimensional computed tomography using the cone-beam filter back projection algorithm to obtain a first reconstructed image. In one embodiment, the algorithm image reconstruction module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0113] The neural network image reconstruction module 720 uses a trained hybrid imaging neural network to perform image reconstruction on the first reconstructed image based on the minimization maximum error algorithm to obtain a second reconstructed image. It then performs elastic registration and motion compensation between the second reconstructed images to obtain a third reconstructed image. The trained hybrid imaging neural network is obtained by performing phase classification and image reconstruction based on a filtered back-projection algorithm on four-dimensional computed tomography projection samples to obtain paired reference image samples and input image samples. The hybrid imaging neural network is then trained using these reference image samples and input image samples. In one embodiment, the neural network image reconstruction module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0114] The skeleton-weighted image reconstruction module 730 is used to perform iterative image reconstruction based on skeleton weighting on the third reconstructed image to obtain a four-dimensional computed tomography hybrid imaging result of the target object. In one embodiment, the skeleton-weighted image reconstruction module 730 can be used to perform the operation S230 described above, which will not be repeated here.

[0115] According to embodiments of the present application, any of the algorithmic image reconstruction module 710, the neural network image reconstruction module 720 and the skeleton weighted image reconstruction module 730 can be combined in one module, or any of them can be split into multiple modules. Alternatively, at least part of the function of one or more of these modules can be combined with at least part of the function of the other modules and implemented in one module. According to embodiments of the present application, at least one of the algorithmic image reconstruction module 710, the neural network image reconstruction module 720 and the skeleton weighted image reconstruction module 730 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging a circuit, etc. or implemented in hardware or firmware, or implemented in any one of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the algorithmic image reconstruction module 710, the neural network image reconstruction module 720 and the skeleton weighted image reconstruction module 730 can be at least partially implemented as a computer program module which, when executed, can perform the corresponding function.

[0116] Figure 8 is a block diagram of an electronic device suitable for implementing the four-dimensional computed tomography hybrid imaging method according to embodiments of the present application.

[0117] As shown in Figure 8 The electronic device 800 according to embodiments of the present application includes a processor 801 which can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 802 or loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 can include, for example, a general purpose microprocessor (e.g. a CPU), an instruction set processor and / or a related chipset and / or a special purpose microprocessor (e.g. an application specific integrated circuit (ASIC)), etc. The processor 801 can also include an on-board memory for cache use. The processor 801 can include a single processing unit or multiple processing units for performing different actions of the method processes according to embodiments of the present application.

[0118] In the RAM 803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via the bus 804. The processor 801 performs various operations of the method flow according to the embodiments of the present application by executing the programs in the ROM 802 and / or the RAM 803. It should be noted that the programs can also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 can also perform various operations of the method flow according to the embodiments of the present application by executing the programs stored in the one or more memories.

[0119] According to the embodiments of the present application, the electronic device 800 can further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 can further include one or more of the following components connected to the input / output (I / O) interface 805: an input part 806 including a keyboard, a mouse, etc.; an output part 807 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 808 including a hard disk, etc.; and a communication part 809 including a network interface card such as a LAN card, a modem, etc. The communication part 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as necessary. A removable recording medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 810 as necessary, so that a computer program read out therefrom is installed in the storage part 808 as necessary.

[0120] The present application also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present application is implemented.

[0121] According to embodiments of the present application, the computer readable storage medium can be a non-transitory computer readable storage medium, such as, for example, without limitation, a portable computer diskette, a hard disk, random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present application, the computer readable storage medium can include the ROM 802 and / or the RAM 803 described above, and / or one or more other memory devices in addition to the ROM 802 and the RAM 803.

[0122] The flow diagrams and the block diagrams in the drawings are illustrations of possible architectures, functions, and operations for systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0123] Those skilled in the art will understand that features recited in various embodiments of the present application can be combined and / or integrated in various ways, even if such combinations or integrations are not expressly noted in the present application. In particular, features recited in various embodiments of the present application can be combined and / or integrated in ways that are not expressly noted in the present application, without departing from the spirit and teachings of the present application. All such combinations and / or integrations are within the scope of the present application.

[0124] The embodiments of the present application have been described above. However, these embodiments are merely meant to be illustrative, and are not meant to limit the scope of the present application. Although each of the embodiments has been described above separately, this does not mean that the measures in each of the embodiments cannot be used advantageously in combination. Various alternatives and modifications can be made to the embodiments of the present application without departing from the scope of the present application, and all such alternatives and modifications are intended to fall within the scope of the present application.

Claims

1. A four-dimensional computed tomography hybrid imaging method, characterized in that, The method includes: The first reconstructed image is obtained by reconstructing the projection of the target object obtained by four-dimensional computed tomography using a filtered back projection algorithm. The first reconstructed image is reconstructed using an error minimization algorithm using a trained hybrid imaging neural network to obtain a second reconstructed image. Elastic registration and motion compensation are then performed between the second reconstructed images to obtain a third reconstructed image. The trained hybrid imaging neural network is obtained by performing phase classification and image reconstruction based on a filtered back projection algorithm on four-dimensional computed tomography projection samples to obtain paired reference image samples and input image samples. The hybrid imaging neural network is then trained using the reference image samples and the input image samples. The third reconstructed image is subjected to iterative image reconstruction based on bone weighting to obtain a four-dimensional computed tomography hybrid imaging result of the target object. This includes: segmenting the bone structure corresponding to each respiratory phase in the four-dimensional computed tomography image from the third reconstructed image to obtain a bone image mask; forward projecting the bone image mask onto the projection corresponding to each phase to obtain a bone projection mask; and using the third reconstructed image as prior information and initial value based on the bone projection mask, eliminating the bar artifacts caused by the bone structure by reducing the weight of the projected pixels of the bone part in the image reconstruction process. The unweighted reconstructed skeletal structure is combined with other parts obtained from the weighted skeletal reconstruction to obtain the four-dimensional computed tomography hybrid imaging result of the target object.

2. The method according to claim 1, characterized in that, Phase classification and image reconstruction based on a filtered back-projection algorithm are performed on four-dimensional computed tomography (CT) projection samples to obtain paired reference image samples and input image samples, including: Based on the physiological information of the experimental organism, the respiratory phase classification was performed on the four-dimensional computed tomography projection sample of the experimental organism to obtain the phase-classified projection sample. The phase-classified projected image samples are reconstructed using a filtered back-projection algorithm to obtain the reference image samples. The four-dimensional computed tomography projection sample corresponding to the reference image sample is downsampled to obtain the downsampled projection sample. The downsampled image samples are reconstructed using a filtered back projection algorithm to obtain input image samples with bar artifacts.

3. The method according to claim 2, characterized in that, Training the hybrid imaging neural network using the reference image samples and the input image samples includes: The hybrid imaging neural network is constructed using three-dimensional convolutional layers and multiple dense convolutional modules with stacking capabilities; The hybrid imaging neural network is trained by normalizing and preprocessing the reference image samples and the input image samples with bar artifacts to obtain a trained hybrid imaging neural network with the ability to eliminate bar artifacts.

4. The method according to claim 1, characterized in that, The first reconstructed image is obtained by reconstructing the projection of the target object obtained through four-dimensional computed tomography using a filtered back projection algorithm, and includes: The four-dimensional computed tomography projection of the target object is acquired using a computed tomography imaging device; Phase classification is performed on the four-dimensional computed tomography projection of the target object based on its physiological information to obtain a phase-classified image. The filtered back projection algorithm is used to reconstruct the image of the phase-classified projection, resulting in a first reconstructed image with bar artifacts.

5. The method according to claim 4, characterized in that, Using the trained hybrid imaging neural network, the first reconstructed image is reconstructed based on an error minimization algorithm to obtain a second reconstructed image, including: The trained hybrid imaging neural network is used to eliminate the bar artifacts in the first reconstructed image to obtain an artifact-free image. After projecting the artifact-free image forward to the angle corresponding to each respiratory phase in the four-dimensional computed tomography projection using a forward projection algorithm, the difference is processed with the real projection data of the first reconstructed image to obtain the difference projection of each respiratory phase. The image is reconstructed by using the filtered back projection algorithm to obtain the difference image of each breathing phase, and the difference image is added to the artifact-free image to obtain the second reconstructed image.

6. The method according to claim 5, characterized in that, Elastic registration and motion compensation are performed between the second reconstructed images to obtain a third reconstructed image, including: The second reconstructed image is elastically registered using an elastic registration algorithm to obtain deformation information between the images corresponding to each breathing phase; The deformation information is used to deform the second reconstructed image, and motion compensation is performed by superimposing all the deformed second reconstructed images onto the anatomical structure of the second reconstructed image of the target object at a preset respiratory phase, to obtain the third reconstructed image.

7. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

Images