Four-dimensional computed tomography mixed imaging method

Abstract

The invention provides a four-dimensional computed tomography mixed imaging method which can be applied to medical images and other technical fields needing X-ray CT imaging. The method comprises the following steps: performing image reconstruction on a projection, obtained through four-dimensional computed tomography, of a target object by using a filtered back projection algorithm to obtain a first reconstructed image; performing image reconstruction based on an error minimization algorithm on the first reconstruction image by using the trained hybrid imaging neural network to obtain second reconstruction images, and mutually performing elastic registration and motion compensation between the second reconstruction images to obtain a third reconstruction image; and performing iterative image reconstruction based on skeleton weighting on the third reconstructed image to obtain a four-dimensional computed tomography mixed imaging result of the target object. The invention further provides a four-dimensional computed tomography mixed imaging device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a four-dimensional computed tomography hybrid imaging method, electronic equipment, and storage medium. Background Art

[0002] Computed tomography (CT), including cone-beam CT (CBCT), is an essential tool in both clinical and preclinical research. It is widely used in clinical diagnosis, image guidance for radiotherapy, and preclinical imaging of live animals. However, respiratory motion during imaging can cause motion blur and streaking artifacts in chest and abdominal CT images. Four-dimensional CT (4DCT) technology can effectively improve these issues: it classifies X-ray projections (hereinafter referred to as projections), which contain information about organ motion, into different phases and then reconstructs images of each phase separately. This effectively reduces artifacts caused by organ motion and provides high-quality, time-correlated dynamic image sequences. Traditional 4DCT imaging typically uses slow gantry rotation or repeated multiple scans to provide sufficient projection data for each respiratory phase. This approach produces high-quality 4DCT images, but at the expense of significantly longer scan times and increased radiation dose. If 4DCT images are acquired using the same scanning protocol as conventional static three-dimensional CT (3DCT) imaging, image quality will be compromised and severe streak artifacts will be produced when a standard filtered back-projection algorithm is employed for image reconstruction due to insufficient sampling of the projection data available at each phase.

[0003] In recent years, deep learning technology has become a breakthrough method in the field of low-dose CT image reconstruction. On clinical CT datasets, reconstruction methods based on deep learning have demonstrated image quality advantages that surpass traditional algorithms such as compressed sensing and motion compensation. However, reconstruction methods based on deep learning usually require a large dataset for training neural networks. In addition, in preclinical studies, the imaging parameters of different micro-CT instruments vary greatly, such as spatial resolution and the X-ray energy used, making it difficult to construct large-scale and universal preclinical animal datasets. It is often necessary to build customized datasets for specific instruments; furthermore, large-scale customized datasets are difficult to obtain, and their data size is far lower than the data volume in clinical scenarios. When there is less training data, the stability and generalization of deep learning methods on small-scale datasets will be greatly limited, and the reconstruction effect will be greatly reduced. Summary of the Invention

[0004] In view of the above problems, the present invention provides a four-dimensional computed tomography hybrid imaging method, an electronic device, and a storage medium, which are used to solve at least one of the problems in the prior art.

[0005] According to a first aspect of the present invention, a four-dimensional computed tomography hybrid imaging method is provided, comprising:

[0006] Reconstructing the projection of the target object obtained by four-dimensional computed tomography using a filtered back projection algorithm to obtain a first reconstructed image;

[0007] Using the trained hybrid imaging neural network to perform image reconstruction based on an error minimization algorithm on the first reconstructed image to obtain a second reconstructed image, and performing elastic registration and motion compensation on the second reconstructed images to obtain a third reconstructed image, wherein the trained hybrid imaging neural network is obtained by performing phase classification and image reconstruction based on a filtered back-projection algorithm on four-dimensional computed tomography projection samples to obtain reference image samples and input image samples having a paired relationship, and the reference image samples and the input image samples are used to train the hybrid imaging neural network;

[0008] The third reconstructed image is subjected to iterative image reconstruction based on bone weighting to obtain a four-dimensional computed tomography hybrid imaging result of the target object.

[0009] According to an embodiment of the present invention, the above-mentioned phase classification of the four-dimensional computed tomography projection samples and image reconstruction based on the filtered back projection algorithm to obtain the reference image samples and the input image samples having a paired relationship includes:

[0010] performing phase classification on a four-dimensional computed tomography projection sample of the experimental organism according to physiological information of the experimental organism to obtain a phase-classified image sample;

[0011] Performing image reconstruction based on the filtered back-projection algorithm on the projection image samples after phase classification to obtain reference image samples;

[0012] downsampling the four-dimensional computed tomography projection samples corresponding to the reference image samples to obtain downsampled projection samples;

[0013] The downsampled image samples are reconstructed based on a filtered back projection algorithm to obtain input image samples with stripe artifacts.

[0014] According to an embodiment of the present invention, the training of the hybrid imaging neural network using the reference image samples and the input image samples includes:

[0015] A hybrid imaging neural network is constructed using a 3D convolutional layer and multiple dense convolutional modules with stacking capabilities;

[0016] After the reference image samples and the input image samples with stripe artifacts are normalized and preprocessed, the hybrid imaging neural network is trained to obtain a trained hybrid imaging neural network capable of eliminating stripe artifacts.

[0017] According to an embodiment of the present invention, the above-mentioned image reconstruction of the projection of the target object obtained by four-dimensional computed tomography using the filtered back projection algorithm to obtain the first reconstructed image includes:

[0018] acquiring four-dimensional computed tomography projections of the target object using a computed tomography imaging device;

[0019] performing phase classification on a four-dimensional computed tomography projection of the target object according to physiological information of the target object to obtain a phase-classified image;

[0020] The projections after phase classification are reconstructed using a filtered back-projection algorithm to obtain a first reconstructed image with stripe artifacts.

[0021] According to an embodiment of the present invention, performing image reconstruction based on an error minimization algorithm on the first reconstructed image using the trained hybrid imaging neural network to obtain a second reconstructed image includes:

[0022] Using the trained hybrid imaging neural network to eliminate the stripe artifacts of the first reconstructed image, to obtain an artifact-free image;

[0023] The artifact-free image is forward-projected to the angle corresponding to each respiratory phase in the 4D computed tomography projection using a forward projection algorithm, and then subtracted from the real projection data of the first reconstructed image to obtain the difference projection of each respiratory phase;

[0024] The difference projections are reconstructed using a filtered back projection algorithm to obtain difference images of each respiratory phase, and the difference images are added to the artifact-free image to obtain a second reconstructed image.

[0025] According to an embodiment of the present invention, performing elastic registration and motion compensation on the second reconstructed images to obtain the third reconstructed image includes:

[0026] elastic registration algorithm is used to perform elastic registration on the second reconstructed image to obtain deformation information between images corresponding to each respiratory phase;

[0027] The second reconstructed image is deformed using the deformation information, and motion compensation is performed by superimposing all the deformed second reconstructed images on the anatomical structure of the second reconstructed image of the preset respiratory phase of the target object to obtain a third reconstructed image.

[0028] According to an embodiment of the present invention, the above-mentioned performing bone-weighted iterative image reconstruction on the third reconstructed image to obtain a four-dimensional computed tomography hybrid imaging result of the target object includes:

[0029] Segmenting the bone structure corresponding to each respiratory phase in the four-dimensional computed tomography image from the third reconstructed image to obtain a bone image mask, and forward-projecting the bone image mask to the projection corresponding to each phase to obtain a bone projection mask;

[0030] The third reconstructed image is subjected to bone-weighted iterative image reconstruction according to the bone projection mask to obtain a hybrid imaging result of a four-dimensional computed tomography scan of the target object.

[0031] According to an embodiment of the present invention, the above-mentioned iterative bone-weighted image reconstruction is performed on the third reconstructed image according to the bone projection mask to obtain a final result of four-dimensional computed tomography hybrid imaging of the target object, including:

[0032] According to the bone projection mask, the third reconstructed image is used as prior information and initial value, and the weight of the projection pixels of the bone part in the image reconstruction process is reduced to eliminate the stripe artifacts caused by the bone structure;

[0033] The unweighted reconstructed bone structure is combined with other parts obtained by bone weighted reconstruction to obtain the final result of four-dimensional computed tomography hybrid imaging of the target object.

[0034] A second aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0035] The third aspect of the present invention further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0036] The 4D computed tomography hybrid imaging method provided by the present invention utilizes a trained hybrid imaging neural network through deep learning to provide prior information for 4DCT image reconstruction. It also uses a bone-weighted iterative reconstruction method to suppress streak artifacts caused by bone structure, thereby achieving high-quality image reconstruction. This 4D computed tomography hybrid imaging method provided by the present invention can improve the accuracy of image reconstruction in scenarios with sparse projection angles, particularly low-dose 4DCT scenarios, effectively reducing 4DCT imaging time and radiation dose. This provides an effective reconstruction method for rapid, low-dose 4DCT and has high clinical and preclinical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0038] Figure 1 2. FIG. 1 is a diagram illustrating an application scenario of a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present invention;

[0039] Figure 2 is a flow chart of a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present invention;

[0040] Figure 3 is a schematic diagram of a preclinical small animal CBCT imaging platform according to an embodiment of the present invention;

[0041] Figure 4 3 is a process diagram of a low-dose four-dimensional cone-beam computed tomography imaging method based on deep learning according to an embodiment of the present invention;

[0042] Figure 5 is a schematic diagram of a hybrid imaging neural network structure according to an embodiment of the present invention;

[0043] Figure 6 is a schematic diagram of comparison results using a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present invention;

[0044] Figure 7 is a structural block diagram of a four-dimensional computed tomography hybrid imaging device according to an embodiment of the present invention;

[0045] Figure 8 is a block diagram of an electronic device suitable for implementing a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0047] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the presence of the features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0048] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0049] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0050] In order to solve the technical problem that existing deep learning-based 4DCT imaging methods have poor imaging quality on small-scale data sets (for example, a set of CT images of a target object obtained under low-dose 4DCT conditions), the present invention provides a four-dimensional computed tomography hybrid imaging method, an electronic device, and a storage medium. Through a deep learning and bone weighting-based approach, high-quality 4DCT imaging is achieved under small-scale data set conditions.

[0051] Figure 1 2 is an application scenario diagram of a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present invention.

[0052] like Figure 1 As shown, the application scenario 100 according to this embodiment may include medical imaging and other scenarios requiring X-ray CT imaging. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0053] A user may use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0054] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0055] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.

[0056] It should be noted that the four-dimensional computed tomography hybrid imaging method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the four-dimensional computed tomography hybrid imaging device provided in the embodiment of the present invention can generally be set in the server 105. The four-dimensional computed tomography hybrid imaging method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the four-dimensional computed tomography hybrid imaging device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0057] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0058] The following will be based on Figure 1 The scene described by Figures 2 to 6 The four-dimensional computed tomography hybrid imaging method of the disclosed embodiment is described in detail.

[0059] Figure 2 4 is a flow chart of a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present invention.

[0060] like Figure 2 As shown, the four-dimensional computed tomography hybrid imaging method of this embodiment includes operations S210 to S230.

[0061] In operation S210 , taking four-dimensional cone-beam computed tomography (4DCBCT) as an example, a cone-beam filtered back projection algorithm is used to reconstruct an image of a target object obtained by 4DCBCT to obtain a first reconstructed image.

[0062] The application of the above-mentioned cone-beam filtered back-projection algorithm (for example, the Feldkamp-Davis-Kress algorithm, referred to as the FDK algorithm) in 4DCBCT image reconstruction is to reconstruct three-dimensional volume data with a time series through weighting, filtering and back-projection operations on three-dimensional dynamic projection data.

[0063] The FDK algorithm is essentially a generalization of the 2D Filtered Back Projection (FBP) algorithm to 3D cone-beam scanning. Its process includes: (1) weighting: applying cone-angle-dependent weighting factors to the projection data to compensate for geometric distortion; (2) filtering: convolving the projection data using a ramp filter or other methods; and (3) back-projection reconstruction: superimposing the corrected projection data back along the ray path onto a 3D spatial grid. Four-dimensional reconstruction extension: 4DCBCT acquires dynamic data through multiple cone-beam scans in a time series. During reconstruction, the FDK algorithm is applied independently to the 3D projection data at each time point, ultimately synthesizing the 4D spatiotemporal volume data.

[0064] The first reconstructed image is the starting point of 4DCBCT image reconstruction. During the FDK reconstruction process, the first reconstructed image obtained is an image of each respiratory phase with stripe artifacts.

[0065] In operation S220, the trained hybrid imaging neural network is used to perform image reconstruction based on an error minimization algorithm on the first reconstructed image to obtain a second reconstructed image, and elastic registration and motion compensation are performed between the second reconstructed images to obtain a third reconstructed image, wherein the trained hybrid imaging neural network performs phase classification and image reconstruction based on a cone-beam filtered back projection algorithm on four-dimensional cone-beam computed tomography projection samples to obtain reference image samples and input image samples with a pairing relationship, and the reference image samples and the input image samples are used to train the hybrid imaging neural network.

[0066] Using the trained hybrid imaging neural network to perform error minimization-based image reconstruction on the first reconstructed image provides prior knowledge for subsequent image reconstruction. Especially in the case of low-dose 4DCBCT, the deep learning-based hybrid imaging neural network provides prior knowledge, providing multi-dimensional feature information of the 4DCBCT projection for subsequent image reconstruction, thereby improving imaging quality.

[0067] Elastic registration motion compensation can reduce image blur caused by motion and improve the quality of reconstructed images.

[0068] In operation S230, iterative image reconstruction based on bone weighting is performed on the third reconstructed image to obtain a final result of four-dimensional cone-beam computed tomography hybrid imaging of the target object.

[0069] The 4D computed tomography hybrid imaging method provided by the present invention utilizes a trained hybrid imaging neural network to provide prior information for 4D CBT image reconstruction through deep learning. It also uses a bone-weighted iterative reconstruction method to suppress streak artifacts caused by bone structure, thereby achieving high-quality image reconstruction. This 4D computed tomography hybrid imaging method provided by the present invention can improve the accuracy of image reconstruction in scenarios with sparse projection angles, particularly low-dose 4D CBT, effectively reducing 4D CBT imaging time and radiation dose. This provides an effective reconstruction method for low-dose 4D CBT and has high clinical and preclinical application value.

[0070] The 4DCBCT hybrid imaging method provided by the present invention is further described in detail below through specific examples.

[0071] To address the technical challenges of limited performance of deep learning CT image reconstruction methods on small datasets, this paper proposes an innovative deep learning-based low-dose 4DCBCT hybrid imaging method. This method, centered on deep learning and incorporating several data fidelity algorithms, enables high-quality low-dose 4DCBCT image reconstruction on small datasets. Using preclinical animal imaging as an example, this method is described in three parts: (1) dataset construction, neural network development, and training; (2) initial image reconstruction based on deep learning; and (3) iterative reconstruction based on bone weighting.

[0072] According to an embodiment of the present invention, the above-mentioned phase classification of four-dimensional cone-beam computed tomography projection samples and image reconstruction based on the cone-beam filter back projection algorithm to obtain reference image samples and input image samples with a paired relationship include: phase classification of the four-dimensional cone-beam computed tomography projection samples of the experimental organism according to the physiological information of the experimental organism to obtain phase-classified image samples; image reconstruction based on the cone-beam filter back projection algorithm on the phase-classified projection image samples to obtain reference image samples; downsampling the four-dimensional cone-beam computed tomography projection samples corresponding to the reference image samples to obtain downsampled projection samples; and image reconstruction based on the cone-beam filter back projection algorithm on the downsampled image samples to obtain input image samples with strip artifacts.

[0073] Before training the hybrid imaging neural network, a training dataset was first constructed. A customized small animal lung 4DCBCT dataset was created using a preclinical CBCT imaging platform. To obtain high-quality 4DCBCT images, the small animal lungs were scanned using an extended imaging time method, acquiring a total of 9,000 projections per scan, taking 450 seconds. After the scan, all projections were classified based on physiological information, and the FDK algorithm was used to reconstruct images for each respiratory phase. High-quality 4DCBCT images free of streaking artifacts were obtained and used as reference images. To obtain paired low-quality 4DCBCT images, the 9,000 projections obtained from the long scan were downsampled to 80 projections per phase. The FDK algorithm was again used to reconstruct low-quality 4DCBCT images with streaking artifacts. These images were used as input images for one-to-one pairing with the reference images. The reference and input images were decomposed into smaller 3D image patches, which were used to train the neural network with streaking artifact removal capabilities.

[0074] According to an embodiment of the present invention, training a hybrid imaging neural network using reference image samples and input image samples includes: constructing a hybrid imaging neural network using a three-dimensional convolution layer and multiple dense convolution modules with stacking functions; normalizing and preprocessing the reference image samples and the input image samples with stripe artifacts, and then training the hybrid imaging neural network to obtain a trained hybrid imaging neural network with the ability to eliminate stripe artifacts.

[0075] After obtaining the training data set and dividing the training data set into reference Figure 1 After the samples and input image samples, a neural network is built and trained: in the low-dose 4DCBCT hybrid imaging method based on deep learning, the neural network uses three-dimensional image data as input and output. The neural network consists of a three-dimensional convolutional layer and multiple DenseBlock modules, and different DenseBlock modules are connected to each other by downsampling, upsampling, and jump connections. The DenseBlock module is a dense convolution module, which includes four three-dimensional convolutional layers and four stacking operations. In order to train the neural network, the reference and input image blocks in the lung 4DCBCT dataset created above are first normalized and preprocessed. These three-dimensional image data are then used to train the neural network so that the neural network has the ability to eliminate strip artifacts in the three-dimensional image data.

[0076] According to an embodiment of the present invention, the above-mentioned use of the cone-beam filtered back projection algorithm to reconstruct the projection of the target object obtained by four-dimensional cone-beam computed tomography to obtain the first reconstructed image includes: using a preclinical cone-beam computed tomography imaging device to collect the four-dimensional cone-beam computed tomography projection of the target object; performing phase classification on the four-dimensional cone-beam computed tomography projection of the target object according to the physiological information of the target object to obtain a phase-classified image; and using the cone-beam filtered back projection algorithm to reconstruct the phase-classified projection to obtain a first reconstructed image with strip artifacts.

[0077] During image acquisition and FDK reconstruction, a low-dose 4DC CBCT scan was acquired using a preclinical small animal CBCT imaging platform, acquiring a total of 900 projections in 45 seconds. After the scan was completed, all projections were classified according to respiratory phase amplitude. Image reconstruction was then performed using the FDK algorithm, resulting in images of each respiratory phase that exhibited streaking artifacts. These images served as the starting point for the image reconstruction process.

[0078] Preclinical micro-CT systems combined with phase classification can achieve micrometer-level spatial resolution in in vivo imaging while significantly reducing radiation dose.

[0079] After phase classification, the FDK algorithm is used for reconstruction. Although the stripe artifacts are retained, it provides structured error input for the subsequent hybrid imaging neural network restoration.

[0080] Respiratory phase classification based on physiological information can effectively reduce motion artifacts in images.

[0081] According to an embodiment of the present invention, the above-mentioned use of the trained hybrid imaging neural network to perform image reconstruction on the first reconstructed image based on the minimization of the maximum error algorithm to obtain the second reconstructed image includes: using the trained hybrid imaging neural network to eliminate the strip artifacts of the first reconstructed image to obtain an artifact-free image; using the forward projection algorithm to forward project the artifact-free image to the angle corresponding to each respiratory phase in the four-dimensional cone-beam computed tomography projection, and then performing subtraction processing with the real projection data of the first reconstructed image to obtain the difference projection of each respiratory phase; using the cone-beam filtered back projection algorithm to reconstruct the difference projection to obtain the difference image of each respiratory phase, and adding the difference image to the artifact-free image to obtain the second reconstructed image.

[0082] During image reconstruction based on the deep learning MKB algorithm, after obtaining a 4DCBCT image with streaking artifacts reconstructed using the FDK algorithm, the hybrid imaging neural network trained during image acquisition and FDK reconstruction performs preliminary streaking artifact removal on the 4DCBCT image reconstructed using the FDK algorithm. Next, the artifact-free 4DCBCT image output by the hybrid imaging neural network is forward-projected to the corresponding angle and subtracted from the true projection data to obtain a difference projection. Finally, the difference projections belonging to each respiratory phase are reconstructed using the FDK algorithm to obtain difference images for each respiratory phase. These difference images are then added to the artifact-free 4DCBCT image output by the neural network to obtain the corrected 4DCBCT image. This operation is known as the McKinnon-Bates algorithm based on deep learning, or simply the deep learning MKB algorithm.

[0083] The hybrid neural network prioritizes the repair of stripe artifacts and combines forward projection difference compensation to improve structural similarity while preserving bone microstructure details.

[0084] The forward projection algorithm is matched with phase classification to effectively control the respiratory phase registration error.

[0085] The difference projection local reconstruction strategy can significantly reduce the amount of iterative calculations, reduce hardware costs, and improve imaging efficiency.

[0086] According to an embodiment of the present invention, the above-mentioned elastic registration and motion compensation are performed between the second reconstructed images to obtain the third reconstructed image, including: elastic registration of the second reconstructed images using an elastic registration algorithm to obtain deformation information between the images corresponding to each respiratory phase; deformation processing of the second reconstructed images using the deformation information, and motion compensation is performed by superimposing all the deformed second reconstructed images on the anatomical structure of the second reconstructed image of the preset respiratory phase of the target object to obtain the third reconstructed image.

[0087] During the elastic registration and motion compensation process, to fully utilize information from other respiratory phases, elastic registration is performed between the phase images obtained using the deep learning MKB algorithm, generating deformation information between the phase images. Based on this deformation information, the other phases can be deformed and superimposed onto the anatomical structures of a specific phase. This operation is called MKB-motion-compensated image reconstruction. The resulting image serves as the initial value for the subsequent iterative image reconstruction process based on bone weighting.

[0088] The elastic registration algorithm can capture more detailed deformation information between respiratory phases; by superimposing the deformation onto the reference phase, it eliminates the anatomical structure displacement caused by respiratory movement; and it complements the previous neural network reconstruction stage: elastic registration resolves motion artifacts, while difference projection iteratively corrects the image anatomical structure.

[0089] According to an embodiment of the present invention, the above-mentioned iterative image reconstruction based on bone weighting on the third reconstructed image to obtain the four-dimensional cone-beam computed tomography hybrid imaging result of the target object includes: segmenting the bone structure corresponding to each respiratory phase in the four-dimensional cone-beam computed tomography image from the third reconstructed image to obtain a bone image mask, and forward projecting the bone image mask to the projection corresponding to each phase to obtain a bone projection mask; performing iterative image reconstruction based on bone weighting on the third reconstructed image according to the bone projection mask to obtain the final result of the four-dimensional computed tomography hybrid imaging of the target object.

[0090] According to an embodiment of the present invention, the above-mentioned iterative bone-weighted image reconstruction of the third reconstructed image according to the bone projection mask to obtain the four-dimensional computed tomography hybrid imaging result of the target object includes: according to the bone projection mask, using the third reconstructed image as prior information and initial value, and eliminating the strip artifacts caused by the bone structure by reducing the weight of the projection pixels of the bone part in the image reconstruction process; combining the unweighted reconstructed bone structure with other parts obtained by the bone weighted reconstruction to obtain the final result of the four-dimensional computed tomography hybrid imaging of the target object.

[0091] The iterative reconstruction process based on bone weighting includes the operation of segmenting bone structure and the operation of bone weighting image reconstruction.

[0092] Segmentation of skeletal structures: First, based on the results of low-dose 4DCBCT acquisition and deep learning-based preliminary image reconstruction using MKB-motion compensation, the main skeletal structures in each phase image are segmented to obtain a skeletal image mask. The skeletal image mask is forward-projected onto the projection corresponding to each phase to obtain a skeletal projection mask.

[0093] Bone-weighted image reconstruction: Using the results of low-dose 4DCBCT acquisition and MKB-motion-compensated image reconstruction from the deep learning-based preliminary image reconstruction as prior information and initial values, compressed sensing image reconstruction constrained by prior information is performed. During the image reconstruction process, the weight of the projected pixels of the skeletal area is reduced according to the bone projection mask to avoid the appearance of stripe artifacts caused by the bone structure. Simultaneously, unweighted image reconstruction is performed to reconstruct the correct bone structure. The final image reconstruction result is represented by the combination of the skeletal structure obtained by unweighted reconstruction and the other parts obtained by bone-weighted reconstruction. This effectively suppresses stripe artifacts caused by the bone structure in other parts without compromising the reconstruction accuracy of the bone structure itself.

[0094] Through the above steps, a deep learning-based low-dose 4D computed tomography hybrid imaging method can be applied to low-dose 4D CBCT imaging applications. Combining deep learning with bone-weighted iterative reconstruction can produce high-quality image reconstruction results based on a neural network trained on a small-scale dataset.

[0095] The following is a specific implementation method and combined with the attached Figures 3 to 6 The above-mentioned four-dimensional computed tomography hybrid imaging method provided by the present invention is further described in detail.

[0096] The four-dimensional computed tomography hybrid imaging method provided by the present invention can achieve high-quality imaging under low-dose 4D CBCT or small-scale image data sets, where low-dose 4D CBCT means that the radiation dose can be reduced to a level not exceeding that of conventional 3D CBCT.

[0097] Figure 3 Schematic diagram of a preclinical small animal CBCT imaging platform according to an embodiment of the present invention.

[0098] Figure 4 3 is a process diagram of a low-dose four-dimensional computed tomography imaging method based on deep learning according to an embodiment of the present invention.

[0099] Figure 5 2 is a schematic diagram of a hybrid imaging neural network structure according to an embodiment of the present invention.

[0100] Figure 6 3 is a schematic diagram of comparison results using a 4DCBCT hybrid imaging method according to an embodiment of the present invention.

[0101] This specific embodiment uses a low-dose 4D C BCT imaging of a mouse lung as an example. This 4D C BCT imaging session captured X-ray projections at a frame rate of 20 frames per second, acquiring 900 projections at even intervals around a 360° circumference, taking 45 seconds. The imaging time and dose are comparable to conventional 3D C BCT.

[0102] like Figure 3 As shown in the figure, the imaging platform used in the present invention mainly consists of an X-ray tube, a stage, and a flat-panel detector. The X-ray tube and flat-panel detector remain fixed, and projections distributed over a 360° circumference are acquired by rotating the stage. After anesthesia, the mouse is vertically fixed and placed on the rotating stage to acquire the projections. Based on the mouse's respiratory amplitude information in the projections, the projections are classified into the corresponding respiratory phase.

[0103] like Figure 4 As shown, the 4DCBCT hybrid imaging method provided by the present invention includes a training process of a hybrid imaging neural network and an actual application process of the 4DCBCT hybrid imaging method.

[0104] In the training process of the hybrid imaging neural network, a data set is collected and the neural network is trained: In this specific embodiment, 13 mice are scanned for a long time to construct a data set. The long-term scan acquires 9,000 projections in 450 seconds, and after phase classification and FDK reconstruction, a high-quality lung 4DCBCT image is obtained. The projections of the long-term scan are downsampled to 80-90 projections per phase, and FDK image reconstruction is performed again to obtain a low-quality lung 4DCBCT image with stripe artifacts paired with the high-quality image. Paired high-low quality lung 4DCBCT images are used Figure 5 The neural network shown is trained to enable the neural network to have the ability to eliminate stripe artifacts, where Figure 5 (1) is a schematic diagram of the structure of the neural network; Figure 5 (2) is a schematic diagram of the structure of the DenseBlock module used in the neural network, where the number of input feature map channels is m and the number of output feature map channels is n; Figure 5 (3) in the figure is the legend used.

[0105] In the practical application of the 4DCBCT hybrid imaging method, projections are acquired during a low-dose 4DCBCT scan, and preliminary image reconstruction based on deep learning is performed. The low-dose 4DCBCT scan acquires 900 projections in 45 seconds. After acquisition, all projections are classified into several respiratory phases. A preliminary reconstruction is performed for all respiratory phases using the deep learning MKB algorithm, resulting in primary images that partially eliminate streak artifacts. Elastic registration is performed between the primary images of all phases, and MKB-motion-compensated reconstruction is performed based on the registration results.

[0106] During the iterative reconstruction of bone weighting, the skeleton is segmented based on the MKB-motion-compensated image reconstruction results. This segmented skeleton is then forward-projected onto the corresponding positions in the projection to determine the skeleton's position in the projection. Next, the MKB-motion-compensated image reconstruction results are used as prior information and initial values ​​for iterative reconstruction. During the reconstruction process, the weights of the projected pixels of the skeleton are reduced. This results in a result free of residual streak artifacts.

[0107] Figure 6 (1) in: reference image; Figure 6 (2) in the figure: the result of using the neural network directly for prediction. The arrow indicates the structure of the incorrect prediction. Figure 6 (3) in the figure: The reconstruction result of this method without using bone weighting. The arrow indicates the stripe artifact caused by the bone. Figure 6(4) in the figure: The reconstruction result of the present method using bone weighting. The method provided by the present invention can successfully suppress the additional stripe artifacts caused by bones while maintaining the accuracy of image reconstruction, significantly improving the quality of image reconstruction.

[0108] In summary, the deep learning-based low-dose four-dimensional computed tomography imaging method provided by the present invention can provide high-quality low-dose 4DCBCT images with a limited data set size.

[0109] Based on the above-mentioned four-dimensional computed tomography hybrid imaging method, the present invention also provides a four-dimensional computed tomography hybrid imaging device. Figure 7 The device is described in detail.

[0110] Figure 7 4 is a structural block diagram of a four-dimensional computed tomography hybrid imaging device according to an embodiment of the present invention.

[0111] like Figure 7 As shown, the four-dimensional computed tomography hybrid imaging device 700 of this embodiment includes an algorithm image reconstruction module 710, a neural network image reconstruction module 720 and a bone weighted image reconstruction module 730.

[0112] The algorithmic image reconstruction module 710 is used to reconstruct the projection of the target object obtained by four-dimensional computed tomography using a cone-beam filtered back projection algorithm to obtain a first reconstructed image; in one embodiment, the algorithmic image reconstruction module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0113] The neural network image reconstruction module 720 uses the trained hybrid imaging neural network to reconstruct the first reconstructed image based on the minimization of the maximum error algorithm to obtain a second reconstructed image, and performs elastic registration and motion compensation between the second reconstructed images to obtain a third reconstructed image, wherein the trained hybrid imaging neural network performs phase classification and image reconstruction based on the filtered back projection algorithm on the four-dimensional computed tomography projection samples to obtain reference image samples and input image samples with a pairing relationship, and the reference image samples and the input image samples are used to train the hybrid imaging neural network; in one embodiment, the neural network image reconstruction module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0114] The bone-weighted image reconstruction module 730 is configured to perform iterative image reconstruction based on bone weighting on the third reconstructed image to obtain a four-dimensional computed tomography hybrid imaging result of the target object. In one embodiment, the bone-weighted image reconstruction module 730 can be configured to perform operation S230 described above, which will not be described in detail herein.

[0115] According to an embodiment of the present invention, any multiple modules among the algorithmic image reconstruction module 710, the neural network image reconstruction module 720, and the bone weighted image reconstruction module 730 can be combined into a single module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in a single module. According to an embodiment of the present invention, at least one of the algorithmic image reconstruction module 710, the neural network image reconstruction module 720, and the bone weighted image reconstruction module 730 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware through any other reasonable means of circuit integration or packaging, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination of any of them. Alternatively, at least one of the algorithmic image reconstruction module 710, the neural network image reconstruction module 720, and the bone weighted image reconstruction module 730 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0116] Figure 8 is a block diagram of an electronic device suitable for implementing a four-dimensional computed tomography hybrid imaging method according to an embodiment of the present invention.

[0117] like Figure 8 As shown, an electronic device 800 according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 802 or programs loaded from a storage unit 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or related chipsets and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0118] Various programs and data required for the operation of the electronic device 800 are stored in the RAM 803. The processor 801, ROM 802, and RAM 803 are connected to each other via a bus 804. The processor 801 executes the programs in the ROM 802 and / or RAM 803 to perform various operations according to the method flow of the embodiment of the present invention. It should be noted that the programs may also be stored in one or more memories other than the ROM 802 and RAM 803. The processor 801 may also execute the programs stored in the one or more memories to perform various operations according to the method flow of the embodiment of the present invention.

[0119] According to an embodiment of the present invention, electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to bus 804. Electronic device 800 may also include one or more of the following components connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 808 including a hard disk; and a communication section 809 including a network interface card such as a LAN card or modem. Communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. Removable media 811, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 810 as needed, so that computer programs read from the removable media can be installed into storage section 808 as needed.

[0120] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0121] According to an embodiment of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present invention, a computer-readable storage medium may include the ROM 802 and / or RAM 803 described above, and / or one or more memories other than ROM 802 and RAM 803.

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0123] It will be understood by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or coupled in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or couplings fall within the scope of the present invention.

[0124] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A four-dimensional computed tomography hybrid imaging method, characterized in that: The method comprises: Reconstructing the projection of the target object obtained by four-dimensional computed tomography using a filtered back projection algorithm to obtain a first reconstructed image; Using the trained hybrid imaging neural network to perform image reconstruction based on an error minimization algorithm on the first reconstructed image to obtain a second reconstructed image, and performing elastic registration and motion compensation on the second reconstructed images to obtain a third reconstructed image, wherein the trained hybrid imaging neural network is obtained by performing phase classification and image reconstruction based on a filtered back projection algorithm on four-dimensional computed tomography projection samples to obtain reference image samples and input image samples having a paired relationship, and the hybrid imaging neural network is trained using the reference image samples and the input image samples; Performing bone-weighted iterative image reconstruction on the third reconstructed image to obtain a four-dimensional computed tomography hybrid imaging result of the target object.

2. The method according to claim 1, characterized in that Phase classification and image reconstruction based on the filtered back-projection algorithm are performed on the four-dimensional computed tomography projection samples to obtain reference image samples and input image samples with a paired relationship, including: performing respiratory phase classification on a four-dimensional computed tomography projection sample of the experimental organism according to physiological information of the experimental organism to obtain a phase-classified projection sample; Performing image reconstruction based on a filtered back-projection algorithm on the projection image samples after the phase classification to obtain the reference image samples; downsampling the four-dimensional computed tomography projection samples corresponding to the reference image samples to obtain downsampled projection samples; The downsampled image samples are reconstructed based on a filtered back projection algorithm to obtain input image samples with stripe artifacts.

3. The method according to claim 2, characterized in that Training the hybrid imaging neural network using the reference image samples and the input image samples includes: Constructing the hybrid imaging neural network using a three-dimensional convolutional layer and a plurality of dense convolutional modules with stacking functions; The reference image samples and the input image samples with stripe artifacts are normalized and preprocessed, and then the hybrid imaging neural network is trained to obtain a trained hybrid imaging neural network capable of eliminating stripe artifacts.

4. The method according to claim 1, wherein The projection of the target object obtained by four-dimensional computed tomography is reconstructed using a filtered back projection algorithm to obtain a first reconstructed image including: Acquiring a four-dimensional computed tomography projection of the target object using a computed tomography imaging device; performing phase classification on a four-dimensional computed tomography projection of the target object according to physiological information of the target object to obtain a phase-classified image; The filtered back projection algorithm is used to perform image reconstruction on the phase-classified projections to obtain a first reconstructed image with stripe artifacts.

5. The method according to claim 4, characterized in that Performing image reconstruction based on an error minimization algorithm on the first reconstructed image using the trained hybrid imaging neural network to obtain a second reconstructed image includes: Eliminating stripe artifacts of the first reconstructed image using the trained hybrid imaging neural network to obtain an artifact-free image; After forward-projecting the artifact-free image to an angle corresponding to each respiratory phase in the four-dimensional computed tomography projection using a forward projection algorithm, performing subtraction processing on the image with the real projection data of the first reconstructed image to obtain a difference projection of each respiratory phase; The filtered back projection algorithm is used to perform image reconstruction on the difference projection to obtain the difference image of each respiratory phase, and the difference image is added to the artifact-free image to obtain the second reconstructed image.

6. The method according to claim 5, characterized in that Performing elastic registration and motion compensation on the second reconstructed images to obtain a third reconstructed image includes: performing elastic registration on the second reconstructed image using an elastic registration algorithm to obtain deformation information between the images corresponding to each respiratory phase; The second reconstructed image is deformed using the deformation information, and motion compensation is performed by superimposing all deformed second reconstructed images on the anatomical structure of the second reconstructed image of the preset respiratory phase of the target object to obtain the third reconstructed image.

7. The method according to claim 4, characterized in that Performing bone-weighted iterative image reconstruction on the third reconstructed image to obtain a four-dimensional computed tomography hybrid imaging result of the target object includes: Segmenting the bone structure corresponding to each respiratory phase in the four-dimensional computed tomography image from the third reconstructed image to obtain a bone image mask, and forward-projecting the bone image mask to the projections corresponding to each phase to obtain a bone projection mask; The third reconstructed image is subjected to bone-weighted iterative image reconstruction according to the bone projection mask to obtain a final imaging result of the four-dimensional computed tomography of the target object.

8. The method according to claim 7, characterized in that Iteratively performing bone-weighted image reconstruction on the third reconstructed image according to the bone projection mask to obtain a four-dimensional computed tomography hybrid imaging result of the target object includes: According to the bone projection mask, the third reconstructed image is used as prior information and an initial value, and the weight of the projection pixels of the bone part in the image reconstruction process is reduced to eliminate the stripe artifacts caused by the bone structure; The unweighted reconstructed bone structure is combined with other parts obtained by the weighted bone reconstruction to obtain a final imaging result of the four-dimensional computer tomography of the target object.

9. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images