Medical image processing method, medical image processing apparatus, X-ray CT apparatus, and medical image processing program

Abstract

To generate a medical image in which object visibility and image quality in the medical image are improved.SOLUTION: A medical image processing method according to an embodiment includes: obtaining a first set of projection data by performing, with a first CT apparatus including a detector with a first pixel size, a first CT scan of an object by using a first imaging region of the detector; obtaining a first CT image with a first resolution by performing reconstruction processing on the first set of projection data; obtaining a processed CT image with a resolution higher than the first resolution by applying a machine-learning model for resolution enhancement to the first CT image; and displaying the processed CT image or outputting the processed CT image for analysis processing. The machine-learning model is obtained by machine learning using a second CT image based on a second set of projection data acquired by executing a second CT scan of the object by using a second imaging region smaller than the first imaging region of the detector with a second CT apparatus including a detector with a second pixel size smaller than the first pixel size.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure generally relates to the field of medical image processing and image diagnosis, and particularly relates to improving the spatial resolution of Computed Tomography (CT) images using a deep learning model.

Background Art

[0002] Detectors for computed tomography have been improved in terms of imaging range and spatial resolution, such as achieving a wide detection range with the size of small detection elements. One advantage of a wide-range CT detection system is an expanded imaging range. This enables faster scanning and dynamic imaging of organs including the heart and brain. The wide-range CT detection system extends the imaging range per rotation, shortening the scan time and eliminating the need for multiple data collections. By using a wide-range CT detection system, it may be possible to collect scans of the entire heart, a neonate's chest, and even the feet and ankles with high uniformity along the Z-axis and a low radiation dose in just a single rotation in a very short time.

[0003] On the other hand, a high-spatial-resolution CT system provides diagnostic images that can be improved, for example, in tumor classification and disease diagnosis.

[0004] However, even if a wide-range ultra-high resolution (UHR) CT detection system is commercially available, its system cost is high, and problems related to complex signal processing and image reconstruction may occur. Although the wide-range ultra-high resolution CT detection system has the advantages of a wider imaging range and higher resolution, in a commercial environment, the disadvantages of high cost and complexity may outweigh the advantages.

[0005] Super-resolution (SR) technology is a technique that improves the resolution of an imaging system. SR improves the resolution of an imaging system by restoring high-resolution information from low-resolution images. SR algorithms fall into four categories: predictive model-based models, edge-based models, image statistics-based models, and example-based models. In this field, there is a demand for deep convolutional neural network (DCNN)-based SR methods that can achieve superior image quality and faster processing speeds compared to conventional methods. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] U.S. Patent Application Publication No. 2013 / 051519 [Overview of the project] [Problems that the invention aims to solve]

[0007] One of the problems that the embodiments disclosed herein and in the drawings aim to solve is to improve the visibility of objects such as anatomical features in medical images and to generate medical images with improved image quality. However, the problems that the embodiments disclosed herein and in the drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0008] The medical image processing method according to the embodiment involves: obtaining a first projection data set by performing a first CT scan on a subject using a first imaging region of a first CT device having a detector of a first pixel size; obtaining a first CT image having a first resolution by reconstructing the first projection data set; obtaining a processed CT image with a resolution higher than the first resolution by applying a machine learning model to improve the resolution of the first CT image; and outputting the processed CT image for display or analysis. The machine learning model is obtained by machine learning using a second CT image based on a second projection data set obtained by performing a second CT scan on a subject using a second imaging region smaller than the first imaging region of a second CT device having a detector of a second pixel size smaller than the first pixel size. [Brief explanation of the drawing]

[0009] [Figure 1A] Figure 1A is a diagram showing an overview of the entire process of the embodiments illustrated in this disclosure. [Figure 1B] Figure 1B is a diagram illustrating an overview of a hardware system used in the training and inference phases of a machine learning model, based on one or more aspects of this disclosure. [Figure 2] Figure 2 shows a workflow for creating data to obtain and fine-tune a trained deep machine learning model (DCNN) based on one or more aspects of this disclosure. [Figure 3] Figure 3 is a flowchart for approximating a wide-area UHR-CT image based on one or more aspects of this disclosure. [Figure 4] Figure 4 is a block diagram showing a training framework for obtaining an optimized pre-trained DCNN model based on one or more aspects of this disclosure. [Figure 5A] Figure 5A shows an example of a DL network, which is a feedforward artificial neural network (ANN), based on one embodiment. [Figure 5B] Figure 5B shows an example of a DL network, which is a convolutional neural network (CNN), based on one embodiment. [Figure 5C] Figure 5C shows an example of realizing a convolutional layer in one neuron node of a convolutional layer, based on one embodiment. [Figure 5D] Figure 5D shows an example of realizing a 3-channel volume convolutional layer for volume image data based on one embodiment. [Figure 6] Figure 6 is a flowchart showing the procedure of a second embodiment for approximating a wide-area UHR-CT image based on one or more aspects of the present disclosure. [Figure 7] Figure 7 is a flowchart showing the procedure of a third embodiment for approximating a wide-area UHR-CT image based on one or more aspects of the present disclosure. [Figure 8] Figure 8 is a flowchart showing the procedure of a fourth embodiment for approximating a wide-area UHR-CT image based on one or more aspects of this disclosure. [Figure 9] Figure 9 shows a fifth embodiment of a workflow for creating data to acquire and fine-tune a trained deep machine learning model (DCNN) based on one or more aspects of this disclosure. [Figure 10] Figure 10 is a flowchart of a fifth embodiment for obtaining a DCNN-applicable image that approximates a wide-area UHR-CT image, based on one or more aspects of the present disclosure. [Figure 11] Figure 11 is a schematic diagram showing a computer embodiment available with one or more embodiments of at least one apparatus, system, method and / or storage medium for generating, optimizing and applying a model to generate a DCNN-applicable image that is very similar to or approximates a wide-area UHR-CT image. [Figure 12]FIG. 12 is a schematic diagram of a computer implementation that can be used with one or more embodiments of at least one apparatus, system, method, and / or storage medium for generating a DCNN applicable image that closely resembles or approximates a wide range of UHR-CT images by generating, optimizing, and applying a model, based on one or more aspects of the present disclosure. [Figure 13] FIG. 13 is a diagram showing a method for generating a pre-trained model for SR, based on another embodiment of the present disclosure. [Figure 14] FIG. 14 is a flowchart showing various procedures in the inference stage, based on another embodiment of the present disclosure. [Figure 15] FIG. 15 is a schematic diagram of a CT imaging apparatus, based on one embodiment of the present disclosure. [Figure 16] FIG. 16 is a diagram showing an example of a medical image processing system having a client-server configuration related to an intermediate network, based on one embodiment of the present disclosure. MODE FOR CARRYING OUT THE INVENTION

[0010] One of the purposes of this disclosure is to provide a method for generating a model for acquiring computed tomography (CT) images that approximate wide-area UHR-CT images. In one embodiment, the method for generating a model for acquiring CT images that approximate wide-area UHR-CT images makes it possible to realize wide-area ultra-high-resolution images without requiring a wide-area UHR-CT detection system. The method includes acquiring a first projection dataset collected by scanning an object to be imaged using a CT imaging modality. The first projection dataset may include ultra-high-resolution (UHR) CT data acquired from an imaging modality such as a UHR-CT scanner. The method may be continued by performing a resolution reduction process on the first projection dataset to acquire a second projection dataset. The second projection dataset may include normal-resolution (NR) CT data. The method is continued by training a machine learning model using a first CT image reconstructed based on the first projection dataset and a second CT image reconstructed based on the second projection dataset to acquire a model for generating CT images that approximate wide-area UHR-CT images. The machine learning model may be a deep convolutional neural network (DCNN) model. The first CT image may include a UHR-CT image, and the second CT image may include a normal resolution (NR) CT image.

[0011] In one or more embodiments of this disclosure, a medical image processing device is provided, comprising one or more memories for storing instructions and one or more processors for executing instructions and generating CT images to which machine learning models can be applied. The medical image processing device includes receiving a projection dataset collected by scanning an object under examination using a medical imaging modality. The projection dataset may include wide-area CT detection data obtained from a wide-area CT detector used as the imaging modality for scanning the object. The medical image processing device reconstructs a CT image of the object based on the projection dataset. The reconstructed image may include a wide-area CT detection image. The medical image processing device specifies either a first trained machine learning model for noise reduction or a second trained machine learning model for super-resolution. Both models can be stored in one or more memories. The specified models may be a deep convolutional neural network (DCNN) machine learning model for noise reduction or a DCNN machine learning model for super-resolution. The medical image processing device obtains a processed image by applying the specified model to the reconstructed CT image. The reconstructed CT image may include a wide-area CT detection image. After applying a trained DCNN model to wide-area CT detection images, processed images are generated. The processed images may include DCNN-applicable images that approximate or are similar to wide-area UHR-CT images.

[0012] One or more embodiments of this disclosure may be used for clinical applications such as medical imaging and research, but are not limited thereto.

[0013] Based on other aspects of this disclosure, one or more additional devices, one or more systems, one or more methods, and one or more storage media using deep convolutional neural networks for generating CT images that approximate wide-range ultra-high-resolution CT images are described. Further features of this disclosure will be understood and revealed from the following description with reference to the accompanying drawings.

[0014] To illustrate various aspects of this disclosure, similar elements are indicated by similar reference numerals in the drawings and are shown in an adoptable form that is simplified to the extent that it is understandable; however, this disclosure is not limited to, or to, the exact arrangement or means shown in the drawings. Refer to the accompanying drawings to assist those skilled in the art in creating and using the subject matter of this disclosure.

[0015] In this disclosure, the term "ultra-high resolution (UHR) CT detection system" is interchangeable with UHR-CT detection scanner or UHR-CT detector imaging. Similarly, in this disclosure, the term "wide-area CT detection system" is interchangeable with wide-area CT detection scanner or wide-area CT detector imaging. In the exemplary embodiments described below, the terms "ultra-high resolution (UHR)" and "normal resolution (NR)" do not imply specific resolutions. The spatial resolution of "UHR" is defined as relatively higher than that of "NR," and the spatial resolution of "NR" is lower than that of UHR. Furthermore, in the exemplary embodiments described below, the terms "wide-area" or "wider-area" do not imply specific ranges or specific sizes of detectors. "Wide-area" means a wider scanning range than that of a normal scanning range detector. Also, the terms "low-dose (LD)" and "high-dose (HD)" do not imply specific radiation doses. "Low radiation dose (LD)" refers to a radiation dose that is relatively lower than "high radiation dose (HD)," while "high radiation dose (HD)" refers to a radiation dose that is relatively higher than "low radiation dose (LD)."

[0016] This disclosure relates to constructing a super-resolution learning model using a UHR-CT detection system to obtain an optimized, trained deep convolutional neural network (DCNN). This trained DCNN is applied to wide-area CT detection images obtained from a wide-area CT detection system. In a clinical setting, super-resolution learning enables the wide-area CT detection system to obtain DCNN-applicable CT images that are similar to or approximate wide-area UHR-CT images by applying the optimized, trained DCNN. That is, the UHR-CT detection system is used to train the machine learning model but is not required in a clinical setting. The advantage is that only a wide-area CT system is needed in a clinical setting. In other words, this disclosure enables the acquisition of CT images that approximate wide-area UHR-CT images without using a wide-area UHR-CT detection system. This is particularly advantageous, for example, when a wide-area UHR-CT detection system is unavailable.

[0017] Next, refer to the details of the drawings. Figure 1A shows an overview of the process disclosed in an exemplary embodiment. Wide-area CT data 101 and UHR-CT data 201 are used to acquire wide-area UHR-CT images 301. UHR-CT data 201 is a projection dataset that has not undergone reconstruction processing. Wide-area CT data 101 may be collected from a wide-area CT detection system 100. UHR-CT data 201 may be collected from a UHR-CT detection system 200. UHR-CT data 201 is used in the training phase to acquire a trained model that is applied to low-resolution CT images, based on one or more aspects of this disclosure. In this disclosure, the UHR-CT data 201 used in the training phase is not required in a clinical setting. That is, the UHR-CT detection system 200 may be located remotely and used for training machine learning models. The wide-area CT detection system 100 is used for patient imaging in a clinical setting. Based on one or more aspects of this disclosure, in the inference phase, super-resolution processing is performed on the acquired wide-area CT data 101. In one aspect of this disclosure, the wide-area CT detection system 100 generates CT images that approximate or are similar to images that can be collected by the wide-area UHR-CT detection system, using a trained machine learning model obtained in the training phase, during the inference phase, without requiring a wide-area UHR-CT detection system. Super resolution (SR) of the image domain is described below with reference to Figures 2, 3, 6, 7, 8, 9, 10, and 13. SR of the data (projection) domain is described below with reference to Figure 14.

[0018] In one embodiment, the wide-area CT detection system 100 and the UHR-CT detection system 200 may each include a console or computer that communicates with a network, as described below with reference to Figures 11 and 12. The wide-area CT detection system 100 and the UHR-CT detection system 200 may also be connected via the network to a CPU associated with a computer (shown in Figures 11 and 12). That is, based on this disclosure, either or both of the detection systems (100, 200) may be connected via the network to a console or computer, or they may have a computer built-in. An example of the configuration of a CT system is described below with reference to Figure 15.

[0019] The hardware system used in the training and inference phases of the machine learning model will be described with reference to Figure 1B. The training of the machine learning model is performed in an information processing device 400 having the same components as the computer 1200' shown in Figure 12. The information processing device 400 receives UHR-CT data 201 from the UHR-CT detection system 200 via the network I / F 1212. The CPU or GPU trains the machine learning model based on the UHR-CT data 201. Details of the training process will be described later. After training is complete, the information processing device 400 acquires the trained model 401 for super-resolution (SR). The trained model 401 for SR is output to the image processing device 150 in the wide-area CT detection system 100 or to the console described above. The image processing device 150 or its CPU stores the trained model 401 in memory. The CPU or GPU of the image processing device 150 generates a wide-area UHR-CT image 301 based on wide-area CT data 101 collected by the wide-area CT detection system 100 and a trained model 401 applied to CT images or CT (projection) data.

[0020] In another embodiment, instead of wide-range CT images, the trained model described above is applied to CT images collected by a standard wide-range CT detection system to generate CT images with improved spatial resolution.

[0021] Figure 2 shows the workflow for data creation and deep convolutional neural network (DCNN) processing for training a machine learning model. This workflow begins with acquiring UHR-CT data in step S100. UHR-CT data is pre-reconstruction projection data collected from an ultra-high resolution CT detection scanner 200. The UHR-CT detection scanner 200 may be located in a different location from or far from the site and used for training to acquire the machine learning model. The next step in the workflow, S102, includes performing a UHR-NR simulation (or resolution reduction process) on the UHR-CT data to acquire normal resolution (NR) CT data. An example of a UHR-to-NR simulation is available downsampling, including, for example, binning of data domains in a 4:1 ratio. However, the use of other types of resolution reduction processes, such as smoothing or other filtering, is also within the scope of this disclosure. The UHR-to-NR simulation simulates pre-reconstruction CT data collected from a normal resolution scanner (e.g., 4:1 pixel binning). Alternatively, if an NR-CT detection system is available, NR-CT data may be collected directly from the NR-CT detection system rather than by downsampling the UHR-CT data. When an NR-CT detection system is used to acquire NR-CT data, steps S100 and S102 may be performed in parallel or individually.

[0022] In step S102, when downsampling the UHR-CT data from step S100 to NR-CT data, the reason for downsampling is that the UHR-CT data has a larger number of pixels compared to conventional CT data. In other words, it is ultra-high resolution data versus normal resolution data. In one embodiment, UHR-CT data may have four times the number of pixels (1024×1024) of conventional CT data (512×512). That is, since the pixels from UHR-CT are four times smaller than those from conventional CT, the pixels from UHR-CT are downsampled to match the pixel size of conventional CT.

[0023] The next steps, S104 and S106, involve the reconstruction of the data collected in steps S100 and S102. Specifically, in step S104, UHR-CT data is reconstructed into UHR-CT images. In step S106, NR-CT data is reconstructed into NR-CT images. UHR-CT images are reconstructed images from the UHR-CT detection scanner 200 and are used as training targets for the DCNN. In other words, UHR-CT images are the training targets for the machine learning model. NR-CT images are reconstructed images from UHR-CT data whose image quality has been reduced to a smaller pixel size (binned) in order to match the training targets.

[0024] The DCNN in the image domain can be any type of DCNN structure, such as U-NET, V-NET, and EDSR, but the types of DCNN structures applicable to this disclosure are not limited to these. In step S108, the NR-CT image is used as input to the DCNN or a machine-based learning model. The UHR-CT image is used as the target of the DCNN learning workflow to optimize the DCNN. During the optimization process, the DCNN outputs a processed NR-CT image in step S108. In step S110, the processed NR-CT image is used to obtain the loss function. In step S112, the DCNN model is optimized by the loss function. In step S114, the information processing device 400 determines whether the termination criterion is met. The processing loop continues until the termination criterion is met ("Y" in step S114). If the termination criterion is not met ("N" in step S114), the processing loop returns to step S108. The processed NR-CT image is compared with the UHR-CT image (target). The processed NR-CT image is the training output image of a DCNN machine learning model that takes NR-CT images as input. The loss function between the UHR-CT image and the processed NR-CT image aims to reduce the difference between the two images. The loss function between the UHR-CT image and the processed NR-CT image may also aim to improve the image quality of the processed NR-CT image through each iteration looped back to the DCNN machine learning model. Image quality improvement of the processed NR-CT image is optimized until there is no further room for improvement in the image, or until no further improvement in image quality is observed. Common loss function settings for applicable neural network training include, for example, the mean square factor (MSA) and the mean squared error (MAE), but the types of loss functions applicable to neural network training in this disclosure are not limited to these. Details of the loss function and optimization process for optimizing the trained DCNN model are described below with reference to Figure 4.

[0025] The DCNN training process is a method for creating a machine learning model to generate DCNN-applicable computed tomography (CT) images that are very similar to or approximate wide-range UHR-CT images. The method includes scanning an object to be imaged using a CT imaging modality to obtain a first projection dataset (UHR-CT data). The method also includes obtaining a second projection dataset (NR-CT data) by performing a resolution reduction process on the first projection dataset. Next, a super-resolution model (trained DCNN) is obtained by training a machine learning model (DCNN) using a first CT image reconstructed based on the first projection dataset and a second CT image reconstructed based on the second projection dataset. In one embodiment, when applying a resolution reduction process, the second projection dataset may be obtained by adding noise data to the first projection dataset. The addition of noise data is done to make the noise level of the second CT image higher than the noise level of the first CT image.

[0026] The addition of noise data is performed when the input CT image is processed by a trained machine learning model, so that the trained machine learning model denoises the input CT image and improves its resolution. In another embodiment, the input to the machine learning model is a three-dimensional (3D) image data of a predetermined size, and the output of the machine learning model is a 3D image data of a predetermined size.

[0027] Next, refer to the flowchart in Figure 3. This flowchart illustrates various steps of a framework known as inference, utilizing the trained DCNN in Figure 2, applied to the wide-area CT detection system 100. Using a wide-area CT detector and a trained DCNN together offers several advantages. One reason is that a whole-heart scan or other biological scan can be achieved in a single scan, leading to reduced radiation dose, shorter scan times, reduced hardware and software complexity, and lower costs. Using a trained DCNN machine learning model allows the resolution to approximate that of the UHR-CT detection system 200 without posing some of the problems associated with wide-area UHR-CT detection systems. Two or more scans, which take more time, are more susceptible to patient movement, potentially resulting in unsatisfactory scans and increased software and hardware complexity.

[0028] The inference framework is initiated in step S200 by collecting wide-area CT detection data. Wide-area CT detection data is pre-reconstructed CT projection data collected from the wide-area CT detection system 100. The inference framework is applied in a clinical environment where patients are scanned using imaging modalities for diagnostic imaging. The image processing device 150 of the wide-area CT detection system 100 reads the reconstruction conditions for the collected wide-area CT data. The reconstruction conditions are determined based on the scanned body part or the purpose of imaging. In this step, the trained DCNN is selected from multiple trained DCNNs specifically trained for the scanned body part or the purpose of imaging. The workflow continues in step S202 with reconstruction to generate wide-area CT detection images based on the wide-area CT data. In step S204, a processed CT image is generated by applying the trained DCNN to the wide-area CT detection image. By applying the trained DCNN to the wide-area CT detection image, a DCNN-applicable CT image that approximates a wide-area UHR-CT image is obtained. In step S206, the processed CT image is output and displayed on a display monitor for image quality check and / or diagnosis. The pre-trained DCNN to be applied is generated by the UHR-CT detection system 200 during the training phase of this disclosure.

[0029] In Figure 3, only the super-resolution (SR) DCNN model trained based on the method in Figure 2 was applied to the CT image. However, in another embodiment, a trained denoising DCNN model (the method for generating this model is described below) may be applied in addition to the SR-DCNN model. It is also possible to apply the SR-DCNN and denoising DCNN sequentially, such as applying the denoising DCNN after the SR-DCNN, or applying the SR-DCNN after the denoising DCNN. In yet another embodiment, multiple DCNNs may be applied in parallel to the same CT image, and the CT image with the SR-DCNN applied and the CT image with the denoising DCNN applied may be combined in a predetermined ratio. Furthermore, the DCNN may be trained to have both denoising and SR effects. This will be explained below with reference to Figure 9 or Figure 13.

[0030] DCNN-applicable CT images that approximate wide-area UHR-CT images are high-resolution images that cover a wide segment. In other words, by applying a trained DCNN, it becomes possible to generate higher-resolution images (UHR-CT images) from wide-area CT detection data. As a result, the advantages of using the wide-area CT detection system 100 (wide scan range, low cost, and less complex signal processing) are obtained simultaneously with the advantage of minimizing any of the problems associated with high resolution from UHR-CT data and the wide-area UHR-CT scanner system 300 (narrow scan range, high cost, complex processing, increased radiation dose, and susceptibility to artifacts).

[0031] In another embodiment, a medical image processing device comprising one or more memories and one or more processors applies a trained DCNN to generate a processed image (a DCNN-applicable CT image approximating a wide-area UHR-CT image) by performing various procedures. The medical image processing device may also be a wide-area CT detection scanner / system. Alternatively, the medical image processing device may be configured to receive data from, for example, a UHR-CT detection system 200 and be an applicable device for a trained machine learning model. The medical image processing device receives a projection dataset collected by scanning an object under examination using a medical imaging modality and reconstructs a CT image of the object based on the projection dataset. A first trained machine learning model for noise reduction and a second trained machine learning model for super-resolution are stored in one or more memories, one of which is designated. The designated model is applied to the reconstructed CT image to obtain a processed image.

[0032] The medical image processing device may be configured to reconstruct a CT image using a first reconstruction filter if a first pre-trained machine learning model is specified, and to reconstruct a CT image using a second reconstruction filter if a second pre-trained machine learning model is specified. The medical image processing device can combine a processed image and a reconstructed CT image in a predetermined ratio. The predetermined ratio may be set based on user input or determined according to a set of imaging conditions. In another embodiment, the medical image processing device is configured to generate a plurality of 3D partial images based on a reconstructed CT image, input the generated plurality of 3D partial images into a specified model to apply the specified model to obtain a plurality of processed images, and synthesize the obtained plurality of processed 3D partial images to obtain a processed image. In some situations, at least two of the 3D partial images partially overlap.

[0033] In another aspect of the present disclosure, the medical image processing device applies a filter to the junction between two adjacent processed 3D partial images from a plurality of processed 3D partial images.

[0034] In at least one embodiment of this disclosure, one feature of wide-area ultra-high-resolution CT to which deep learning neural networks can be applied is the use of a trained DCNN acquired from the UHR-CT detection system 200. As described above, one application of the trained DCNN of this disclosure is to use optimization processing for training a machine learning model, as schematically shown in Figure 4.

[0035] Figure 4 shows in more detail the optimization process applied to the DCNN learning framework described in Figure 2. As shown in Figure 4, the framework is started in step S300 with input (X). In step S310, the DCNN learning process is designed to map the learning input (X) to the desired target (Y). After obtaining the input (X), in steps S302 and S304, the following DCNN(f(X / Θ)) algorithm is used to output

number

number

[0036] Θ represents the set of parameters of the neural network to be optimized, and N is the total number of training instances in the learning process. f represents the neural network to be optimized, and x i y represents the i-th element of the training input. i represents the i-th element to be learned. By solving this optimization equation, we can find the optimal network parameters that minimize the difference between the network output and the target image (Y).

number

number

number

[0037] Figures 5A, 5B, 5C, and 5D show various examples of machine learning model 401 (also known as DL network 401).

[0038] Figure 5A shows an example of a typical artificial neural network (ANN) with N inputs, K hidden layers, and 3 outputs. Each layer consists of nodes (also called neurons), each node performing a weighted sum of the inputs and comparing the result of this weighted sum to a threshold to generate an output. The ANN constitutes a class of functions. Terms of this class can be obtained by changing structural details such as the threshold, connection weights, or the number of nodes and / or node connectivity. The nodes of an ANN are also called neurons (or neuron nodes). Neurons can be interconnected between different layers of the ANN system. The DL network 401 typically has four or more neuron layers and the same number of output neurons as input neurons.

number

[0039] Mathematically, the network function m(x) of neurons is a function of another function n i Defined as a composition of (x). Function n i (x) can be defined as a composition of other functions. This can be represented as a network structure for convenience. Figure 5A shows the dependencies between variables with arrows. For example, ANN is a nonlinear weighted sum m(x) = K(Σ i w i n i (x)) is available. K (commonly called the activation function) is a predefined function such as the sigmoid function, hyperbolic tangent function, or rectified linear unit (ReLU).

[0040] In Figure 5A (and similarly in Figure 5B), neurons (i.e., nodes) are represented by circles around a threshold function. In the non-restrictive example shown in Figure 5A, the input is represented by circles around a linear function, and arrows indicate the direction of connections between neurons. In one embodiment, the machine learning model 401 is a feedforward network illustrated in Figures 5A and 5B (which can be represented, for example, as a directed acyclic graph).

[0041] The machine learning model 401 operates to achieve specific tasks such as super-resolution processing of CT images by searching within the class of the function F to be learned using a set of observation results, and optimally solves the specific task.

number

number

[0042] Figure 5B shows a non-restrictive example where machine learning model 401 is a convolutional neural network (CNN). CNNs are a type of ANN with properties useful for image processing. Therefore, they are particularly relevant for image denoising and sinogram reconstruction. CNNs use a feedforward ANN that can represent image processing convolutions through the connection patterns between neurons. For example, a CNN can be used to optimize image processing by using multiple layers of small sets of neurons that process a portion of the input image called the receptive field. The outputs of these sets are tiled so that they overlap each other, allowing for a better representation of the original image. This processing pattern is repeated for multiple layers, alternating between convolutional and pooling layers. Figure 5B also shows an example of a fully connected network where the nodes of a subsequent layer are defined using all the nodes of the preceding layer. The content shown in the figure should be understood strictly as an example of a DNN. Regarding CNNs, a loosely connected (partially connected) network configuration, where the nodes of a subsequent layer are defined using some of the nodes of the preceding layer, is common.

[0043] Figure 5C shows an example of a 5x5 kernel applied to mapping values ​​from the input layer to a two-dimensional image of the first hidden layer, which is a convolutional layer. The kernel maps each 5x5 pixel region to the corresponding neuron in the first hidden layer.

[0044] Following the convolutional layers, the CNN may include local and / or global pooling layers that combine the outputs of neuron clusters in the convolutional layers. Furthermore, in some embodiments, the CNN may also include various combinations of convolutional and fully connected layers, added point by point, nonlinearly, at the end of each layer, or after each layer.

[0045] Regarding image processing, CNNs offer several advantages. Convolutional processing is introduced on small regions of the input to reduce the number of unused parameters and improve generalization. A key advantage of one implementation of CNNs is the use of common weights in the convolutional layers; that is, the same filter (weight bank) is used as the coefficient for each pixel in the layer. This reduces the memory implementation area and improves performance. Compared to other image processing methods, CNNs have the advantage of requiring relatively little preprocessing. This means the network is responsible for learning filters manually designed by conventional algorithms. Not relying on past knowledge or human effort regarding design characteristics is a major advantage of CNNs.

[0046] Figure 5D shows an example of a machine learning model 401 that leverages the similarity between adjacent layers of a three-dimensional reconstructed image. Signals within adjacent layers are typically highly correlated, but their noise differs. In other words, three-dimensional volumetric images of CT can generally capture more volumetric features and therefore provide more diagnostic information than horizontal two-dimensional images of a single cross-section. Based on this idea, an example of the method described herein enhances CT images using a volume-based deep learning algorithm.

[0047] As shown in Figure 5D, a given tomographic image and its adjacent tomographic images (i.e., the tomographic images above and below the central tomographic image) are recognized as inputs to the 3 channels of the network. A W×W×3 kernel is applied M times to these 3 layers to generate M values ​​for the convolutional layer, which are then used for the next network layer / hierarchy (e.g., a pooling layer). It is also possible to consider the W×W×3 kernel as three W×W kernels and apply each as a 3-channel kernel to the three tomographic images of the volume image data. The result becomes the output for the central layer and is used as input to the next network hierarchy. The value M represents the total number of filters for a given tomographic image in the convolutional layer, and W represents the size of the kernel.

[0048] In another embodiment, a different method (e.g., a 3D method) may be applied instead of the three-channel method described above.

[0049] In one embodiment of this disclosure, in order to reduce the computational cost during the training and inference phases, CT images can be divided into smaller image datasets and input into a machine learning model for training and inference.

[0050] By splitting, weighting, and reconstructing data, wide-area CT detection systems can buffer computational power and data to process advanced networks, such as super-resolution 3D networks for wide-area UHR images. This disclosure proposes a data flow for splitting, weighting, and reconstructing data that is useful for its implementation. For example, by decomposing a 1024×1024 image into 81 128×128 sub-images in the XY dimensions (including overlaps to prevent boundary effects), the system can process small batches of images at a time. After processing by the network, the image is reconstructed to its original size (e.g., 1024×1024). Overlapping pixels are weighted. A similar method is applied to the Z dimension. That is, an ultra-high resolution image is split into multiple sub-images, which are processed by the wide-area CT detection system and reconstructed into a larger image based on an arbitrary preferred weighting and reconstruction data flow. The data splitting, weighting, and reconstruction method is applicable to images of various sizes. The image described above is merely an example and does not limit the various sizes to which the method can be applied.

[0051] Next, we refer to the flowchart in Figure 6 illustrating a second embodiment of the present disclosure. This embodiment proposes a mixed process that allows the user to adjust the output texture (weak, standard, strong, etc.) and output a UHR-CT system image or NR-CT system image according to their preference. In this embodiment, the trained DCNN is acquired in the same manner as in Figure 2 described above. The flowchart starts in step S400 by collecting wide-area CT detection data. Next, in step S402, the wide-area CT detection image is reconstructed (to a smaller pixel size). In step S404, a DCNN-applicable CT image that approximates the wide-area UHR-CT image is output by applying the DCNN-trained machine learning model to the wide-area CT detection image. However, the user may not be satisfied with the texture of a 100% DCNN-applicable wide-area UHR-CT image. In this embodiment, the user of the wide-area CT detection system 100 can select the output texture of the DCNN-applicable CT image that approximates the wide-area UHR-CT image in the mixed step S408. The mixing step S408 may include, for example, three options selectable by the operator, such as weak, standard, and strong. Alternatively, the user may select a mixing ratio in the mixing step. For example, if the operator selects 50%, the mixing adjusts the output texture to consist of 50% of the original NR-CT detection system image and 50% of the UHR-CT detection system image. If the operator prefers a texture closer to the UHR-CT detection system image, they may select 75% of the UHR-CT detection system image and 25% of the original NR-CT detection system image. The original NR-CT detection system image and UHR-CT detection system image can be changed from zero to 100% depending on the type of mixing desired by the user. After the mixing step, in step S410, the final DCNN-applicable image approximating the wide-range UHR-CT image is output according to the operator's desired mixing ratio and displayed on the monitor.

[0052] Consider currently commercially available wide-area CT detection systems. These systems may not have the computational power or data buffering capabilities to process advanced networks for wide-area UHR images (e.g., super-resolution 3D networks). We propose a third embodiment, as shown in the flowchart in Figure 7. The inference process shown in Figure 7 includes a resizing operator. To be properly handled in a normal-resolution, wide-area CT detection system, the resizing operator reads hardware and associated software specifications (such as system information) and resizes the output image (e.g., downsampling within the XY or Z dimensions).

[0053] As shown in Figure 7, the process begins in step S500 by collecting wide-area CT detection data from the wide-area CT detection system 100. Next, in step S502, the wide-area CT detection data is reconstructed into a wide-area CT detection image. The wide-area CT detection image is input to a trained DCNN in step S504 and output as a DCNN-applicable CT image that approximates a wide-area UHR-CT image. This method may be continued by considering system information in step S508, such as specific system specifications, and determining the size of the DCNN-applicable CT image that approximates a wide-area UHR-CT image that can be processed by the system based on the system information. This workflow shows that system information is acquired after the output of the DCNN-applicable CT image that approximates a wide-area UHR-CT image, but the system information may be acquired in parallel with the process of outputting the DCNN-applicable CT image that approximates a wide-area UHR-CT image, or before the process of generating the DCNN-applicable CT image that approximates a wide-area UHR-CT image. In step S510, following the acquisition of system information, a resizing operator is used to resize the generated DCNN-applicable CT image, which approximates a wide-range UHR-CT image, based on the information obtained from the system information. As a result, in step S512, a final DCNN-applicable CT image approximating a wide-range UHR-CT image, which can be appropriately processed by a normal-resolution wide-range CT detection system, is output and displayed on the monitor. In one embodiment of this disclosure, system information is acquired from a wide-range CT detection system 100.

[0054] Next, a fourth embodiment of the present disclosure will be described with reference to Figure 8. Figure 8 shows a flowchart of at least one embodiment in which mixing and resizing processes are performed before the generation of the final DCNN-applicable CT image to generate a DCNN-applicable CT image that approximates a wide-area UHR-CT image. The first step S600 includes collecting wide-area CT detection data from a wide-area CT detection scanner 100. Next, in step S602, the wide-area CT detection data is reconstructed into a wide-area CT detection image and input to a trained DCNN in step S604. The trained DCNN outputs a DCNN-applicable CT image that approximates a wide-area UHR-CT image. The mixing step in S608 allows the operator to adjust the output texture to more closely approximate one of the wide-area CT detection image and the DCNN-applicable CT image that closely resembles the wide-area UHR-CT image, according to their preference. After the mixing step is performed, in step S610, in order to properly resize, system information of the detection system 100 is obtained by reading hardware and related software specifications (such as system information). The resizing process in step S612 ensures that the normal-resolution, wide-area CT detection system 100 can reliably and appropriately process the output image (the final DCNN-applicable CT image that approximates the wide-area UHR-CT image). After the resizing step, in step S614, the final DCNN-applicable CT image that approximates the wide-area UHR-CT image is output and displayed on the monitor.

[0055] A fifth embodiment of the present disclosure will be described with reference to Figure 9. Figure 9 shows the workflow of a DCNN training process that performs a denoising task to achieve ultra-high resolution of a wide-area CT detection system 100 while minimizing the noise level associated with UHR-CT data. This embodiment of the present disclosure begins in step S700 with the collection of UHR low-radiation CT data (LD-CT data). UHR-LD-CT data is unreconstructed CT data collected from an ultra-high-resolution CT scanner 200 (actual or simulated) set to low radiation dose. In step S710, a target image is acquired. The target image is a UHR high-radiation (HD) CT image. A UHR-HD-CT image is a reconstructed high-radiation image from the UHR-CT scanner 200 that is used as the training image. In this embodiment, the UHR-HD-CT image and the UHR-LD-CT image are generated from the same UHR-HD-CT data. In another embodiment, the UHR-HD-CT image is generated from different CT data than the CT data from which the UHR-LD-CT data is generated.

[0056] In step S702, the UHR-NR simulation from step S102 is performed on UHR-LD-CT data to acquire NR-LD-CT data. The NR-LD-CT data in step S702 simulates pre-reconstruction CT data collected from a normal resolution scanner system. Next, in step S704, the NR-LD-CT data is reconstructed into an NR-LD-CT image, which is used as the input image for DCNN in step S706. The NR-LD-CT image is a reconstructed image from low-radiation UHR-CT data with reduced image quality (binned) to a smaller pixel size in order to match the training target. In step S706, the image processing device 150 in the wide-area CT detection system 100 applies DCNN to one of the input images and outputs a processed NR-LD-CT image. In step S712, a loss function analysis is performed between the output image (processed NR-LD-CT image) and the training target image (UHR-HD-CT image) to optimize DCNN learning in step S714. In step S716, the continuation of the optimization loop is determined based on the criteria. If the criteria are not met, the optimization loop from step S706 is continued. Alternatively, if the DCNN optimization criteria are met, the optimization loop terminates. The processed NR-LD-CT images become the training output images of the DCNN, which takes low-radiation NR-CT images as input. The loss function applied between the output image and the target image is the same as that described with reference to Figures 2 and 4. The loss function minimizes the difference between the output image and the target image, resulting in a trained DCNN that is optimal for application in the inference process. This allows the DCNN training portion, which aims to suppress noise and improve resolution of DCNN-applicable CT images that approximate wide-range UHR-CT detection images, to be terminated.

[0057] Next, we refer to the flowchart in Figure 10, which shows the inference portion for noise suppression and resolution improvement after acquiring the trained DCNN. The flowchart starts in step S800 by collecting wide-area CT detection data with low radiation dose (LD). Wide-area detection CT data is the unreconstructed CT data collected from the wide-area CT detection scanner 100. In step S802, the wide-area CT detection data (LD) is reconstructed into a wide-area CT detection LD image. A wide-area CT detection LD image is a CT image reconstructed to a smaller pixel size corresponding to the pixel size of the UHR-CT data. Next, in step S804, the trained DCNN is applied to the wide-area CT detection LD image to generate a denoised DCNN-applicable CT image that approximates the wide-area UHR-CT image. In step S806, the processed CT image (DCNN-applicable CT image) is output and displayed on the monitor. The DCNN-applicable CT image (after denoising) that is similar to the wide-area UHR-CT image is advantageous in that it provides a noise-reduced image with a wide segment range and high resolution.

[0058] Various embodiments of this disclosure are applicable to UHR-CT trained DCNNs on wide-area detection CT data. This is advantageous for several reasons. Compared to current wide-area CT detection images, these embodiments provide superior resolution and noise reduction performance, derived from the UHR-CT trained network and finer reconstructed pixel size. Compared to current UHR-CT images, these embodiments offer benefits such as lower radiation dose, improved image uniformity, faster temporal resolution, and a simpler scanning workflow due to the wider detection range (SI direction) of a single scan at the patient's bed position. Furthermore, the larger pixel size obtained from wide-area CT data leads to superior noise performance. Compared to wide-area UHR-CT systems that do not currently exist commercially, these embodiments provide significantly lower costs and reduced complexity in both hardware and software signal processing.

[0059] This disclosure relates to a system, method, and / or apparatus for wide-area, ultra-high-resolution CT to which a deep learning neural network can be applied. The DCNN is trained on an existing UHR-CT detection scanner and applied to wide-area CT detection system data to improve resolution, reduce noise, and preserve the edges of the wide-area scan. Specifically, this disclosure can provide advantages in cost and system complexity compared to a commercially unavailable wide-area UHR-CT detection system 300 by combining the advantages of two different modalities (UHR-CT detection scanner and wide-area CT detection scanner).

[0060] Next, refer to Figures 11 and 12. In at least one embodiment, a console or a computer such as computer 1200, 1200' may be used exclusively for generating DCNN-applicable CT images that approximate wide-area UHR-CT images.

[0061] The electrical signals used for imaging may be transmitted to one or more processors of the computers 1200, 1200', etc. (not limited to these) described below, via cables or wiring 113, etc. (see Figure 11) (not limited to these).

[0062] Figure 11 shows various components of the computer system 1200. The computer system 1200 comprises a central processing unit (CPU) 1201, ROM 1202, RAM 1203, a communication interface 1205, a hard disk (and / or other storage device) 1204, a screen (or monitor interface) 1209, a keyboard (or input interface; which may include a mouse or other input devices in addition to the keyboard) 1210, and buses and other connecting lines (e.g., connecting line 1213) connecting one or more of the above components (e.g., shown in Figure 11). Furthermore, the computer system 1200 may include one or more of the above components. For example, the computer system 1200 may include a CPU 1201, RAM 1203, an input / output (I / O) interface (such as the communication interface 1205), and a bus (which may include one or more wires 1213 as a communication system between the components of the computer system 1200). In one or more embodiments, the computer system 1200 and at least the CPU 1201 may communicate with one or more of the above components of an apparatus or system using an ultra-high-resolution detection scanner and / or wide-area CT detection scanner or the same ultra-high-resolution detection scanner 200, wide-area CT detection scanner 100 (but not limited to these). One or more other computer systems 1200 may comprise one or more combinations of other above components. The CPU 1201 is configured to read and execute computer-executable instructions stored on a storage medium. Computer-executable instructions may include instructions to perform methods and / or operations described in this disclosure. In addition to the CPU 1201, the computer system 1200 may comprise one or more processors. The processors, including the CPU 1201, may be used to control and / or manufacture an apparatus, system or storage medium for generating DCNN-applicable CT images for the same use or approximating wide-area UHR-CT images described in this disclosure. The system 1200 may further comprise one or more processors connected via a network (e.g., network 1206).The CPU 1201 and additional processors used by System 1200 may be located on the same communication network or on different communication networks (for example, enabling remote control of execution, manufacturing, control, and / or technology utilization).

[0063] The I / O or communication interface 1205 provides a communication interface to the input / output device. The input / output device may include an ultra-high resolution detection scanner 200, a wide-area CT detection scanner 100, communication cables and a network (wired or wireless), a keyboard 1210, a mouse (see mouse 1211 in Figure 12), a touchscreen or screen 1209, a light pen, etc. The monitor interface or screen 1209 provides a communication interface to the input / output device.

[0064] The methods and / or data of this disclosure, such as methods for using and / or manufacturing devices, systems, or storage media for the same purpose, and / or methods for generating DCNN-applicable CT images that approximate the wide-area UHR-CT images described in this disclosure, are stored on a computer-readable storage medium. Computer-readable and / or writable storage media are commonly used, but are not limited to, one or more hard disks (hard disk 1204, magnetic disks, etc.), flash memory, CDs, optical discs (compact discs (CDs), digital versatile discs (DVDs), Blu-ray® discs, etc.), magneto-optical disks, RAM (random access memory) (RAM 1203, etc.), DRAM, ROM (read-only memory), storage devices for distributed computing systems, memory cards, etc. (e.g., non-volatile memory cards, solid-state drives (SSDs, see SSD 1207 in Figure 12), other semiconductor memories such as SRAM, etc.), any combination thereof, servers / databases, etc. A computer-readable / writable storage medium may be used to cause a processor, such as the processor or CPU 1201 of the computer system 1200, to perform the procedures of the method described herein. The computer-readable storage medium may be a non-temporary computer-readable medium and / or may comprise all readable media except for temporary propagating signals. The computer-readable storage medium may comprise (but not limited to) RAM (random access memory), register memory, processor cache, etc., which stores information only when there is a predetermined, limited, or short time and / or power.Furthermore, embodiments of the present disclosure may be implemented by a computer in a system or device that reads and executes computer-executable instructions (e.g., one or more programs) recorded on a recording medium (more precisely, also referred to as a non-temporary computer-readable storage medium) to perform one or more functions of the above embodiments, and / or by a computer in a system or device that includes one or more circuits (application-specific integrated circuits (ASICs)) to perform one or more functions of the above embodiments. Embodiments of the present disclosure may be implemented by a computer in a system or device that, for example, reads and executes computer-executable instructions from a storage medium, and / or by controlling the one or more circuits to perform one or more functions of the above embodiments.

[0065] Based on at least one aspect of this disclosure, methods, apparatus, systems, and computer-readable storage media associated with processors such as the processor of computer 1200, the processor of computer 1200', etc., described above, can be realized by utilizing suitable hardware as shown in the drawings. Such hardware can be realized by utilizing any known technology such as any known processor capable of executing standard digital circuits, software and / or firmware programs, programmable read-only memory (PROM), programmable array logic (PAL), or one or more programmable digital devices or systems. The CPU 1201 (as shown in Figure 11 or 12) may consist of and / or one or more microprocessors, nanoprocessors, one or more GPUs (graphics processing units: also referred to as VPUs (visual processing units)), one or more FPGAs (field programmable gate arrays), or other types of processing components (e.g., application-specific integrated circuits: ASICs), etc. Furthermore, various aspects of this disclosure may be implemented by software and / or firmware that can be stored on a suitable storage medium (e.g., a computer-readable storage medium, a hard drive, etc.) or a portable and / or distributable medium (e.g., a floppy disk, a memory chip, etc.). The computer may include a network of individual computers or processors that read and execute computer-executable instructions. Computer-executable instructions may be provided to the computer from, for example, a network or a storage medium.

[0066] As described above, Figure 12 shows the hardware structure of another embodiment of the computer or console 1200'. The computer 1200' includes a central processing unit (CPU) 1201, a GPU 1215, RAM 1203, a network interface 1212, an operation interface 1214 such as USB (Universal Serial Bus), and memory such as a hard disk drive or solid-state image sensor (SSD) 1207. Preferably, the computer or console 1200' includes a display 1209. The computer 1200' may be connected to an ultra-high-resolution detection scanner 200 and / or a wide-area CT detection scanner 100 and / or one or more other components of the system via the network interface 1212 or the operation interface 1214. In one or more embodiments, the computer, such as computer 1200, 1200', may include an ultra-high-resolution detection scanner 200 and / or a wide-area CT detection scanner 100. The operation interface 1214 is connected to an operating unit such as a mouse device 1211, a keyboard 1210, or a touch panel device. Computer 1200' may have two or more of each component. Alternatively, depending on the design of computers such as Computer 1200, 1200', the CPU 1201 or GPU 1215 may be replaced with an FPGA (field programmable gate array), an application-specific integrated circuit (ASIC), or other types of processing units.

[0067] The computer program is stored in SSD1207. The CPU1201 loads the program into RAM1203, executes instructions within the program, and performs basic input, output, arithmetic, and memory write and read operations, along with one or more operations described in this disclosure.

[0068] Computers such as computer 1200, 1200' communicate with the ultra-high resolution detection scanner 200 and / or wide-area CT detection scanner to perform imaging and generate DCNN-applicable CT images that approximate wide-area UHR-CT images. A monitor or display 1209 displays the DCNN-applicable CT images that approximate wide-area UHR-CT images, but may also display other information about imaging conditions and the object being imaged. Monitor 1209 provides a graphical user interface (GUI) for the user to operate the system, for example, when generating DCNN-applicable CT images that approximate wide-area UHR-CT images. Operation signals are input from an operating unit (e.g., a mouse device 1211, a keyboard 1210, or a touch panel device, but not limited to these) to the operation interface 1214 of computer 1200'. In response to the operation signals, computer 1200' instructs the system to set or change imaging conditions and to start or end imaging, and / or to start or end the DCNN training or inference process for generating DCNN-applicable CT images that approximate wide-area UHR-CT images.

[0069] Referring to Figure 13, another exemplary embodiment of the method for generating a trained model for SR is described. This method includes one of the features of the method shown in Figure 2 for generating both UHR-CT images and NR-CT images from UHR-CT data. This method is also similar to the method shown in Figure 9 for training a DCNN for SR using NR low-radiation CT images and UHR high-radiation CT images. Unless otherwise specified, the following steps are performed by the information processing device 400. The CPU or GPU (hereinafter referred to as the processing circuit) performs each step.

[0070] In step S1301, UHR-CT data, i.e., high-resolution CT data, is acquired. In step S1302, the UHR-CT image, i.e., the high-resolution CT image, is reconstructed based on the UHR-CT data. This reconstruction method may be one of the iterative reconstruction methods capable of generating images with a higher resolution than filtered back projection (FBP). The UHR-CT image is used as the target image in the learning phase. In step S1303, the processing circuit generates noisy UHR-CT data by adding noise to the UHR-CT data. By adding Gaussian noise and / or Poisson noise, low-radiation CT data may be simulated more appropriately. In step S1304, resolution reduction processing (n:1 binning, smoothing, other filtering, etc.) is performed on the noisy UHR-CT data to generate low-resolution CT data, which is then simulated. In step S1305, the low-resolution CT image is reconstructed based on the low-resolution CT data. Here, the reconstruction method may be filtered back projection (FBP) or any reconstruction method commonly used in clinical settings. Typically, there are various options for the reconstruction function and FBP filter, but in step S1305, for FBP reconstruction, it is possible to select a reconstruction function with little or no noise reduction effect in order to preserve the signal as much as possible. It is also possible to select one of the filters with little or no normalization effect. Low-resolution CT images are used as input images to train the SR-DCNN. The corresponding input images and target images generated from UHR data are associated with each other and form training data pairs. Steps S1301 to S1305 are repeated for different UHR data to generate multiple training data pairs. In step S1306, a processed CT image is obtained by applying the DCNN to one of the input images. In step S1307, the loss function described in step S306 in Figure 4 is obtained. In step S1308, the DCNN is optimized and used as an improved DCNN in the next step S1306. The loop in steps S1306, S1307, and S1308 continues to train the DCNN for super-resolution processing until the termination criterion is met ("Y" in S1309).

[0071] The binning process in step S1303 has some noise reduction effect, but due to differences in the noise addition process and reconstruction method in step S1303, the target image (high-resolution CT image) has higher spatial resolution and higher noise characteristics than the input image (low-resolution CT image). The trained SR-DCNN exhibits both noise reduction and super-resolution effects. In another embodiment, instead of or in addition to noise addition to CT data, noise may be added to the low-resolution reconstructed CT image to obtain the input image in the projection domain. The above DCNN is trained to exhibit both denoising and super-resolution effects and may be more beneficial in at least some situations than applying both separately trained denoising DCNN and SR-DCNN. Hereinafter, another embodiment of the inference stage processing is described. This processing involves applying another type of DCNN model specifically for denoising CT images and selecting either (1) a denoising DCNN or (2) a DCNN trained by the above method, referencing Figure 7 or Figure 13. Note that DCNNs (hereinafter sometimes referred to as SR and denoising DCNN) can exhibit both noise reduction and super-resolution effects. The inference stage execution process may be incorporated into an image processing device 150 within the wide-area CT detection system 100, a console of another type of CT imaging system, or an image processing device outside the CT imaging system, such as a hospital workstation or an image processing server that receives medical images, analyzes medical data, and reconstructs medical images. In the following description, each step is performed by a processing circuit which is a CPU or GPU included in the image processing device, console, workstation, or image processing server.

[0072] A denoising DCNN can be trained using multiple pairs of training images. In one embodiment, the input image may be a low-radiation CT image, and the corresponding target image may be a high-radiation CT image. Low-radiation CT images can be collected by CT scanning of the object to be examined. Similarly, high-radiation CT images can be collected by CT scanning of the object to be examined. Furthermore, low-radiation CT images can also be generated from high-radiation CT images collected by CT scanning by adding noise and simulating the low-radiation image. High-radiation CT images can be generated from low-radiation CT images collected by CT scanning by image processing that simulates the high-radiation image. In another embodiment, the target image can be obtained by iterative reconstruction processing of CT data collected by CT scanning of the object to be examined, and the input image can be obtained by adding noise to the CT data based on the FBP method and reconstructing the noisy CT data. The denoising model can exhibit effects in reducing various types of artifacts.

[0073] In the first step, if the process is executed within a CT imaging system, CT data is acquired from the CT detector. If the process is executed within a workstation or image processing server, CT data can be acquired from the CT imaging system. Furthermore, if the process is executed within a CT imaging system, workstation, or image processing server, CT data can also be acquired from memory.

[0074] In the second step, the processing circuit determines whether to apply a denoising model (denoising DCNN) or an SR model (denoising and SR-DCNN) to the reconstruction process of the obtained image. If the denoising model is selected, the processing circuit reconstructs the first CT image according to the first reconstruction conditions and the second CT image according to the second reconstruction conditions. Compared to the first reconstruction conditions, the second reconstruction conditions include a reconstruction function selected for FBP reconstruction and a filter with little or no noise reduction effect to preserve image information. Also, because the pixel density or number of pixels in the reconstruction region is greater under the second reconstruction conditions than under the first reconstruction conditions, the resolution is further improved in the SR processing of the second CT image. If specific noise reduction processing is performed in the reconstruction of both the first and second CT images, the degree of noise reduction for the second CT image may be lower than that for the first CT image in order to preserve the image information of the second CT image.

[0075] In the next step, a denoising DCNN is applied to the first CT image to obtain a denoised CT image. Then, SR and a denoising DCNN are applied to the first CT image to obtain an SR-CT image. The obtained CT images are output and displayed or analyzed. If the obtained CT images are output and displayed, the processing circuit generates a graphical user interface containing the obtained CT images and sends it to the display. If the display is connected to an image processing device or workstation, the processing circuit causes the display device to display the obtained CT images.

[0076] The above process does not include the case where no DCNN model is selected to apply, in which case the processing circuit selects either a first reconstruction condition or a third reconstruction condition different from the first and second reconstruction conditions.

[0077] In one embodiment, multiple SR (and denoising) DCNNs and multiple DCNN models can be created on a somato-type basis. That is, regardless of whether they are for SR or denoising, DCNN models can be trained for specific somato-type areas and / or specific clinical applications by using only images for those specific somato-type areas and / or specific clinical applications.

[0078] If multiple SR-DCNNs are stored in memory, the processing circuit selects one of them depending on the body part being imaged.

[0079] Reconstruction conditions and DCNN can be selected based on scan information, either before or after CT data acquisition.

[0080] Referring to Figure 14, another exemplary embodiment of the present disclosure is described. In this embodiment, the DCNN model is applied to CT data (CT projection data) in the projection domain, and the image is reconstructed. In one embodiment, the DCNN can be trained using multiple pairs of training data, each containing target data which is UHR-CT data and input data which is NR-CT data. The NR-CT data can be collected by a different scan than the UHR-CT data, or can be generated by performing the UHR-NR simulation described above. In another embodiment, the target data is HD-UHR-CT data, and the input data is LD-NR-CT data. The LD-NR-CT data can be collected by a different scan than the HD-UHR-CT data, or can be generated by adding noise and performing the UHR-NR simulation described with reference to Figure 13.

[0081] The steps shown below are performed by the CPU or GPU of the image processing device 150 (processing circuit), but can also be performed by the processing circuit of a workstation or image processing server.

[0082] In step S1400, wide-area CT data is acquired. In another embodiment, the CT data may be normal-area CT data. In step S1402, a trained DCNN is applied to the wide-area CT data to acquire wide-area SR-CT data. In step S1404, reconstruction processing is performed on the wide-area SR-CT data to acquire wide-area SR-CT images. In step S1406, the wide-area SR-CT images are output and displayed or further analyzed.

[0083] Figure 15 shows an example of an embodiment of an X-ray stand apparatus included in a CT imaging system corresponding to a wide-area CT detection system. As shown in Figure 15, the X-ray stand apparatus 1500, shown from the side, comprises an X-ray tube 1501, an annular frame 1502, and a multi-row or two-dimensional array type X-ray detector 1503. The X-ray tube 1501 and the X-ray detector 1503 are positioned opposite each other with respect to an object OBJ on the annular frame 1502. The annular frame 1502 is rotatably supported around a rotation axis RA. A rotating unit 1507 rotates the annular frame 1502 at a predetermined speed while the object OBJ is moved in or out of the drawing along the rotation axis RA. The console or image processing apparatus 1550 comprises a reconstruction apparatus 1514, a storage device 1512, a display device 1516, an input device 1515, and a preprocessing apparatus 1506.

[0084] There are various types of X-ray CT scanners. For example, there are rotary-rotating scanners in which both the X-ray tube and X-ray detector rotate around the object being examined, and stationary-rotating scanners in which many detection elements are arranged in a ring or plane, and only the X-ray tube rotates around the object being examined. The present invention is applicable to any type of scanner. Here, we will illustrate with the rotary-rotating scanner, which is currently the mainstream type.

[0085] The multislice X-ray CT scanner further includes a high-voltage generator 1509 that generates a tube voltage. The tube voltage is applied to the X-ray tube 1501 via a slip ring 1508 so that the X-ray tube generates X-rays. The X-rays are emitted toward the object OBJ. The tomographic region of the object OBJ is indicated by a circle. For example, the average X-ray energy of the X-ray tube during the first scan is less than the average X-ray energy during the second scan. That is, two or more scans are acquired corresponding to different X-ray energies. The X-ray detector 1503 is located on the opposite side of the X-ray tube 1501 from the object OBJ and detects the emitted X-rays that have passed through the object OBJ. The X-ray detector 1503 further includes individual detection elements or detection units.

[0086] The CT scanner further includes other devices for processing detection signals from the X-ray detector 1503. The data acquisition circuit or data acquisition system (DAS) 1504 converts and amplifies the signals output from the X-ray detector 1503 channel by channel into voltage signals, and then converts them into digital signals. The X-ray detector 1503 and DAS 1504 are configured to process the number of projections per revolution (TPPR).

[0087] The above data is transmitted to a preprocessor 1506 located on a console outside the X-ray rig 1500 via a non-contact data transmitter 1505. The preprocessor 1506 performs predetermined corrections, such as sensitivity correction, on the raw data. The memory 1512 stores the resulting data (also called projection data immediately before reconstruction). The memory 1512, along with the reconstruction unit 1514, input unit 1515, and display device 1516, is connected to the system controller 1510 via a data / control bus 1511. The system controller 1510 controls a current regulator 1513 that limits the current to the level required to drive the CT system.

[0088] In this example configuration of the CT imaging apparatus, the reconstruction apparatus 1514 performs the processes and methods described in Figures 3, 6, 7, 8, 10, and 14.

[0089] The detector is rotated and / or stationary relative to the patient while the CT scanner system performs various generation tasks. In one embodiment, the CT system described above is an example of a combined system of a third-generation and a fourth-generation configuration. In the third-generation system, the X-ray tube 1501 and the X-ray detector 1503 are positioned opposite each other on an annular frame 1502 and rotate in accordance with the rotation of the annular frame 1502 around its axis of rotation RA. In the fourth-generation system, the detector is fixedly positioned around the patient, and the X-ray tube rotates around the patient. In another embodiment, the X-ray stand apparatus 1500 comprises a plurality of detectors arranged on an annular frame 1502 supported by a C-arm and a base.

[0090] Memory 1512 can store measured values ​​indicating the X-ray irradiance at the X-ray detector 1503.

[0091] Furthermore, the reconstruction device 1514 can perform pre-reconstruction image processing such as volume rendering and image difference processing as needed.

[0092] The reconstruction preprocessing of projection data performed by the preprocessor 1506 may include, for example, detector calibration, correction for detector nonlinearity, and polarity effects.

[0093] The post-reconstruction processing performed by the reconstruction device 1514 may include, as necessary, image filtering and smoothing, volume rendering, and image difference processing. The reconstruction device 1514 can store, for example, projection data, reconstructed images, calibration data, parameters, computer programs, etc., in memory.

[0094] The reconfiguration device 1514 may include a CPU (processing circuit). The CPU can be implemented as a discrete logic gate, an application-specific integrated circuit (ASIC), an FPGA (Field-Programmable Gate Array), or other complex programmable logic device (CPLD). The FPGA or CPLD implementation may be encrypted using VHDL, Verilog, or other hardware description languages. The encryption may be stored directly in the electronic memory within the FPGA or CPLD, or in separate electronic memory. Furthermore, the memory 1512 may be non-volatile, such as ROM, EPROM, EEPROM, or flash memory. The memory 1512 may also be volatile, such as static or dynamic RAM. It is also possible to provide a processor, such as a microcontroller or microprocessor, to manage the interaction between the FPGA or CPLD and the memory together with the electronic memory.

[0095] Furthermore, the CPU of the reconfiguration device 1514 is capable of executing a computer program that includes a set of computer-readable instructions for performing the functions described herein. The computer program is stored in the non-temporary electronic memory and / or on a hard disk drive, CD, DVD, flash drive, or other known storage medium. In addition, the computer-readable instructions can be provided as a utility application, background daemon, operating system component, or a combination thereof, which runs in cooperation with a given processor and a given operating system or any operating system known to those skilled in the art. The CPU can also be implemented as multiple processors that work together in parallel to execute the instructions.

[0096] In one embodiment, the reconstructed image can be displayed on the display device 1516. The display device 1516 can be an LCD display, a CRT display, a plasma display, an OLED, an LED, or any other known display device.

[0097] Memory 1512 can be a hard disk drive, CD-ROM drive, DVD drive, flash drive, RAM, ROM, or other known electronic storage device.

[0098] Figure 16 shows an example of a medical image processing system having a client-server configuration involving an intermediate network. As shown in Figure 16, the medical image processing system includes a medical image diagnostic device 1601 as the client-side device and a medical image processing device 1610 as the server-side device connected to the medical image diagnostic device 1601 via network N.

[0099] The medical imaging diagnostic device 1601 may generally be an X-ray CT scanner as shown in Figure 15 or a wide-area CT detection system 100 as shown in Figure 2.

[0100] The medical image processing device 1610 comprises a transmitter / receiver 1611, a memory 1612, and a processing circuit 1613. The processing circuit 1613 includes a reconstruction device 1614, which includes a reconstruction processor 16141 and an image processor 16142. The transmitter / receiver 1611 transmits and receives data to and from the medical image diagnostic device 1601 via the network N. The memory 1612 stores information such as medical image data received from the medical image diagnostic device 1601, as well as various dedicated programs for performing the above-mentioned reconstruction processing, denoising processing, etc. The processing circuit 1613 is a processor that realizes the functions of the above-mentioned reconstruction device 1614.

[0101] With these configurations, the medical imaging diagnostic device 1601 does not need to implement the functions of the reconstruction device 1514 shown in Figure 15. Therefore, the processing load and associated costs of the medical imaging diagnostic device 1601 can be reduced. In addition, the reconstruction and denoising processes are performed in a unified manner by the medical image processing device 1610, which is the server side. This makes it possible to prevent variations in image quality, etc., that may occur due to differences in operators used when performing reconstruction and denoising processes on each medical imaging diagnostic device in the field.

[0102] According to at least one embodiment described above, it is possible to generate medical images that improve the visibility of objects such as anatomical features in medical images and improve image quality.

[0103] While this disclosure has been described with reference to several embodiments, these embodiments are presented as examples of the principles and uses of this disclosure and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0104] With regard to the above embodiments, the following additional notes are disclosed as aspects of the invention and selective features. (Note 1) A first CT apparatus having a detector of a first pixel size is used to perform a first CT scan on a subject using a first imaging region of the detector, thereby acquiring a first set of projection data. A first CT image having a first resolution is obtained by reconstructing the first group of projection data. By applying a machine learning model to improve the resolution of the first CT image, a processed CT image with a higher resolution than the first image is obtained. The processed CT image is output for display or analysis. The machine learning model is obtained by machine learning using a second CT image based on a second set of projection data obtained by performing a second CT scan on a subject using a second imaging region smaller than the first imaging region of a second CT device having a detector with a second pixel size smaller than the first pixel size. A medical image processing method comprising the following. (Note 2) The medical image processing method may, when applying the machine learning model, generate the first CT image by reconstructing the first projection data set with a first matrix size, or, when not applying the machine learning model, generate other CT images by reconstructing the first projection data set with a second matrix size smaller than the first matrix size. (Note 3) The first matrix size may be any of the following: 512×512, 1024×1024, 2048×2048, or 4096×4096. (Note 4) The second matrix size may be any of the following: 256×256, 512×512, 1024×1024, or 2048×2048. (Note 5) The first matrix size may be 1024 × 1024 or larger, and the second matrix size may be 512 × 512 or larger. (Note 6) The aforementioned medical image processing method is When applying the machine learning model, the first CT image is generated by using the first projection data set and performing a reconstruction process based on the first reconstruction function. Instead of applying the aforementioned machine learning model, if another machine learning model different from the aforementioned machine learning model is to be applied to reduce noise, another CT image may be generated by using the first projection data set and performing a reconstruction process based on a second reconstruction function that has a greater noise reduction effect than the first reconstruction function, and the other machine learning model may be applied to the other CT image. (Note 7) The medical image processing method may acquire the processed CT image by combining the first CT image and an image obtained by applying the machine learning model to the first CT image in a predetermined ratio. (Note 8) The predetermined ratio may be obtained based on user input or from a set of imaging conditions. (Note 9) The aforementioned medical image processing method, in applying the machine learning model, Multiple 3D partial images are generated based on the first CT image, By inputting the multiple 3D partial images into a designated model among the aforementioned machine learning model and other machine learning models, the designated model is applied to obtain multiple processed 3D partial images. A processed image may be obtained by combining the aforementioned multiple processed 3D partial images. (Note 10) The medical image processing method may generate the plurality of 3D partial images such that at least two of the plurality of 3D partial images partially overlap. (Note 11) The medical image processing method may combine the multiple processed 3D partial images by applying a filter to the joint between two adjacent processed 3D partial images. (Note 12) The machine learning model may be a machine learning model for applying super-resolution processing to the first CT image. (Note 13) The machine learning model may be a machine learning model for applying super-resolution processing and noise reduction processing to the first CT image. (Note 14) According to the medical image processing method described above, in generating the machine learning model, the machine learning model may be trained using the training images, which include the second CT image and a third CT image generated based on either the second CT image or the second projection data set, having lower resolution and greater noise than the second CT image. (Note 15) According to the medical image processing method described above, in generating the machine learning model, the machine learning model may be trained using the training images, which include the second CT image and a fourth CT image based on a third projection data set obtained by applying noise addition processing and further resolution reduction processing to the second projection data set. (Note 16) A first CT apparatus having a detector of a first pixel size, an acquisition unit that acquires a first group of projection data obtained by performing a first CT scan on a subject using a first imaging region of the detector, A processing unit that obtains a first CT image having a first resolution by reconstructing the first group of projection data, and obtains a processed CT image with a resolution higher than the first resolution by applying a machine learning model to improve the resolution of the first CT image, An output unit that outputs the processed CT image for display or analysis processing, Equipped with, The machine learning model is obtained by machine learning using a second CT image based on a second set of projection data obtained by performing a second CT scan on a subject using a second imaging region smaller than the first imaging region of the detector, with the second CT device having a detector with a second pixel size smaller than the first pixel size. Medical image processing equipment. (Note 17) The X-ray CT scanner may have the medical image processing device described in Appendix 16. (Note 18) On the computer, A first CT apparatus having a detector of a first pixel size is used to perform a first CT scan on a subject using a first imaging region of the detector, thereby acquiring a first set of projection data. A first CT image having a first resolution is obtained by reconstructing the first group of projection data. By applying a machine learning model to improve the resolution of the first CT image, a processed CT image with a higher resolution than the first image is obtained. The processed CT image is then displayed or output for analysis. The machine learning model is obtained by machine learning using a second CT image based on a second set of projection data obtained by performing a second CT scan on a subject using a second imaging region smaller than the first imaging region of the detector, with the second CT device having a detector with a second pixel size smaller than the first pixel size. Medical image processing program. [Explanation of symbols]

[0105] 100 Wide-Range CT Detection System, Standard Resolution - Wide-Range CT Detection System, Wide-Range CT Detection Scanner 101 Wide-area CT data 113 Cable or wiring 150 Image Processing Devices 200 UHR-CT detection system, ultra-high resolution detection scanner, ultra-high resolution CT scanner, ultra-high resolution CT detection scanner, UHR-CT detection scanner, UHR-CT scanner 201 UHR-CT data 300 Wide-Range UHR-CT Detection System, Wide-Range UHR-CT Scanner System 301 Wide-area UHR-CT images 400 Information Processing Devices 401 Pre-trained models for super-resolution (SR), DL networks, machine learning models, pre-trained models 1200 Computer systems, systems, computers 1200' Computer, Console 1201 Central Processing Unit (CPU) 1202 ROM 1203 RAM 1204 Hard disk (and / or other storage device) 1205 Communication Interface 1207 SSD (Solid State Drive) 1209 Screen (or monitor interface), display 1210 Keyboard (or input interface; may include a mouse or other input devices in addition to the keyboard) 1211 Mouse device, mouse 1212 Network Interface 1213 Connection wires, wiring 1214 Operation Interface 1215 GPU 1500 X-ray rigging system 1501 X-ray tube 1502 Circular Frame 1503 Multi-row or two-dimensional array type X-ray detector, X-ray detector, 1504 Data acquisition system (DAS) 1505 Contactless Data Transmitter 1506 Preprocessing device 1507 Rotating part 1508 Slip Ring 1510 System Controller 1511 Control bus, Data / Control bus 1512 Storage devices, memory 1513 Current regulator 1514 Reconfiguration device 1515 Input Device 1516 Display device 1550 Console or image processing device 1601 Medical imaging diagnostic equipment 1610 Medical Image Processing Equipment 1611 Transmitter / Receiver 1612 memory 1613 Processing Circuit 1614 Reconfiguration device 16141 Reconfiguration Processor 16142 Image Processor

Claims

1. A first CT apparatus having a detector of a first pixel size is used to perform a first CT scan on a subject using a first imaging region of the detector, thereby acquiring a first set of projection data. A first CT image having a first resolution is obtained by reconstructing the first group of projection data. By applying a machine learning model to improve the resolution of the first CT image, a processed CT image with a higher resolution than the first resolution is obtained. The processed CT image is output for display or analysis. The machine learning model is obtained by machine learning using a downsampled image obtained by downsampling a second CT image, which is obtained by performing a second CT scan on a subject using a second imaging region smaller than the first imaging region of a detector in a second CT apparatus having a detector with a second pixel size smaller than the first pixel size, using the second imaging region of the detector, to the first pixel size, and the second CT image. A medical image processing method comprising the following.

2. In acquiring the processed CT image, the processed CT image is acquired by combining the first CT image and an image obtained by applying the machine learning model to the first CT image in a predetermined ratio. The medical image processing method according to claim 1.

3. The predetermined ratio is obtained based on user input or from a set of imaging conditions. The medical image processing method according to claim 2.

4. In applying the aforementioned machine learning model, Multiple 3D partial images are generated based on the first CT image. By inputting the multiple 3D partial images into the machine learning model, multiple processed 3D partial images are obtained. A processed image is obtained by combining the aforementioned multiple processed 3D partial images. The medical image processing method according to claim 1.

5. In generating the plurality of 3D partial images, the plurality of 3D partial images are generated such that at least two of the plurality of 3D partial images partially overlap. The medical image processing method according to claim 4.

6. In synthesizing the multiple processed 3D partial images, a filter is applied to the joint between two adjacent processed 3D partial images to synthesize the multiple processed 3D partial images. The medical image processing method according to claim 4.

7. The aforementioned machine learning model is a machine learning model for applying super-resolution processing to the first CT image. A medical image processing method according to any one of claims 1 to 6.

8. The machine learning model is a machine learning model for applying super-resolution processing and noise reduction processing to the first CT image. A medical image processing method according to any one of claims 1 to 7.

9. In the generation of the machine learning model, the machine learning model is trained using the training images, which are the second CT image and a third CT image generated based on either the second CT image or the second projection data set, having lower resolution and higher noise than the second CT image. A medical image processing method according to any one of claims 1 to 8.

10. In the generation of the machine learning model, the second CT image and a fourth CT image based on a third projection data set obtained by applying noise addition processing and further resolution reduction processing to the second projection data set are used as training images, and the machine learning model is trained using the training images. The medical image processing method according to claim 9.

11. A first CT apparatus having a detector of a first pixel size, an acquisition unit that acquires a first group of projection data obtained by performing a first CT scan on a subject using a first imaging region of the detector, A processing unit that obtains a first CT image having a first resolution by reconstructing the first group of projection data, and obtains a processed CT image with a resolution higher than the first resolution by applying a machine learning model to improve the resolution of the first CT image, An output unit that outputs the processed CT image for display or analysis, Equipped with, The machine learning model is obtained by machine learning using a second CT image obtained by downsampling a second CT image, which is obtained by performing a second CT scan on a subject using a second imaging region smaller than the first imaging region of the detector in a second CT apparatus having a detector with a second pixel size smaller than the first pixel size, and the second projection data group obtained by downsampling the second CT image to the first pixel size, and the second CT image. Medical image processing equipment.

12. An X-ray CT apparatus having the medical image processing apparatus described in claim 11.

13. On the computer, A first CT apparatus having a detector of a first pixel size is used to perform a first CT scan on a subject using a first imaging region of the detector, thereby acquiring a first set of projection data. A first CT image having a first resolution is obtained by reconstructing the first group of projection data. By applying a machine learning model to improve the resolution of the first CT image, a processed CT image with a higher resolution than the first resolution is obtained. The processed CT image is output for display or analysis. To make it happen, The machine learning model is obtained by machine learning using a second CT image obtained by downsampling a second CT image, which is obtained by performing a second CT scan on a subject using a second imaging region smaller than the first imaging region of the detector in a second CT apparatus having a detector with a second pixel size smaller than the first pixel size, and the second projection data group obtained by downsampling the second CT image to the first pixel size, and the second CT image. Medical image processing program.

Images