Method and system for generating dual energy images from single energy imaging system

Abstract

The invention relates to a method and system for generating dual energy images from a single energy imaging system. Various methods and systems are provided for transforming an image from one energy level to another. In an example, a method includes obtaining (802) an image at a first energy level acquired with a single energy computed tomography (CT) imaging system; identifying (804) a contrast period of the image; logging (812, 814) the image as input into an energy transformation model trained to output a transformed image at a second energy level different from the first energy level, the energy transformation model being selected (808, 810) from a plurality of energy transformation models based on the comparison period; and displaying (818) a final transformed image and / or saving the final transformed image in a memory, wherein the final transformed image is the transformed image or is generated based on the transformed image.

Description

Technical Field

[0001] Embodiments of the subject matter disclosed herein relate to medical imaging, and more particularly to computed tomography (CT). Background Art

[0002] In a computed tomography (CT) imaging system, an x-ray source emits an x-ray beam toward a subject or object (such as a patient). After being attenuated by the subject, the x-ray beam is projected onto a detector array. The intensity of the attenuated beam radiation received at the detector array depends on the attenuation of the x-ray beam by the subject. Each detector element of the detector array generates a separate electrical signal that is transmitted to a data processing system for analysis and generation of medical images. CT scans at various energy levels can provide improved quality of tissue characterization and contrast quantification / visualization. Summary of the invention

[0003] In one example, a method includes obtaining an image at a first energy level acquired using a single-energy computed tomography (CT) imaging system; identifying a contrast period of the image; entering the image as input into an energy transformation model trained to output a transformed image at a second energy level different from the first energy level, the energy transformation model being selected from a plurality of energy transformation models based on the contrast period; and displaying a final transformed image and / or storing the final transformed image in a memory, wherein the final transformed image is the transformed image or is generated based on the transformed image.

[0004] It should be understood that the above brief description is provided to introduce in a simplified form selected concepts that are further described in the detailed description. It is not meant to identify key features or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that solve any disadvantages noted above or in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The present disclosure will be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:

[0006] Figure 1 shows a pictorial view of a CT imaging system incorporating the disclosed embodiments;

[0007] Figure 2 Shows Figure 1 A schematic block diagram of the system illustrated in FIG.

[0008] Figure 3 An example image processing system is schematically illustrated;

[0009] Figure 4 Schematically illustrating an example process of generating transformed images from an initial image using a deep learning (DL) based framework, wherein the transformed images are at different energy levels than the initial image;

[0010] Figure 5 schematically illustrates an example process for sequentially generating transformed images from initial images at different energy levels;

[0011] Figure 6 An example process for training a contrast period classifier is schematically illustrated;

[0012] Figure 7 An example process for training an energy transformation model is schematically illustrated;

[0013] Figure 8 is a flow chart illustrating a method for transforming an image from a first energy level to a second energy level using a contrast period classifier and an energy transformation model according to an embodiment of the present disclosure;

[0014] Fig. 9 is a flow chart illustrating a method for transforming an image from a first energy level to a second energy level and a third energy level using a contrast period classifier and a plurality of energy transformation models according to an embodiment of the present disclosure;

[0015] Fig.10 is a flow chart illustrating a method for training a contrast period classifier according to an embodiment of the present disclosure;

[0016] Fig.11 is a flowchart illustrating a method for training an energy conversion model according to an embodiment of the present disclosure; and

[0017] Figures 12 to 17 Example images at various energy levels generated in accordance with the disclosed embodiments are shown. DETAILED DESCRIPTION

[0018] The following description relates to transforming an image from one energy level to another energy level. Specifically, the following description relates to transforming an image obtained at a single peak energy (e.g., using a monospectral computed tomography (CT) system) to one or more different energy levels. Unlike a single energy CT imaging system, projection data obtained using a dual energy CT imaging system can be used to generate a CT image at any selected energy level. For example, a dual energy CT imaging system can obtain projection data at a first higher peak energy level (e.g., 140 kVp) and a second lower peak energy level (e.g., 40 kVp) in an interlaced or sequential manner, and by performing a linear combination of material-based images, a virtual monochromatic image can be generated at any desired energy level (keV) between 40 keV and 140 keV. Therefore, a dual energy CT imaging system may be beneficial for certain imaging tasks. For example, images at lower energy levels can increase contrast visualization of a subject's region of interest (ROI) of a CT image, which can reduce the frequency of missed or misdiagnosed conditions, particularly in tumor applications. However, CT images at lower energy levels are also prone to noise and image artifacts, which degrade visualization of the subject's ROI and reduce overall image quality.

[0019] Although dual-energy CT imaging systems offer many advantages in terms of image quality, dual-energy CT imaging systems may not be available at all imaging facilities. In addition, the types of scans that can be performed by dual-energy CT imaging systems may be limited. Although single-energy CT imaging systems may be more widespread and increase the types of scans that can be performed, images generated from single-energy CT imaging systems may not include the increased contrast visualization described above, or conversely, lower energy CT images obtained using single-energy CT imaging systems may exhibit noise and image artifacts.

[0020] Therefore, the problems described above can be solved by transforming an image obtained at a single peak energy level (e.g., 120kVp) to one or more different energy levels, such as transforming an image obtained at a higher energy level to look like an image obtained at a lower energy level. An image can be transformed using a deep learning-based energy transformation model trained for a specific energy transformation (e.g., trained to transform an image from 70keV to 50keV). However, transforming images to different energy levels can be challenging. Specifically, when transforming an image to different energy levels, not only is the contrast period transformed, but each tissue present in the image is also transformed, thereby increasing the complexity of the image transformation process. As described in more detail below, in order to transform an image, for example, in addition to the values ​​of water density tissue, fat density tissue, bone density tissue, etc., the contrast values ​​for each contrast period are mapped from a higher energy level image to a lower energy level image.

[0021] A plurality of energy transformation models may be utilized to achieve appropriate mapping, wherein each energy transformation model corresponds to (e.g., is trained for) a particular contrast period and a particular energy level transformation. The energy level transformation may include transforming an initial image obtained at a predetermined first energy level (e.g., 120 kVp, which may be equal to 70 keV) into a final transformed image that appears to be obtained at a predetermined second energy level (50 keV). The energy transformation model may be selected based on the contrast period of the initial image, which may be determined based on the output from a contrast period classifier model that identifies contrast periods present in the initial image. In this manner, an energy transformation model may be selected based on a single contrast period.

[0022] In some examples, the initial image may be obtained during a transition from one contrast period to another contrast period. In such examples, the contrast period classifier may identify more than one contrast period (e.g., two contrast periods) in the initial image. Based on the identified contrast periods, more than one energy transformation model may be selected, for example, an energy transformation model corresponding to each identified contrast period. The initial image may be input to each selected energy transformation model. A corresponding transformed image may be output from each selected energy transformation model, each transformed image corresponding to a particular contrast period. A final transformed image at a second energy level may be generated by blending the transformed images. Blending the transformed images may include weighting each transformed image based on a ratio of the identified contrast periods.

[0023] In some examples, it may be desirable to transform the initial image to a different energy level having a relatively large difference in energy level, such as to a lower energy level than the energy transform described above, to further increase contrast visibility and visualization of the subject's ROI. However, the quality of the transformed image may depend on the magnitude of the energy transform, e.g., the energy level change between the initial energy level and the final energy level. As an example, when the magnitude of the energy transform is too large (e.g., transforming from 120 kVp to 30 keV), the mapping of contrast periods and each tissue type from the initial energy level to the final energy level may be unsatisfactory, which in turn may reduce the image quality of the transformed image and reduce the visualization of the subject's ROI.

[0024] Therefore, it may be beneficial to sequentially transform an initial image at a first energy level into a first transformed image at a second energy level, and then transform the first transformed image into a second transformed image at a third energy level. The first transformed image may be transformed into the second transformed image at the third energy level by entering the first transformed image into an energy transformation model corresponding to a different energy level transformation (e.g., from 50 keV to 30 keV). In this way, the initial image at the first energy level may be transformed to the third energy level without degrading image quality.

[0025] Figure 1 An exemplary CT imaging system 100 configured for CT imaging is illustrated. In particular, the CT imaging system 100 is configured to image a subject 112 (such as a patient, an inanimate object (such as a phantom), one or more manufactured parts), and / or a foreign object (such as a dental implant, an artificial joint, a stent, and / or a contrast agent present in the body). In one embodiment, the CT imaging system 100 includes a gantry 102, which in turn may also include at least one x-ray source 104, which is configured to project an x-ray radiation beam for imaging the subject 112. Specifically, the x-ray source 104 is configured to project x-rays toward a detector array 108 positioned on an opposite side of the gantry 102. Although Figure 1 A single x-ray source 104 is depicted, but in some embodiments, multiple x-ray sources and detectors may be employed to project multiple x-rays for acquisition of projection data at different energy levels corresponding to, for example, a patient. In some embodiments, the x-ray source 104 may achieve dual energy spectrum imaging by fast peak kilovolt (kVp) switching. In some embodiments, the x-ray detector employed is a photon counting detector capable of distinguishing x-ray photons of different energies. In other embodiments, the x-ray detector is an energy integrating detector, in which the detected signal is proportional to the total energy deposited by all photons without specific information about each individual photon or its energy. In some embodiments, two sets of x-ray sources and detectors are used to generate dual energy projections, in which one set of x-ray sources and detectors is set to low kVp and the other set is set to high kVp. It should therefore be understood that the methods described herein may be implemented with single energy acquisition techniques as well as dual energy acquisition techniques to train an energy transformation deep learning (DL) model and obtain images for input into the trained energy transformation DL model.

[0026] In certain embodiments, the CT imaging system 100 further includes an image processor unit 110 configured to reconstruct an image of a target volume of the subject 112 using an iterative or analytical image reconstruction method. For example, the image processor unit 110 may use an analytical image reconstruction method such as filtered back projection (FBP) to reconstruct an image of the patient's target volume. As another example, the image processor unit 110 may use an iterative image reconstruction method (such as advanced statistical iterative reconstruction (ASIR), conjugate gradient (CG), maximum likelihood expectation maximization (MLEM), model-based iterative reconstruction (MBIR), etc.) to reconstruct an image of the target volume of the subject 112. In some examples, in addition to the iterative image reconstruction method, the image processor unit 110 may also use an analytical image reconstruction method such as FBP. In some embodiments, the image processor unit 110 may use a direct image reconstruction method, such as a neural network trained using deep learning.

[0027] In some CT imaging system configurations, an x-ray source 104 emits a cone beam that is collimated to lie in a plane of an XYZ Cartesian coordinate system and is often referred to as an "imaging plane". The radiation beam passes through an object being imaged, such as a patient or subject 112. The beam, after being attenuated by the object, is projected onto a detector array 108 that includes radiation detectors. The intensity of the attenuated radiation beam received at the detector array 108 depends on the attenuation of the radiation beam by the object. Each detector element of the array produces a separate electrical signal that is a measure of the beam attenuation of the ray path between the source and the detector element. The attenuation measurements from all detector elements are collected individually to produce a transmit profile.

[0028] In some CT imaging systems, a gantry is used to rotate the radiation source and detector array around the object to be imaged in the imaging plane so that the angle at which the radiation beam intersects the object is constantly changing. A set of radiation attenuation measurements (e.g., projection data) from the detector array at one gantry angle is called a "view". A "scan" of an object includes a set of views made at different gantry angles or viewing angles during one rotation of the radiation source and detector. It is conceivable that the benefits of the methods described herein derive from medical imaging modalities other than CT, so as used herein, the term "view" is not limited to the use described above with respect to projection data from one gantry angle. The term "view" is used to mean a data acquisition whenever there are multiple data acquisitions from different angles (whether from CT, positron emission tomography (PET) or single photon emission CT (SPECT) acquisitions), and / or any other modality (including modalities yet to be developed) and their combination in fusion or hybrid embodiments.

[0029] The projection data is processed to reconstruct an image corresponding to a two-dimensional slice acquired through the object, or in some examples where the projection data includes multiple rotations or scans or a two-dimensional (2D) array of detectors, a three-dimensional (3D) rendering of the object is reconstructed. A method for reconstructing an image from a set of projection data is known in the art as a filtered back projection technique. Transmission and emission tomography reconstruction techniques also include statistical iterative methods such as maximum likelihood expectation maximization (MLEM) and ordered subset expectation reconstruction techniques and iterative reconstruction techniques. The method can convert attenuation measurements from the scan into values ​​called "CT numbers" or "Houëtsberg units" (HU), which are used to control the brightness of corresponding pixels on a display device.

[0030] To reduce the total scan time, a "spiral" scan can be performed. To perform a "spiral" scan, the patient is moved while acquiring data for a prescribed number of slices. In such a system, the position of the source relative to the patient follows a spiral trajectory. The spiral mapped out by the source produces projection data from which an image in each prescribed slice can be reconstructed.

[0031] As used herein, the phrase "reconstructing an image" is not intended to exclude embodiments of the invention in which data representing an image is generated rather than a visible image. Thus, as used herein, the term "image" broadly refers to both a visible image and data representing a visible image. However, many embodiments generate (or are configured to generate) at least one visible image.

[0032] Figure 2 An example similar to Figure 1 1. An exemplary imaging system 200 of the CT imaging system 100 is shown. The imaging system 200 is configured to image the subject 112. In one embodiment, the imaging system 200 includes a detector array 108 (see Figure 1 ). The detector array 108 also includes a plurality of detector elements 202 that together sense an x-ray beam passing through a subject 112 (such as a patient) to acquire corresponding projection data. Thus, in one embodiment, the detector array 108 is fabricated in a multi-slice configuration including multiple rows of units or detector elements 202. In such a configuration, additional rows or rows of detector elements 202 are arranged in a parallel configuration to acquire projection data.

[0033] In certain embodiments, the imaging system 200 is configured to traverse different angular positions around the subject 112 to acquire desired projection data. Thus, the gantry 102 and components mounted thereon can be configured to rotate about the rotation center 206 for acquiring, for example, projection data at different energy levels. Alternatively, in embodiments where the projection angle relative to the subject 112 changes over time, the mounted components can be configured to move along a generally curved line rather than along a segment of a circle.

[0034] As the x-ray source 104 and the detector array 108 rotate, the detector array 108 collects data of the attenuated x-ray beam. The data collected by the detector array 108 then undergoes pre-processing and calibration to condition the data to represent the line integral of the attenuation coefficient of the scanned subject 112. The processed data is often referred to as a projection.

[0035] In some examples, individual detectors or detector elements 202 in the detector array 108 may include photon counting detectors that register interactions of individual photons into one or more energy bins. It should be appreciated that the methods described herein may also be implemented using energy integrating detectors.

[0036] The acquired projection data set can be used for base material decomposition (BMD). During BMD, the measured projections are converted into a set of material density projections. The material density projections can be reconstructed to form a pair or set of material density maps or images (such as bone maps, soft tissue maps, and / or contrast agent maps) for each corresponding base material. The density maps or images can then be associated to form a volume rendering of the base material (e.g., bone, soft tissue, and / or contrast agent) in the imaging volume.

[0037] Once reconstructed, the base material image produced by the imaging system 200 shows the internal features of the subject 112 represented by the density of the two base materials. The density image can be displayed to show these features. In traditional methods of diagnosing medical conditions (such as disease states), and more generally diagnosing medical events, a radiologist or physician can consider a hard copy or display of a density image to discern characteristic features of interest. Such features can include lesions, size and shape of specific anatomical structures or organs, as well as other features that should be discernible in the image based on the skills and knowledge of the individual practitioner.

[0038] In one embodiment, the imaging system 200 includes a control mechanism 208 to control movement of components, such as rotation of the gantry 102 and operation of the x-ray source 104. In certain embodiments, the control mechanism 208 also includes an x-ray controller 210 configured to provide power and timing signals to the x-ray source 104. Additionally, the control mechanism 208 includes a gantry motor controller 212 configured to control the rotational speed and / or position of the gantry 102 based on imaging requirements.

[0039] In certain embodiments, the control mechanism 208 also includes a data acquisition system (DAS) 214 that is configured to sample analog data received from the detector elements 202 and convert the analog data into digital signals for subsequent processing. The DAS 214 can be further configured to selectively aggregate analog data from a subset of the detector elements 202. The data sampled and digitized by the DAS 214 is sent to a computer or computing device 216. In one example, the computing device 216 stores the data in a mass storage device or storage device 218. For example, the storage device 218 may include a hard drive, a floppy disk drive, a compact disk read / write (CD-R / W) drive, a digital versatile disk (DVD) drive, a flash drive, and / or a solid-state storage drive.

[0040] In addition, the computing device 216 provides commands and parameters to one or more of the DAS 214, the x-ray controller 210, and the gantry motor controller 212 to control system operations, such as data acquisition and / or processing. In certain embodiments, the computing device 216 controls system operations based on operator input. The computing device 216 receives operator input, which includes, for example, commands and / or scan parameters, via an operator console 220 operatively coupled to the computing device 216. The operator console 220 may include a user interface (not shown) that may include one or more of a keyboard, a touch screen, a mouse, a touch pad, etc. to allow an operator to specify commands and / or scan parameters.

[0041] Although Figure 2 One operator console 220 is illustrated, but more than one operator console may be coupled to the imaging system 200, e.g., for inputting or outputting system parameters, requesting exams, plotting data, and / or viewing images. In addition, in certain embodiments, the imaging system 200 may be coupled to multiple displays, printers, workstations, and / or the like located locally or remotely, e.g., within an institution or hospital or at an entirely different location, via one or more configurable wired and / or wireless networks (such as the Internet and / or a virtual private network, a wireless telephone network, a wireless local area network, a wired local area network, a wireless wide area network, a wired wide area network, etc.).

[0042] In one embodiment, the imaging system 200 includes or is coupled to a picture archiving and communication system (PACS) 224. In one exemplary implementation, the PACS 224 is further coupled to a remote system (such as a radiology department information system, a hospital information system) and / or to an internal or external network (not shown) to allow operators at different locations to supply commands and parameters and / or gain access to image data.

[0043] The computing device 216 uses operator-supplied and / or system-defined commands and parameters to operate the table motor controller 226, which in turn may control the table 114 (see Figure 1 ) or 228, the inspection table can be an electric inspection table. Specifically, the inspection table motor controller 226 can move the inspection table 114 (see Figure 1 ) or 228 to appropriately position the subject 112 in the gantry 102 to acquire projection data corresponding to a target volume of the subject 112.

[0044] As previously described, the DAS 214 samples and digitizes the projection data acquired by the detector elements 202. Subsequently, the image reconstructor 230 uses the sampled and digitized x-ray data to perform high-speed reconstruction. Figure 2 The image reconstructor 230 is illustrated as a separate entity, but in certain embodiments, the image reconstructor 230 may form part of the computing device 216. Alternatively, the image reconstructor 230 may not be present in the imaging system 200, but the computing device 216 may perform one or more functions of the image reconstructor 230. In addition, the image reconstructor 230 may be located locally or remotely, and may be operably connected to the imaging system 200 using a wired or wireless network. In particular, an exemplary embodiment may use computing resources in a "cloud" network cluster for the image reconstructor 230.

[0045] In one embodiment, the image reconstructor 230 stores the reconstructed image in the storage device 218. Alternatively, the image reconstructor 230 may send the reconstructed image to the computing device 216 to generate usable patient information for diagnosis and evaluation. In certain embodiments, the computing device 216 may transmit the reconstructed image and / or the patient information to a display or display device 232 that is communicatively coupled to the computing device 216 and / or the image reconstructor 230. In some embodiments, the reconstructed image may be transmitted from the computing device 216 or the image reconstructor 230 to the storage device 218 for short-term or long-term storage.

[0046] refer to Figure 3, wherein an image processing system 302 is shown, which is configured to receive projection data. In some embodiments, the image processing system 302 is incorporated into the CT imaging system 100. For example, the image processing system 302 may be provided in the CT imaging system 100 as an image processor unit 110 or as a computing device 216. In some embodiments, at least a portion of the image processing system 302 is disposed at a device (e.g., an edge device, a server, etc.) that is communicatively coupled to the CT imaging system 100 via a wired connection and / or a wireless connection. In some embodiments, at least a portion of the image processing system 302 is disposed at a separate device (e.g., a workstation) that can receive projection / image data from the CT imaging system or from a storage device that stores projection / image data generated by the CT imaging system. The image processing system 302 may be operably / communicatively coupled to a user input device 332 and a display device 334. The user input device 332 may be integrated into the CT imaging system, such as at a user input device of the CT imaging system 100. Similarly, the display device 334 may be integrated into a CT imaging system, such as at the display device of the CT imaging system 100 .

[0047] The image processing system 302 includes a processor 304 configured to execute machine-readable instructions stored in a non-transitory memory 306. The processor 304 may be a single-core or multi-core processor, and the program executed thereon may be configured for parallel processing or distributed processing. In some embodiments, the processor 304 may optionally include separate components distributed on two or more devices, which may be located at a distance and / or configured for coordinated processing. In some embodiments, one or more aspects of the processor 304 may be virtualized and performed by a remotely accessible networked computing device configured in a cloud computing configuration.

[0048] The non-transitory memory 306 may store a contrast period classifier module 308, an energy transformation module 310, a mixing module 312, a training module 314, and a projection / image database 316. The contrast period classifier module 308 may include a period classifier configured to identify various contrast periods included in the image of the subject. In some examples, the period classifier stored in the contrast period classifier module 308 may include one or more machine learning (ML) models configured to identify various contrast periods included in the image of the subject, and may include trained and / or untrained ML models, and may also include various data or metadata about the one or more ML models stored therein. As an example, the period classifier may be a deep learning model, such as a neural network. The period classifier model may be trained using training data including multiple sets of 3-plane annotated images. Each set of 3-plane annotated images may include three different scan planes of the subject obtained at a known contrast period in multiple contrast periods, and the contrast period may be indicated by an annotation in the image. Different sets of multiple sets of 3-plane annotated images may be obtained at different contrast periods, so that images of each contrast period are included in the training data.

[0049] The energy transformation module 310 may include multiple energy transformation models, which may be ML models (e.g., deep learning models) that may be configured to transform an image at a first energy level to a second energy level. Each energy transformation model is trained for a specific contrast period and energy level transformation. For example, a first energy transformation model may transform an image including a first contrast period and at a first predetermined kVp / keV into an image at a second predetermined keV. A second energy transformation model may transform an image including a second contrast period and at a first predetermined kVp / keV into an image at a second predetermined keV. A third energy transformation model may transform an image including a first contrast period and at a second predetermined keV into an image at a third predetermined keV. The energy transformation module 310 may include trained and / or untrained ML models, and may also include various data or metadata about the one or more ML models stored therein.

[0050] In addition, the non-transitory memory 306 may store a blending module 312 that stores instructions for blending transformed images output from two or more of the energy transformation models based on contrast period data output from the contrast period classifier (e.g., when the output from the contrast period classifier indicates that more than contrast periods are present in the initial image). Specifically, the contrast classifier may output a ratio of the identified contrast periods present in the image. The ratio of the contrast periods may be used as a weighting factor for blending the transformed images output from each of the two or more energy transformation models.

[0051] The non-transitory memory 306 may also store a training module 314, which may include instructions for training one or more of the ML models stored in the contrast period classifier module 308 and / or the energy transformation module 310. The training module 314 may include instructions that, when executed by the processor 304, cause the image processing system 302 to perform one or more of the steps of the following methods: a training method for training a contrast period classifier to identify contrast periods present in an image, and a training method for training multiple energy transformation models to generate a transformed image at a second energy level from an initial image at a first energy level. In some examples, each energy transformation model may also be trained using an inverse transformed image generated by a corresponding inverse energy transformation model, as explained in more detail below.

[0052] In some embodiments, the training module 314 may include instructions for implementing one or more gradient descent algorithms, applying one or more loss functions, and / or training routines for adjusting parameters of one or more ML models of the contrast period classifier module 308 and / or the energy transformation module 310. The training module 314 may include a training data set for the one or more ML models of the contrast period classifier module 308 and / or the energy transformation module 310.

[0053] The non-transitory memory 306 also stores a projection / image database 316. The projection / image database 316 may include, for example, projection data acquired via a CT imaging system and images reconstructed from the projection data. For example, the projection / image database 316 may store projection data acquired via the CT imaging system 100 and / or received from other communicatively coupled CT imaging systems or image databases. In some examples, the projection / image database 316 may store images generated by the energy transformation module 310 or the mixing module 312. The projection / image database 316 may also include one or more training data sets for training the one or more ML models of the contrast period classifier module 308 and / or the energy transformation module 310.

[0054] In some embodiments, non-transitory memory 306 may include components disposed on two or more devices that may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of non-transitory memory 306 may include a remotely accessible networked storage device configured in a cloud computing configuration.

[0055] The user input device 332 may include one or more of a touch screen, keyboard, mouse, trackpad, motion sensing camera, or other device configured to enable a user to interact with and manipulate data within the image processing system 302. In one example, the user input device 332 may enable a user to select projection data for use in training a machine learning model, or for further processing using a trained machine learning model (e.g., the period classifier model and energy transformation model disclosed herein).

[0056] The display device 334 may include one or more display devices utilizing virtually any type of technology. In some embodiments, the display device 334 may include a computer monitor and may display CT images, including images generated by the energy conversion module 310 and the mixing module 312. The display device 334 may be combined with the processor 304, the non-transitory memory 306, and / or the user input device 332 in a common housing, or may be a peripheral display device and may include a monitor, a touch screen, a projector, or other display devices known in the art that may enable a user to view the CT images generated by the CT imaging system and / or interact with various data stored in the non-transitory memory 306.

[0057] It should be understood that Figure 3 The image processing system 302 shown is for illustration and not for limitation. Another suitable image processing system may include more, fewer, or different components.

[0058] Steering Figure 4 , which illustrates a process 400 for generating one or more final transformed images 410 from one or more corresponding images 402, which may be performed by Figure 3 400 may be performed by the image processing system 302. Process 400 may include obtaining images 402. Images 402 may be generated from projection data collected during a single-energy CT scanning protocol, and thus each image in images 402 may be a single-energy image reconstructed from projection data obtained when the x-ray source of the CT imaging system is operated at a first energy level, which first energy level may be a single peak energy level, such as 120 kVp. The projection data may be obtained when a single-energy CT scanning protocol is performed on a region of interest (ROI) of a subject. Although images 402 are described herein as being obtained using a single-energy CT imaging system, it should be understood that one or more of the images may also be obtained using other types of CT imaging systems (e.g., dual energy), despite operating in a single-energy mode (such that the projection data used to reconstruct the image is only obtained at a single peak energy level).

[0059] Process 400 may include entering an image from image 402 (eg, an initial image) into a contrast period classifier model 404 (which may be the same as described above with respect to Figure 3 The process 400 may include selecting an energy transformation model from a plurality of energy transformation models 406, wherein each energy transformation model corresponds to a corresponding contrast period and an energy transformation pair. An energy transformation pair refers to an initial energy level (e.g., a first energy level) of an image to be transformed and a final energy level (e.g., a second energy level) of the transformed image. The plurality of energy transformation models 406 may be trained based on image pairs obtained using a dual energy CT scanning protocol, wherein one image in the image pair is at a first energy level and the other image is at a second energy level.

[0060] One or more energy transformation models are selected from the plurality of energy transformation models 406 based on the identified contrast period in the initial image. For example, if the initial image is identified as including only one contrast period, one energy transformation model corresponding to the identified contrast period may be selected. If the initial image is identified as including two contrast periods (e.g., transitioning from a first contrast period to a second contrast period), two energy transformation models may be selected, each energy transformation model corresponding to a respective contrast period and trained for a desired energy transformation (e.g., the energy transformation pair described above). Thus, each energy transformation model in the plurality of energy transformation models 406 is trained to transform an image from a first predetermined energy level (e.g., 120 kVp) to a second predetermined energy level (e.g., 50 keV). As an example, a first model P1 may be configured to transform an image without contrast, a second model P2 may be configured to transform an image in the venous phase, until an nth model PN, which may be configured to transform an image in the delayed phase (additional models may be included in the plurality of energy transformation models 406 to, for example, transform images in the portal venous phase and the arterial phase, which are not described herein for brevity). Figure 4 ).

[0061] The initial image may be entered as an input to each selected energy transformation model, and each selected energy transformation model may output a corresponding transformed image. When the initial image includes only one contrast period, the transformed image output by the selected energy transformation model may be the final transformed image displayed to a user, for example, on a display device. For images that include more than one contrast period, the transformed images output by two or more selected contrast periods may be input to a mixer 408 to generate a final transformed image. The final transformed image may be generated from the two or more transformed images by applying a weighting factor to each of the transformed images output from the energy transformation model and then summing the weighted images. The weighting factor may be based on a ratio of a contrast period relative to other identified contrast periods included in the initial image.

[0062] Thus, the above process may be repeated for each image in image 402 to produce a final transformed image 410. The final transformed images are each at a second energy level. In some embodiments, the first energy level is higher than the second energy level. Thus, the final transformed image 410 may have increased contrast visibility compared to image 402. In this way, visualization of the subject's ROI may be increased. In some embodiments, the first energy level is lower than the second energy level. For example, it may be beneficial to transform the images in image 402 to a higher energy level during non-contrast or low-contrast image generation without performing an additional non-contrast scan (e.g., to generate a non-contrast or low-contrast image from a contrast scan), or to facilitate downstream tasks that utilize low-contrast and low-noise images.

[0063] Figure 5 The image 502 may be a process 500 of sequentially transforming the image 502 from a first energy level to a third energy level. Figure 4 The image included in the described image 402 is, therefore, obtained at a first energy level. As described herein, each energy transformation model can be trained to perform an energy transformation, i.e., an energy transformation of a predetermined magnitude between an initial energy level and a final energy level (e.g., a predetermined energy level change). If the magnitude of the energy transformation is greater than a threshold value (e.g., greater than 20 keV), the transformed image may exhibit poor image quality and ROI visualization, which may render the image non-diagnostic and / or may result in missed or misdiagnosed conditions, particularly in oncology applications. Therefore, when it is desired to transform an image at an initial energy level (e.g., image 502) to a different energy level that is greater than a threshold difference from the initial energy level, a series of energy transformations can be performed on image 502 to indirectly generate a transformed image at a third energy level from image 502 at the first energy level, which can ensure that the image quality is satisfactory.

[0064] Thus, process 500 includes: entering an image 502 at a first energy level into a contrast period classifier model, which can be compared to Figure 4 The first energy transformation model 506 may be selected based on the contrast period of the image 502 as determined by the contrast period classifier model 404. The first energy transformation model 506 may be Figure 4 500. The process 500 may include inputting the image 502 as an input to the first energy transformation model 506 to generate a first transformed image 508 at the second energy level.

[0065] The second energy conversion model 510 may be selected based on the comparison period and the desired final energy level. Figure 4 As explained, multiple energy transformation models may be trained, each model being specific to a different contrast period. In addition, more than one energy transformation model may be trained for each contrast period, so that different energy transformation models may be trained to perform different energy transformations. For example, a second energy transformation model 510 may be trained to transform images of a given contrast period from a second energy level to a third energy level (and additional energy transformation models may be trained to transform images of the remaining contrast periods from the second energy level to the third energy level). Thus, the second energy transformation model 510 may be trained to transform an image (such as the first transformed image 508) from a second energy level to a third energy level. A second transformed image 512 at a third energy level may be generated by feeding the first transformed image 508 as input to the second energy transformation model 510. The second energy level is different from the third energy level. In an example, the third energy level may be 30 keV. When compared to the first transformed image 508, the second transformed image 512 may exhibit increased contrast visibility, which may increase visualization of the subject's ROI. The contrast visibility of the second transformed image may enable a medical professional to diagnose the subject in a manner that reduces the frequency of missed or misdiagnosed diagnoses.

[0066] It should be understood that in examples where the initial image includes more than one contrast period, a modified version of process 500 may be performed. The modified version may include selecting two or more first energy transformation models, wherein each first energy transformation model is trained to transform an image from a first energy level to a second energy level. Each first energy transformation model is selected based on a contrast period identified in the initial image. The first energy transformation models each output a corresponding first transformed image, and the plurality of first transformed images are blended into a final first transformed image, which is entered as an input into two or more second energy transformation models. The two or more second energy transformation models are each trained to transform an image from a second energy level to a third energy level. Each second energy transformation model is selected based on a contrast period identified in the initial image. The second energy transformation models each output a corresponding second transformed image, and the plurality of second transformed images are blended into a final transformed image.

[0067] Go to Figure 6 , which illustrates a process 600 for training a comparison period classifier model 616 (eg, comparison period classifier model 404). Fig.10 One or more operations described in more detail may train the contrast period classifier model 616 to recognize the image quality of the imaged system using a CT imaging system such as Figure 1 The process 600 may be performed by one or more computing systems such as Figure 3 The image processing system 302 of the present invention is implemented to train the contrast period classifier model 616 to identify the contrast period included in the images acquired using the CT imaging system based on multiple sets of 3-plane annotated maximum intensity projection (MIP) images. Once trained, the contrast period classifier model 616 can be trained according to the following reference Figure 8 and 9 One or more operations described in more detail may be used to identify the contrast period classifier model 616 for use with a CT imaging system (e.g., Figure 1 The contrast period included in the image acquired by the CT imaging system 100).

[0068] Process 600 includes obtaining MIP images 602 of one or more subjects. For example, projection data of a 3D volume of an ROI (e.g., brain, heart, liver) of each of the one or more subjects may be obtained during a contrast scan in which a contrast agent is administered to each subject, and the projection data may be obtained before, during, and / or after contrast agent uptake and discharge, and a set of MIP images may be generated from each volume (e.g., in three different scan planes, such as axial, coronal, and sagittal planes). The MIP images 602 may be annotated with one or more contrast phases. A corresponding annotation of each annotated MIP image may indicate a contrast phase included in each annotated MIP image. Depending on the timing of acquiring the corresponding MIP image relative to the injection of the contrast agent, the corresponding MIP image may include more than one contrast phase. Thus, the corresponding MIP image may include one or more annotations indicating no contrast, a venous phase, a portal venous phase, an arterial phase, and a delayed phase.

[0069] Process 600 includes: using a data set generator 604 to generate a plurality of training triplet data. The plurality of training triplet data may be stored in a training module 606. The training module 606 may be used with Figure 3 The training module 314 of the image processing system 300 may be the same or similar. The plurality of training triplet data may be divided into training triplet 608 and test triplet 610. Each of the training triplet 608 and the test triplet 610 may include a set of 3-plane annotated images from the MIP image 602, including a first annotated MIP image in a first scanning plane, a second annotated MIP image in a second scanning plane, and a third annotated MIP image in a third scanning plane, each image being generated from the same volume of projection data.

[0070] After each triple is generated, each triple may be assigned to a training triple 608 or a test triple 610. In one embodiment, triples may be randomly assigned to a training triple 608 or a test triple 610 in a predetermined ratio (e.g., 90% to training triples / 10% to test triples, or 85% to training triples / 15% to test triples). It should be understood that the examples provided herein are for illustrative purposes, and triples may be assigned to a training triple 608 dataset or a test triple 610 dataset via different processes and / or in different ratios without departing from the scope of the present disclosure.

[0071] A plurality of training triplets 608 and test triplets 610 may be selected to ensure that sufficient training data is available to prevent overfitting, whereby the initial comparison period classifier model 612 will learn to map features unique to samples of the training set that are not present in the test set. Process 600 includes training the initial comparison period classifier model 612 based on the training triplets 608. Process 600 may include a validator 614 that validates the performance of the initial comparison period classifier model 612 (when the initial model is trained) against the test triplets 610. The validator 614 may take as input a trained or partially trained model (e.g., the initial comparison period classifier model 612, but after the model has been trained and updated) and a data set of test triplets 610, and may output an evaluation of the performance of the trained or partially trained comparison period classifier model based on the data set of test triplets 610.

[0072] Thus, an initial contrast period classifier model is trained based on training triplets, wherein the training triplets include a first annotated MIP image (in a first scan plane), a second annotated MIP image (in a second scan plane), and a third annotated MIP image (in a third scan plane), each of which is of the same subject and the same contrast period. Additional training triplets may be used to train the initial contrast period classifier model, each of which includes three images of the same subject and the same contrast period in three different scan planes, but it should be understood that different triplets may be of different subjects and / or contrast periods. The corresponding annotation of each annotated MIP image indicates the contrast period included in each annotated image. In this way, each annotated MIP training image may be annotated with all contrast periods included in the corresponding annotated MIP image. The corresponding annotation may be considered as the ground truth contrast period. The ground truth contrast period may be compared with the identified contrast period output from the initial contrast period classifier model to calculate a loss function for adjusting the model parameters of the initial contrast period classifier model. In some examples, a contrast period classifier model may be trained to output, for each possible contrast period (e.g., no contrast, venous phase, portal venous phase, arterial phase, and delayed phase), a relative probability that an input set of 3-plane MIP images includes a contrast period, and the ratios of contrast periods described herein may be relative probabilities.

[0073] Once the validator 614 determines that the comparison period classifier model has been sufficiently trained, the comparison period classifier model can be stored in Figure 36. The contrast period classifier model 616, when deployed, can identify contrast periods in images acquired using a CT imaging system. New acquired images can be entered as inputs into the contrast period classifier model 616 to output contrast period ratios 618 for the identified contrast periods. The contrast period ratios 618 can be used to select an energy transformation model and also used as a weighting factor for blending multiple transformed images according to embodiments described herein.

[0074] Figure 7 The process 700 for training an energy transformation model 716 (eg, in the plurality of energy transformation models 406) is illustrated in FIG. Fig.11 In more detail, one or more operations of the energy conversion model 716 may be trained to convert (using a CT imaging system such as Figure 1 The process 700 may be performed by one or more computing systems (such as Figure 3 Once trained, the image processing system 302 can be used according to the following reference Figure 8 One or more operations described in more detail, the energy transformation model 716 is used to transform the image at the first energy level using the CT imaging system (e.g., Figure 1 Images acquired by a CT imaging system 100).

[0075] Process 700 includes obtaining images 702 of one or more subjects. Images 702 may be obtained according to a dual energy CT scanning protocol (e.g., using a dual energy CT imaging system), wherein for each acquisition, projection data is acquired at two different energy levels. According to each acquisition of projection data, two images (e.g., monochrome images) are generated, each image being at a different energy level (e.g., a first energy level and a second energy level). Each image includes a region of interest (ROI) of the subject in a single contrast period. In one example, the ROI may be a portion and feature of the brain, heart, or other anatomical structure of the subject. Images 702 may include a single contrast period, which is one of no contrast, venous phase, portal venous phase, arterial phase, or delayed phase. In some examples, images 702 may include at least some images having multiple (e.g., two) contrast periods.

[0076] Process 700 includes: using a data set generator 704 to generate a plurality of training data pairs. The plurality of training data pairs can be stored in a training module 706. The training module 706 can be used with Figure 3The plurality of training data pairs may be divided into training pairs 708 and test pairs 710. Each of the training pairs 708 and the test pairs 710 may include a first image at a first energy level and a second image at a second energy level, wherein the first energy level is different from the second energy level. As a non-limiting example, the first energy level may be 70 keV and the second energy level may be 50 keV.

[0077] Similar to the process 600 described above, once each pair is generated, each pair may be assigned to either a training pair 708 or a test pair 710. It should be understood that the examples provided herein are for illustrative purposes, and that the pairs may be assigned to the training pair 708 dataset or the test pair 710 dataset via different processes and / or in different proportions without departing from the scope of the present disclosure.

[0078] A plurality of training pairs 708 and test pairs 710 may be selected to ensure that sufficient training data is available to prevent overfitting, whereby the initial model 712 will learn to map features unique to samples of the training set that are not present in the test set. The process 700 includes training the initial model 712 based on the training pairs 708. The process 700 may include a validator 714 that validates the performance of the initial model 712 (when the initial model is trained) against the test pairs 710. The validator 714 may take as input a trained or partially trained model (e.g., the initial model 712, but after the model has been trained and updated) and a data set of the test pairs 710, and may output an evaluation of the performance of the trained or partially trained energy conversion model based on the data set of the test pairs 710.

[0079] The initial model 712 may be an initial energy transformation model trained based on a training pair, wherein the training pair includes a first image at a first energy level and a ground truth image (e.g., a second image at a second energy level). The first energy level may be an initial energy level, and the second energy level may be a desired energy level. The ground truth image may be compared with the transformed image output from the initial model to calculate a first loss function for adjusting the model parameters of the initial model. In addition, in some examples, an initial inverse model 713 may be used to re-transform the transformed image output from the initial model back to the first energy level. In other words, the initial inverse model 713 may output an inverse transformed image using the transformed image as input. The first image (e.g., at the first energy level) may be compared with the inverse transformed image output from the initial inverse model 713 to calculate a second loss function for adjusting the model parameters of both the initial model 712 and the initial inverse model 713.

[0080] Once the validator 714 determines that the energy conversion model has been sufficiently trained, the energy conversion model 716 can be stored in Figure 3 The energy transformation model 716, when deployed, may transform an image at one energy level (e.g., a first energy level) into a transformed image at another energy level (e.g., a second energy level). A newly acquired image (e.g., from image 702) obtained at a single peak energy level (e.g., the first energy level) and determined to be at the same contrast period as the image used to train the energy transformation model 716 may be entered as input into the energy transformation model 716 to generate a transformed image 718. The transformed image 718 may be displayed via a display device or saved to a memory, as described above with respect to Figure 3 As described, or may be subjected to additional image processing, such as blending according to the embodiments described herein.

[0081] It should be understood that process 700 may be repeated to train multiple additional energy transformation models. For example, a first subset of energy transformation models may be trained to transform an image from a first energy level to a second energy level, wherein each energy transformation model in the first subset of energy transformation models is trained using images that include corresponding different contrast periods, such that each energy transformation model in the first subset is specific to a contrast period (e.g., the first energy transformation model in the first subset may be specific to the arterial phase, the second energy transformation model in the first subset may be specific to the venous phase, and so on). A second subset of energy transformation models may be trained to transform an image from a second energy level to a third energy level (e.g., 30 keV), wherein each energy transformation model in the second subset of energy transformation models is trained using images that include corresponding different contrast periods, such that each energy transformation model in the second subset is specific to a contrast period (e.g., the first energy transformation model in the second subset may be specific to the arterial phase, the second energy transformation model in the second subset may be specific to the venous phase, and so on).

[0082] Figure 8 8 is a flowchart illustrating a method 800 for generating a final transformed image at a second energy level from an image at a first energy level according to an embodiment of the present disclosure. The method 800 may be based on a non-transitory memory and a computing device such as Figure 2 The computing device 216 and / or Figure 3 The image processing system 302 may be executed by executing instructions of one or more processors of the image processing system 302.

[0083] At 802, method 800 includes obtaining an image at a first peak energy level acquired using a single energy imaging system and / or according to a single energy scanning protocol. The image may be generated from projection data of a region of interest (ROI) of a subject, wherein the projection data is acquired at a single peak x-ray tube energy level. The ROI may include an anatomical portion or feature, such as a brain of a subject, a chest of a subject, etc. In some embodiments, the projection data may be acquired using a single energy imaging system and / or according to a single energy scanning protocol. Figure 1 and Figure 2 The image at the first peak energy level may be a suitable three-dimensional (3D) rendering of projection data of a certain volume of the subject, such as a maximum intensity projection (MIP) image, a minimum intensity projection image, an image generated using a custom projection, etc. In other embodiments, the image at the first peak energy level may be a two-dimensional (2D) slice image generated from the projection data.

[0084] At 804, method 800 includes identifying a contrast period of the image by inputting the image into a contrast period classifier. The contrast period classifier may be relative to Figures 4 to 6 The contrast period classifier model described, or another suitable classifier configured to identify the contrast period of an image. Identifying the contrast period of an image using a contrast period classifier may include identifying a single contrast period or more than one contrast period in the image. In some examples, the contrast period classifier may output a ratio of contrast periods. In some examples, as explained above, the ratio may include a corresponding probability that the image includes each corresponding contrast period. In other examples, the ratio of contrast periods may refer to: for each contrast period, the percentage of pixels within the image that are in the contrast period. In another example, the contrast period classifier may utilize a contrast enhancement curve generated during a scan of the subject, and may identify the contrast period based on the timing of acquiring projection data used to generate the image relative to the contrast enhancement curve. Potential contrast periods include no contrast, venous phase, portal venous phase, arterial phase, and delayed phase.

[0085] In some examples, the contrast period classifier may identify that the image includes a single contrast period, which may be a first contrast period. For example, the contrast period classifier may output a value for the first contrast period that is above a first threshold value, such as above 0.8, above 0.9, or equal to or approximately equal to 1. When the value of the first contrast period is above the first threshold value, it may be determined that almost all or all of the relevant pixels in the image are in the first contrast period. The relevant pixels in the image may refer to tissue pixels that may ingest contrast agent. In an example, the first contrast period may be a venous phase. Therefore, based on the value of the ratio of the first contrast period, the image may include only the venous phase.

[0086] In other examples, the contrast period classifier may identify that more than one contrast period is included in the image. For example, the contrast period classifier may identify that the image includes a first contrast period and a second contrast period, and the first contrast period and the second contrast period are different. Based on the fact that neither value output by the contrast period classifier is higher than a first threshold and the values ​​corresponding to the first contrast period and the second contrast period are higher than a second threshold (e.g., higher than 0.1), the image may be identified as including both the first contrast period and the second contrast period. For example, the first value for the first contrast period may be equal to 0.65, and the second value for the second contrast period may be equal to 0.35, indicating that approximately 65% ​​of the relevant pixels in the image are in the first period and 35% of the relevant pixels are in the second period. For example, the first contrast period may be the venous period, and the second contrast period may be the portal venous period. Thus, the image may include tissue pixels in the venous period and tissue pixels in the portal venous period.

[0087] In some examples, as previously explained, the contrast period classifier may output a value for each potential contrast period that may be included in the image. For example, the contrast period classifier may output a first value for a first contrast period, a second value for a second contrast period, a third value for a third contrast period, a fourth value for a fourth contrast period, and a fifth value for a fifth contrast period. These values ​​may range from a value of 0 to a value of 1. A value of 0 may indicate that the corresponding contrast period is not included in the image. A value of 1 may indicate that the image only includes the corresponding contrast period, and that other contrast periods are not included in the image. Values ​​between 0 and 1 (and specifically below the first threshold and above the second threshold as described above) indicate that more than one contrast period may be included in the image.

[0088] At 806, method 800 includes determining whether a single contrast period is identified by a contrast period classifier. The determination of whether the image includes a single contrast period or more than one contrast period may be performed as explained above, for example, based on a value output by a contrast period classifier. When the value for a contrast period is above a first threshold, the image may be identified as including only that contrast period, or when no value output by the contrast period classifier is above the first threshold, the image may be identified as including more than one contrast period. In one example, the contrast period classifier may output a value of 0 for no contrast, a value of 0.5 for a delayed phase, a value of 0.4 for a venous phase, a value of 0.05 for a portal venous phase, and a value of 0.05 for an arterial phase. Since the contrast period classifier outputs more than one value above the second threshold, but no value above the first threshold, the image is identified as including more than one contrast period.

[0089] In response to determining that only one contrast period is identified in the image, method 800 includes, at 810, selecting an energy transformation model based on the contrast period. As described herein, to transform the image, the contrast value for each contrast period is mapped from a first (e.g., higher) energy level to a second (e.g., lower) energy level. However, mapping a particular contrast period from the first energy level to the second energy level may differ depending on the particular contrast period. Thus, based on Fig.11 The method described herein trains an energy transformation model for a specific contrast period. In this manner, an energy transformation model may be selected based on the identified contrast period output from the contrast period classifier. In addition, the energy transformation model may be selected based on the desired energy level transformation (e.g., the energy level change from the initial energy level to the final energy level).

[0090] For a given energy level transformation, the selected energy transformation model can be selected from: a first energy transformation model for a first comparison period, a second energy transformation model for a second comparison period, a third energy transformation model for a third comparison period, a fourth energy transformation model for a fourth comparison period, and a fifth energy transformation model for a fifth comparison period, wherein the first comparison period, the second comparison period, the third comparison period, the fourth comparison period, and the fifth comparison period correspond to different comparison periods.

[0091] Each of the first contrast period, the second contrast period, the third contrast period, the fourth contrast period, and the fifth contrast period can be one of no contrast, venous phase, portal venous phase, arterial phase, and delayed phase. The energy transformation model is selected for training a single contrast period included in the image. In one example, the contrast period classifier can identify that the image at the first energy level includes the venous phase. Therefore, the energy transformation model trained for the venous phase is selected from the following: the first energy transformation model, the second energy transformation model, the third energy transformation model, the fourth energy transformation model, and the fifth energy transformation model.

[0092] At 814, method 800 includes generating a final transformed image at a second energy level by inputting the image into a selected energy transformation model that is trained to output a final transformed image at the second energy level based on the image. In some embodiments, the second energy level may be lower than the first energy level. Thus, the final transformed image may exhibit increased contrast visibility compared to the image at the first peak energy level. By increasing contrast visibility, the ROI of the subject may have increased visualization, which may reduce the frequency of missed and / or misdiagnosed subjects.

[0093] At 818, method 800 includes displaying and / or saving the final transformed image. The final transformed image may be displayed using a display device, such as a display device communicatively coupled to an image processing system, which may be Figure 3 The image processing system 302 of FIG. 302 can be used to process the image. In this way, the medical professional can visually evaluate the content of the final transformed image and make a diagnosis based on the content of the final transformed image. By transforming the image at the first peak energy level to the second peak energy level, the medical professional can more easily correctly diagnose the subject because poor contrast visibility will not degrade the image quality and render the image unusable for diagnosis. In addition, the final transformed image can be stored in a memory (e.g., Figure 3 The transformed image may be stored in a non-transitory memory 306 or in an image archive (such as a PACS) to enable a user or medical professional to access the final transformed image at a later time. The method 800 then ends.

[0094] Returning to 806, in response to determining that more than one contrast period is identified in the image, method 800 includes, at 808, selecting an energy transformation model for each identified contrast period. Different tissues may take up contrast at different rates, which may result in images with multiple contrast periods. For example, some types of tissue (e.g., brain) may be in a different contrast period than other types of tissue (e.g., aortic arch). Due to the presence of multiple contrast periods and the challenge of mapping tissue in different contrast periods from a first energy level to a second energy level, each energy transformation model is trained for a specific contrast period and energy transformation (e.g., the energy level change between an initial energy level and a final energy level).

[0095] An energy transformation model may be selected for each contrast period included in the image. Although each selected energy transformation model corresponds to a different contrast period, Fig.11 The method described trains each selected energy transformation model for the same energy transformation (eg, the same first energy level and second energy level). In this way, each contrast period can be appropriately mapped from the first energy level to the second energy level without degrading image quality.

[0096] As an example, a contrast period classifier may identify a first contrast period and a second contrast period in an image. A first energy transformation model may be selected for the first contrast period, and a second energy transformation model may be selected for the second contrast period. In this manner, pixels corresponding to tissue in the first contrast period and pixels corresponding to tissue in the second contrast period may be mapped separately from a first energy level to a second energy level.

[0097] At 812, method 800 includes generating a transformed image at a second energy level by inputting the image into each selected energy transformation model. The image at the first peak energy level may be entered as input into each selected energy transformation model in the selected energy transformation models to generate a plurality of transformed images at the second energy level. Each selected energy transformation model may output a transformed image at the second energy level. For example, the first energy transformation model described above may generate a first transformed image at the second energy level, and the second energy transformation model may generate a second transformed image at the second energy level. By entering the image into both the first energy transformation model and the second energy transformation model, pixels corresponding to tissue in the first contrast period and pixels corresponding to tissue in the second contrast period may be mapped separately.

[0098] At 816, method 800 includes generating a final transformed image at the second energy level by blending the transformed images. A weighting factor may be applied to each transformed image at the second energy level to blend the transformed images to generate the final transformed image. The weighting factor for each transformed image may be a value for the corresponding contrast period output from the contrast period classifier model.

[0099] As an example, pixels in a first transformed image may be weighted by applying a first value for a first contrast period to pixel values ​​of the first transformed image, and pixels in a second transformed image may be weighted by applying a second value for a second contrast period to pixel values ​​of the second transformed image, and the weighted first transformed image and the weighted second transformed image may be summed in a pixel-by-pixel manner to generate a final transformed image. Although the example provided includes two contrast periods (e.g., a first contrast period and a second contrast period), method 800 may be applied to images in which more than two contrast periods are identified without departing from the scope of the present disclosure.

[0100] At 818, method 800 includes displaying and / or saving the final transformed image. The final transformed image may be displayed on and saved to the system as previously described to enable a medical professional to make a reliable diagnosis. Method 800 then ends.

[0101] In some examples, one or more aspects of method 800 may be performed in response to receiving user input at a user input device and / or as part of a scanning protocol. In some examples, as part of a scanning protocol, an image may be automatically transformed into a final transformed image. For example, a scanning protocol may provide for generating a specific image at a specific energy level from projection data obtained during a scan of a subject. In other examples, an image may be transformed into a final transformed image in response to a user request, wherein the user requests the transformation of the image by interacting with a user input device. For example, a user may specify which images (e.g., images in which scanning planes and at what energy level) are to be generated from projection data obtained during a scan of a subject. In some examples, a user may initially view an image and request that the image be transformed into a final transformed image while viewing the image in order to increase the visibility of a contrast agent. Thus, in some examples, an image may be displayed on a display device before the transformation and generation of a final transformed image. However, in other examples, an image may not be displayed on a display device before using the image to generate a final transformed image. In such examples, an image may be generated only for the purpose of generating a final transformed image. Furthermore, in some examples, the image at the first peak energy level may be displayed side-by-side with the final transformed image at the second energy level.

[0102] Fig. 9 1 is a flowchart illustrating a method 900 for generating a final transformed image at a third energy level by sequentially transforming images at a first energy level according to an embodiment of the present disclosure. The method 900 may be based on a non-transitory memory and a computing device such as Figure 2 The computing device 216 and / or Figure 3 The image processing system 302 may be executed by executing instructions of one or more processors of the image processing system 302.

[0103] At 902, method 900 includes obtaining an image of a subject at a first peak energy level acquired using a single energy CT imaging system. The image may be acquired using a CT imaging system such as Figure 1 The image can be stored in an image processing system (e.g., Figure 3 ) in a projection / image database. The image may include a region of interest (ROI) of the subject, such as the brain, spine, etc. The image at the first peak energy level may be a suitable 3D rendering of the projection data of a certain volume of the subject, such as a MIP image. In other embodiments, the image at the first peak energy level may be a 2D slice image obtained from the projection data.

[0104] As described herein, it may be desirable to transform an image to a different (e.g., lower) energy level. However, relatively large energy level transforms (e.g., greater than 20 keV) may result in image quality degradation. To achieve a desired energy level transform, a sequential energy transform may be performed on the image, wherein the image is transformed from a first energy level to a second energy level and from the second energy level to a third energy level. The sequential energy transform may continue until a desired final energy level is achieved.

[0105] At 904, method 900 includes identifying a contrast period of the image by inputting the image into a contrast period classifier. Figure 8 As explained, an image at a first energy level may be entered into a contrast period classifier, which outputs a contrast period included in the image. The contrast period classifier may identify more than one contrast period in the image. However, for simplicity, method 900 is described herein for an example in which the contrast period classifier identifies a single contrast period in the image, which may be one of no contrast, venous phase, portal venous phase, arterial phase, and delayed phase.

[0106] At 906, method 900 includes selecting a first energy transformation model and a second energy transformation model based on the identified contrast period. The first energy transformation model and the second energy transformation model may be trained for the identified contrast period and may be selected to achieve a desired energy level change. Specifically, the first energy transformation model may be trained for a first energy transformation (e.g., 70keV to 50keV), and the second energy transformation model may be trained for a second energy level transformation (e.g., from 50keV to 30keV). In other words, the first energy transformation model may be trained to transform an image from a first energy level to a second energy level, and the second energy transformation model may be trained to transform an image from a second energy level to a third energy level.

[0107] In an example, it may be desirable to transform an image at 120 kVp to 30 keV. However, an energy transformation model that transforms an image directly from 120 kVp to 30 keV may produce a transformed image with reduced image quality and contrast visibility compared to the original image. To prevent degradation of image quality, a first energy transformation model may transform the image from 120 kVp to 50 keV, and a second energy transformation model may transform the image from 50 keV to 30 keV. In this way, images may be sequentially transformed from 120 kVp to 30 keV without affecting image quality.

[0108] At 908 , method 900 includes generating a first transformed image at a second energy level by inputting the image into a first energy transformation model for the contrast period.

[0109] At 910, method 900 includes generating a second transformed image at a third energy level by inputting the first transformed image into a second energy transformation model for a contrast period. The second transformed image at the third second energy level may exhibit a desired image quality, such as increased image quality and contrast visibility compared to the first transformed image at the second energy level.

[0110] At 912, method 900 includes displaying and / or saving the second transformed image. The second transformed image may be displayed using a display device, such as a display device communicatively coupled to an image processing system, which may be Figure 3 The image processing system 302 may be used to process the second transformed image. In this manner, the medical professional may visually assess the content of the second transformed image and make a diagnosis based on the content of the second transformed image. By transforming the image at the first energy level to the third energy level, the medical professional may more easily diagnose the subject because poor contrast visibility will not degrade the image quality and render the image unusable for diagnosis. Additionally, the second transformed image may be stored in a memory (e.g., Figure 3 The transformed image may be stored in a non-transitory memory 306 or in an image archive (such as a PACS) to enable a user or medical professional to access the final transformed image at a later time. The method 900 then ends.

[0111] It will be appreciated that the following methods are exemplary and do not limit the scope of the present disclosure. Method 900 may be varied without departing from the scope of the present disclosure. For example, method 900 may be performed on an image that includes more than one contrast period. In such examples, more than one first energy transformation model may be selected, and each image output by the first energy transformation model may be blended to form a first transformed image. Similarly, more than one second energy transformation model may be selected, and each image output by the second energy transformation model may be blended to form a second transformed image. In addition, method 900 may include additional energy level transformations for achieving a desired energy level.

[0112] Reference now Fig.10 , which shows a flow chart of a method 1000 for training a contrast period classifier model. According to one embodiment, the contrast period classifier model may be Figure 6 600 of the present invention. In some embodiments, the contrast period classifier model can be a deep neural network with multiple hidden layers. The method 1000 can be performed by an image processing system (such as Figure 3 The method 1000 may be executed by a processor of the image processing system 302. The method 1000 may be executed by a processor of the image processing system 302. The method 1000 may be executed by a processor of the image processing system 302. Figure 3) and by a processor of the image processing system (e.g. Figure 3 The instructions are executed by the processor 304 of the image processing system 302.

[0113] The contrast period classifier model can be trained based on training data including a plurality of training triplets. For example, each training triplet can include a set of projection images generated from a 3D volume. The set of projection images can include a first annotated maximum intensity projection (MIP) image in a first scan plane, a second annotated MIP image in a second scan plane, and a third annotated MIP image in a third scan plane, as described below. In some embodiments, the plurality of training triplets can be stored in a projection / image database (such as a database) of an image processing system. Figure 3 The projection / image database 316 of the image processing system 302 is stored in the image processing system 302. It should be understood that in a given triplet, each MIP image is of the same subject and is in the same contrast period.

[0114] At 1002, method 1000 includes receiving a plurality of annotated training images in various contrast periods, each annotated training image annotated with a ground truth contrast period. The plurality of annotated training images may be acquired using a CT imaging system such as Figure 1 The plurality of annotated training images may be stored in an image processing system (e.g., Figure 3 ) in a projection / image database. The multiple annotated training images can be images of one or more regions of interest (ROI) (such as the brain, spine, etc.) of one or more subjects. In some embodiments, the multiple annotated training images can be multiple annotated maximum intensity projection (MIP) images, wherein the multiple annotated MIP images are classified and organized into multiple 3-plane groups of annotated MIP images in different scanning planes. Contrast is enhanced in MIP images compared to other types of 3D renderings of projection data for viewing contrast images. By training a contrast period classifier model based on MIP images, a contrast period classifier can identify contrast periods with higher accuracy.

[0115] More specifically, each set of 3-plane annotated training MIP images may include a first annotated MIP training image in a first scan plane, a second annotated MIP training image in a second scan plane, and a third annotated MIP training image in a third scan plane. In some embodiments, the first scan plane may be a sagittal plane, the second scan plane may be a coronal plane, and the third scan plane may be an axial plane. In this way, a contrast period classifier model may be trained to identify the contrast period of an ROI in an image regardless of the orientation / view plane of the ROI within the image.

[0116] Different sets of 3-plane annotated training MIP images may include different subjects and / or different ROIs. In an example, a first set of 3-plane annotated training MIP images may include a first subject for a first ROI, where the first ROI is the brain in a different scan plane. A second set of 3-plane annotated training MIP images may include a second subject for the first ROI in a different scan plane, the first subject being different from the second subject. In another example, a third set of 3-plane images may include the first subject in a second ROI, where the second ROI is the chest in a different scan plan.

[0117] Each annotated MIP training image in each set of 3-plane annotated training images may have a corresponding annotation, wherein each corresponding annotation includes a contrast period included in the corresponding annotated MIP training image. The corresponding annotation may be considered as the ground truth annotation. Thus, the contrast period classifier model may be trained to identify the contrast periods of different ROIs of different subjects.

[0118] At 1004, method 1000 includes selecting a set of 3-plane annotated training images (e.g., training triplets) of the same subject in the same comparison period from the plurality of annotated training images. Instructions configured, stored, and executed by a processor in a memory may cause the processor to select a set of 3-plane annotated training images from the plurality of annotated training images.

[0119] The selected training triplets may include the first set of 3-plane annotated training MIP images for the first subject described above. In some embodiments, the first set of 3-plane annotated training MIP images may include a first contrast period and a second contrast period. The first contrast period is different from the second contrast period. For example, the first contrast period may be in the venous phase and the second contrast period may be in the portal venous phase. Therefore, each of the annotated training MIP images includes an annotation indicating that the image includes tissue in the venous phase and tissue in the portal venous phase.

[0120] In some embodiments, the selected training triplets may include annotated training MIP images, where the annotations include different combinations of no contrast, venous phase, portal venous phase, arterial phase, and delayed phase, depending on the timing after administration of the contrast agent and the uptake rates of different tissues. In one example, when the ROI is the brain of the subject rather than the chest of the subject, the brain tissue may have a different uptake rate than the lung tissue. Therefore, an image of the brain may include a different number of contrast phases compared to the chest because it includes different types of tissue.

[0121] At 1006, method 1000 includes: inputting the set of 3-plane annotated training images into a contrast period classifier model. Figure 3The one or more processors of the image processing system are configured, stored and executed with instructions such that the set of 3-plane annotated MIP training images is fed as input into a contrast classifier model.

[0122] At 1008, method 1000 includes receiving ratios of contrast periods for the set of 3-plane annotated training images output from a contrast period classifier model. For example, the annotations for the set of 3-plane annotated training images may include a value for each possible contrast period, e.g., ranging from 0 to 1 (or another suitable range), where 0 indicates the lowest likelihood that the image includes the contrast period, and 1 indicates the highest likelihood that the image includes the contrast period. The contrast period classifier model may therefore output a corresponding value for each possible contrast period, indicating the likelihood / probability that the input image includes each contrast period.

[0123] At 1010, method 1000 includes comparing the ratio of the ground truth contrast period to the output contrast period to determine a loss / cost function, and adjusting model parameters of the contrast period classifier model via back propagation based on the loss / cost function. For example, a loss function may be calculated for each possible contrast period based on the value output by the contrast period classifier model for the contrast period and the ground truth value for the contrast period. The loss functions (e.g., one loss function per contrast period) may be summed to form a cost function for updating the parameters of the contrast period classifier model.

[0124] At 1012, method 1000 includes: determining whether there are additional annotated training images remaining in the plurality of annotated training images. In some embodiments, the training module may be used at the beginning of a time period to determine the total number of groups of 3-plane annotated training MIP images in the plurality of annotated training images stored in the projection / image database. The instructions configured, stored, and executed by the processor in the training module may cause the processor to determine the number of groups of 3-plane annotated training MIP images input into the contrast period classifier DL model. In this way, the training module may monitor the number of groups of 3-plane annotated training MIP images used to train the contrast period classifier DL model, which is compared with the total number of groups of 3-plane annotated training MIP images. If there are additional annotated training images remaining (e.g., training is not completed), method 1000 returns to 1004 to select the next group of 3-plane annotated images for training. Otherwise, method 1000 ends.

[0125] Reference now Fig.11 , which shows a flow chart of a method 1100 for training an energy conversion model. According to one embodiment, the energy conversion model may be Figure 7The energy transformation model 716 of the process 700 is a non-limiting example. In some embodiments, the energy transformation model can be a deep neural network with multiple hidden layers. The method 1100 can be performed by an image processing system (such as Figure 3 The method 1100 may be executed by a processor of the image processing system 302. The method 1100 may be executed by a processor of the image processing system 302. The method 1100 may be executed by a processor of the image processing system 302. Figure 3 ) and by a processor of the image processing system (e.g. Figure 3 The energy conversion model may be trained based on training data including one or more sets of pairs. Each pair in the one or more sets of pairs may include one image at a first energy level and another image at a second energy level, as described below. In some embodiments, the one or more sets of pairs may be stored in a projection / image database (such as a Figure 3 in the projection / image database 316 of the image processing system 302).

[0126] At 1102, the method 1100 includes receiving a plurality of training image pairs in various contrast periods, each pair including a first image at a first energy level and a second image at a second energy level generated from dual energy projection data (e.g., the first image and the second image may each be a monochrome image). The dual energy projection data may be acquired at two peak energy levels (such as 40 kVp and 140 kVp) in an interleaved manner (e.g., rapid kVp switching) or via two consecutive scans. For each acquisition / set of dual energy projection data, two training images may be generated, such as by reconstructing a material basis image and then performing a linear combination of the material basis images to obtain a first image at a first energy level and a second image at a second energy level.

[0127] At 1104, method 1100 includes: sorting the plurality of training images into data sets based on the contrast periods included in the corresponding images. For example, all training images acquired during the first contrast period are included in the first data set, all training images acquired during the second contrast period are included in the second data set, and so on. Thus, five separate training data sets may be formed, each of which includes a plurality of training image pairs (e.g., each of which includes a first image at a first energy level and a second image at a second energy level). In some examples, at least some of the data sets may include images having more than one contrast period. For example, the first data set may include some image pairs that include only the first contrast period and other image pairs that include the first contrast period and the second contrast period. Image pairs acquired during a mixed contrast period may be included in more than one data set, for example, image pairs acquired during a transition from the first contrast period to the second contrast period may be included in both the first data set and the second data set. In this way, each energy transformation model may be trained to perform energy transformation for both single contrast period images and mixed contrast period images, which may ensure that all cases (including boundary cases) are covered.

[0128] At 1106, method 1100 includes selecting a training image pair from a first data set corresponding to a first comparison period, and entering a first image of the selected training image pair into an untrained first energy transfer model at 1108. The first image may be at a first energy level, such as 70 keV.

[0129] At 1110, method 1100 includes receiving a transformed training image output from a first energy transformation model. The transformed training image may be a transformed version of the first image that is intended to appear as if obtained at a second energy level.

[0130] At 1112, method 1100 includes: entering the transformed training image into an inverse energy transformation model. The inverse energy transformation model can be configured to re-transform the transformed training image back to the first energy level. At 1114, method 1100 includes: receiving the inverse transformed training image output from the inverse transformation model.

[0131] At 1116, method 1100 includes comparing the second image of the selected pair to the transformed training image, and adjusting model parameters of the first energy transformation model via back propagation based on the comparison. For example, a first loss function may be determined based on the transformed training image and the second image of the selected pair, and the first loss function may be used to update the parameters of the first energy transformation model.

[0132] At 1118, method 1100 includes comparing the first image of the selected pair with the inverse training image, and adjusting the model parameters of the first energy transformation model via back propagation based on the comparison. For example, a second loss function may be determined based on the inverse training image and the first image of the selected pair, and the second loss function may be used in combination with the first loss function to update the parameters of the first energy transformation model. In addition, the second loss function may be used to update the parameters of the inverse transformation model. In some examples, the first energy transformation model and the inverse transformation model may be initialized with the same parameters. As training continues and the parameters of the first energy transformation model are updated based on each first loss function and each second loss function, the first energy transformation model may be trained to produce a transformed image at a second energy level, which may also be transformed back to the first energy level. The parameters of the inverse transformation model may be updated based on each second loss function (rather than the first loss function) so that the inverse transformation model learns to transform the image from the second energy level to the first energy level. This forward-backward training ensures data and cycle consistency so that for each position in the image, the transformation μ(E1)→μ(E2)→μ(E1) is applicable. This constrains the forward transformation and prevents any unwanted behavior at each voxel neighborhood. For example, forward-backward training maintains structural integrity and prevents tissue or contrast mixing across voxels. With respect to cycle consistency, the second image is constrained to reduce or prevent deviations in geometry and tissue (Hu) integrity, as this would hinder the inverse transformation.

[0133] At 1120, method 1100 includes determining whether additional training images remain in the plurality of training images. As explained above, the first data set may include a plurality of training image pairs that may be used to train the first energy transformation model. If less than all of the training image pairs in the first data set have been selected and used to train the first energy transformation model (e.g., there are at least some training images remaining), or if it is otherwise determined that the first energy transformation model has not been fully trained, method 1100 returns to 1106 to select the next training image pair from the first data set and use the next training image pair to train the first energy transformation model. However, if at 1120, it is determined that each training image pair has been selected and used to train the first energy transformation model (and no more training images remain), or if it is otherwise determined that the first energy transformation model is fully trained, method 1100 proceeds to 1122, which includes training additional energy transformation models using the remaining data sets, one data set for each comparison period. Thus, the method may be repeated for each data set in order to train a plurality of different energy transformation models. Method 1100 then returns.

[0134] Figure 12 to Figure 14 An example contrast image of a subject in the portal venous phase is shown in accordance with an embodiment of the present disclosure. Figure 12 to Figure 14 The images shown are contrast-enhanced axial images of the subject's trunk obtained during the portal venous phase of contrast. Fig.12 A first example image 1200 of a subject reconstructed from projection data acquired at two energy levels (eg, 40 kVp and 140 kVp) is illustrated. The first example image 1200 may be an image at a first energy level 70 keV formed by a first linear combination of material-based images reconstructed from the projection data. Fig.13 1 is a second example image 1300 of the subject reconstructed from the same dual energy projection data as the first example image 1200, but the second example image is an image at a second energy level of 50 keV formed by a second linear combination of the material basis images. Therefore, the second example image 1300 may be a ground truth image, and the first example image 1200 and the second example image 1300 may be examples of a pair of images that may be used to train an energy transformation model to transform an image from a first energy level to a second energy level. Fig.14 The third example image 1400 at the second energy level 50 keV generated according to the embodiments described herein is included. Specifically, the third example image 1400 may be generated by inputting the first example image 1200 as input into the trained energy conversion model. Figure 12 to Figure 14 As understood, the transformed image (e.g., the third example image 1400) has improved contrast detectability relative to the original higher energy image (e.g., the first example image 1200) and is similar to the target lower energy image (e.g., the second example image 1300). In addition, the transformed image may have lower noise and fewer artifacts than an image acquired at the target energy (e.g., an image acquired at 50 keV).

[0135] Figures 15 to 17 Additional example contrast images of a subject in the arterial phase are shown in accordance with embodiments of the present disclosure. Figures 15 to 17 The image shown is a contrast-enhanced coronal image of the subject's torso obtained during the arterial phase of contrast. Fig.15 A first example image 1500 of a subject reconstructed from projection data acquired at two energy levels (eg, 40 kVp and 140 kVp) is illustrated. The first example image 1500 may be an image at a first energy level 70 keV formed by a first linear combination of material-based images reconstructed from the projection data. Fig.161 is a second example image 1600 of the subject reconstructed from the same dual energy projection data as the first example image 1500, but the second example image is an image at a second energy level of 50 keV formed by a second linear combination of the material basis images. Therefore, the second example image 1600 may be a ground truth image, and the first example image 1500 and the second example image 1600 may be an example of a pair of images that may be used to train an energy transformation model to transform an image from a first energy level to a second energy level. Fig.17 A third example image 1700 at a second energy level of 50 keV is included, generated according to the embodiments described herein. Specifically, the third example image 1700 may be generated by inputting the first example image 1500 as input into a trained energy conversion model. Figures 15 to 17 As understood, the transformed image (e.g., the third example image 1700) has improved contrast detectability relative to the original higher energy image (e.g., the first example image 1500) and is similar to the target lower energy image (e.g., the third example image 1400). In addition, the transformed image may have lower noise and fewer artifacts than an image acquired at the target energy (e.g., an image acquired at 50 keV).

[0136] A technical effect of transforming an image from a first energy level to a second energy level using an energy transformation model that is selected based on the contrast period of the image (and trained specifically for the contrast period) is that the transformation can be performed in a contrast-aware manner to generate an image at a desired energy level to improve contrast detectability while avoiding noise and artifact issues. Doing so can allow an image at a desired energy level to be obtained even if projection data acquired at only a single peak energy level is available, thereby avoiding the need for an additional imaging system.

[0137] The present disclosure also provides support for a method, the method comprising: obtaining an image at a first energy level acquired using a single-energy computed tomography (CT) imaging system; identifying a contrast period of the image; entering the image as input into an energy transformation model trained to output a transformed image at a second energy level different from the first energy level, the energy transformation model being selected from a plurality of energy transformation models based on the contrast period; and displaying a final transformed image and / or storing the final transformed image in a memory, wherein the final transformed image is the transformed image or is generated based on the transformed image. In a first example of the method, identifying the contrast period of the image comprises identifying the contrast period of the image using a contrast period classifier, the contrast period classifier comprising a deep learning model trained using a plurality of training triplets, each training triplet comprising a set of projection images generated from a 3D volume of a subject. In a second example of the method that optionally includes the first example, each set of projection images includes a first annotated maximum intensity projection (MIP) training image in a first scanning plane, a second annotated MIP training image in a second scanning plane, and a third annotated MIP training image in a third scanning plane, and wherein the corresponding annotation of each annotated MIP training image indicates a contrast period included in the annotated MIP training image. In a third example of the method that optionally includes one or both of the first and second examples, the energy transformation model is trained using training pairs, each training pair includes a first training image at the first energy level and a second training image at the second energy level, and wherein the first training image and the second training image are monochromatic images acquired using a dual-energy CT imaging system. In a fourth example of the method that optionally includes one or more or each of the first to third examples, during training, the energy transformation model is configured to output a transformed training image based on an input first training image, and wherein the energy transformation model is further trained based on an inverse training image, which is generated by an inverse energy transformation model based on the transformed training image. In a fifth example of the method, which optionally includes one or more or each of the first to fourth examples, the energy transformation model is a first energy transformation model, and the transformed image is a first transformed image, and wherein the final transformed image is generated based on the first transformed image by entering the first transformed image as an input into a second energy transformation model trained to output the final transformed image at a third energy level, the second energy level being different from the third energy level. In a sixth example of the method, which optionally includes one or more or each of the first to fifth examples, the contrast period is a first contrast period, and wherein identifying the contrast period of the image includes identifying the first contrast period and a second contrast period of the image and a ratio of the first contrast period to the second contrast period.In a seventh example of the method, which optionally includes one or more or each of the first to sixth examples, the energy transformation model is a first energy transformation model, and the transformed image is a first transformed image, and the method further includes: entering the image as an input to a second energy transformation model trained to output a second transformed image at the second energy level, the second energy transformation model being selected from the plurality of energy transformation models based on the second contrast period. In an eighth example of the method, which optionally includes one or more or each of the first to seventh examples, the method further includes: mixing the first transformed image and the second transformed image to generate the final transformed image. In a ninth example of the method, which optionally includes one or more or each of the first to eighth examples, the mixing includes weighting the first transformed image and the second transformed image based on the ratio of the first contrast period relative to the second contrast period.

[0138] The present disclosure also provides support for a system, the system comprising: one or more processors; and a memory storing instructions executable by the one or more processors to perform the following operations: obtaining an image at a first energy level, the image being reconstructed from projection data acquired at a single peak energy level; identifying the contrast period of the image using a contrast period classifier model; entering the image as input into an energy transformation model trained to output a transformed image at a second energy level different from the first energy level, the energy transformation model being selected from a plurality of energy transformation models based on the contrast period; and displaying a final transformed image and / or storing the final transformed image in a memory, wherein the final transformed image is the transformed image or is generated based on the transformed image. In a first example of the system, the contrast period includes one or more of no contrast, venous phase, portal venous phase, arterial phase, and delayed phase. In a second example of the system, optionally including the first example, the first energy level is greater than the second energy level. In a third example of the system, which optionally includes one or both of the first example and the second example, training the contrast period classifier model includes: obtaining multiple training triplets, each training triplet including a set of 3 projection images of a corresponding contrast period among multiple contrast periods; entering a selected training triplet from the multiple training triplets as input to the contrast period classifier model; receiving one or more predicted contrast periods included in the selected training triplet from the contrast period classifier model; comparing the one or more predicted contrast periods with one or more underlying true contrast periods indicated by annotations of the selected training triplet; and adjusting model parameters of the contrast period classifier model based on the comparison. In a fourth example of the system that optionally includes one or more or each of the first to third examples, training the energy transformation model includes: entering a first image of a training image pair into the energy transformation model, the first image being at the first energy level; receiving a first transformed training image output from the energy transformation model; determining a loss function based on the first transformed training image and a second image of the training image pair, the second image being at the second energy level; and updating the energy transformation model based on the loss function, wherein the first image and the second image are monochromatic images generated from dual-energy projection data. In a fifth example of the system that optionally includes one or more or each of the first to fourth examples, training the energy transformation model also includes: calculating a second loss function based on the first image of the training image pair and an inverse transformed image, the inverse transformed image being generated from an inverse transformation model based on the first transformed training image; and updating the energy transformation model based on the second loss function.

[0139] The present disclosure also provides support for a method, which includes: obtaining an image of a subject at a first energy level, the image being reconstructed using projection data acquired using a single-energy computed tomography (CT) imaging system; identifying a first contrast period and a second contrast period in the image using a contrast period classifier model; selecting a first energy transformation model for the first contrast period and a second energy transformation model for the second contrast period; entering the image as input into the first energy transformation model and the second energy transformation model, each of the first energy transformation model and the second energy transformation model being trained to output a corresponding transformed image at a second energy level based on the image at the first energy level; mixing each corresponding transformed image to form a final transformed image at the second energy level; and displaying the final transformed image on a display device and / or storing the final transformed image in a memory. In a first example of the method, the first energy transformation model outputs a first transformed image at the second energy level, and the second energy transformation model outputs a second transformed image at the second energy level, wherein the contrast period classifier model outputs a ratio of the first contrast period to the second contrast period, and wherein the blending includes weighting the first transformed image and the second transformed image based on the ratio. In a second example of the method that optionally includes the first example, the final transformed image is a first final transformed image, and the method further includes: selecting a third energy transformation model for the first contrast period and a fourth energy transformation model for the second contrast period; and entering the first final transformed image at the second energy level as input to the third energy transformation model and the fourth energy transformation model, each of which is trained to output a corresponding other transformed image at a third energy level based on the first final transformed image at the second energy level; and blending each corresponding other transformed image to form the second final transformed image at the third energy level. In a third example of the method, which optionally includes one or both of the first example and the second example, the third energy level is different from both the first energy level and the second energy level, and the third energy transformation model outputs a third transformed image at the third energy level and the fourth energy transformation model outputs a fourth transformed image at the third energy level, wherein the contrast period classifier model outputs a ratio of the first contrast period relative to the second contrast period, and wherein the mixing includes weighting the third transformed image and the fourth transformed image based on the ratio.

[0140] As used herein, the element or step that is listed in the singular and begins with the word "one" or "an" should be understood as not excluding a plurality of the elements or steps, unless such exclusion is explicitly stated. In addition, the reference to "an embodiment" of the present invention is not intended to be interpreted as excluding the existence of the additional embodiment that also includes the quoted feature. In addition, unless explicitly stated to the contrary, the embodiment of "including", "comprising" or "having" the element or multiple elements with a specific characteristic may include additional such elements that do not have the characteristic. The terms "including" and "in..." are used as the concise language equivalents of the corresponding terms "including" and "wherein". In addition, the terms "first", "second" and "third" etc. are only used as marks, and are not intended to impose numerical requirements or specific position order on their objects.

[0141] This written description uses examples to disclose the invention, including the best mode, and also to enable a person skilled in the relevant art to practice the invention, including making and using any devices or systems and performing any included methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to a person skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insignificant differences from the literal language of the claims.

Claims

1. A method, comprising: Obtaining (802) an image at a first energy level acquired using a single-energy computed tomography (CT) imaging system; identifying (804) a comparison period of the image; entering (814, 812) the image as input into an energy transformation model trained to output a transformed image at a second energy level different from the first energy level, the energy transformation model being selected (810, 812) from a plurality of energy transformation models based on the comparison period; and A final transformed image is displayed (818) and / or saved in a memory, wherein the final transformed image is or is generated based on the transformed image.

2. The method of claim 1 , wherein identifying the contrast period of the image comprises identifying ( 804 , 904 ) the contrast period of the image using a contrast period classifier, the contrast period classifier comprising a deep learning model trained ( 1002 ) using a plurality of training triplets, each training triplet comprising a set of projection images generated from a 3D volume of the subject.

3. The method of claim 2 , wherein each set of projection images comprises a first annotated maximum intensity projection (MIP) training image in a first scanning plane, a second annotated MIP training image in a second scanning plane, and a third annotated MIP training image in a third scanning plane, and wherein a corresponding annotation of each annotated MIP training image indicates the contrast period included in the annotated MIP training image.

4. The method according to claim 1, wherein the energy transformation model is trained using training pairs (1102), each training pair comprising a first training image at the first energy level and a second training image at the second energy level, and wherein the first training image and the second training image are monochrome images acquired using a dual-energy CT imaging system.

5. The method of claim 4, wherein during training (1108), the energy transformation model is configured to output a transformed training image based on an input first training image, and wherein the energy transformation model is further trained (1114) based on an inverse training image, the inverse training image being generated by the inverse energy transformation model based on the transformed training image.

6. The method of claim 1 , wherein the energy transformation model is a first energy transformation model, and the transformed image is a first transformed image, and wherein the final transformed image is generated based on the first transformed image by inputting ( 912 ) the first transformed image as input to a second energy transformation model trained to output the final transformed image at a third energy level different from the third energy level.

7. The method of claim 1, wherein the contrast period is a first contrast period, and wherein identifying the contrast period of the image comprises identifying (804) the first contrast period and a second contrast period of the image and a ratio of the first contrast period relative to the second contrast period.

8. The method of claim 7, wherein the energy transformation model is a first energy transformation model and the transformed image is a first transformed image, and the method further comprises: The image is entered as input (812) to a second energy transformation model trained to output a second transformed image at the second energy level, the second energy transformation model being selected (808) from the plurality of energy transformation models based on the second comparison period.

9. The method according to claim 8, further comprising: The first transformed image and the second transformed image are blended (816) to generate the final transformed image.

10. The method of claim 9, wherein the blending (816) includes weighting the first transformed image and the second transformed image based on the ratio of the first contrast period relative to the second contrast period.

11. A system, comprising: one or more processors (304); and A memory (306) storing instructions executable by the one or more processors to: obtaining (802) an image (502) at a first energy level, the image (502) being reconstructed from projection data acquired at a single peak energy level; identifying (804) a contrast period of the image (502) using a contrast period classifier model (616, 404); entering (814, 816) the image (502) as input into an energy transformation model (716) trained to output a transformed image (502) at a second energy level different from the first energy level, the energy transformation model (716) is selected from a plurality of energy conversion models (406) based on the comparison period; as well as A final transformed image (502) is displayed (818) and / or stored in a memory, wherein the final transformed image (502) is the transformed image (502) or is generated based on the transformed image (502).

12. The system of claim 11, wherein the contrast phase comprises one or more of no contrast, a venous phase, a portal venous phase, an arterial phase, and a delayed phase.

13. The system of claim 11, wherein training the contrast period classifier model (616, 404) comprises: obtaining (1002) a plurality of training triplets (608), each training triplet comprising a set of three projection images (402, 702) at a corresponding contrast period in a plurality of contrast periods; Entering (1006) a selected training triplet from the plurality of training triples (608) as input to the comparison period classifier model (616, 404); receiving (1008) from the contrast period classifier model (616, 404) one or more predicted contrast periods included in the selected training triples; comparing the one or more predicted contrast periods to one or more ground truth contrast periods indicated via annotations of the selected training triples (1010); and Model parameters of the comparison period classifier model (616, 404) are adjusted (1010) based on the comparison.

14. The system of claim 11, wherein training the energy conversion model (716) comprises: entering (1108) a first image (502) of a pair of training images (502) into the energy transformation model (716), the first image (502) being at the first energy level; Receiving (1110) a first transformed training image (502) output from the energy transformation model (716); determining (1116) a loss function based on the first transformed training image (502) and a second image (502) of the pair of training images (502), the second image (502) being at the second energy level; and The energy transformation model (716) is updated (1116) based on the loss function, wherein the first image (502) and the second image (502) are monochromatic images (402, 702) generated from dual energy projection data.

15. The system of claim 14, wherein the training of the energy conversion model (716) further comprises: Calculating (1118) a second loss function based on the first image (502) and the inverse transformed image (502) of the training image (502) pair, wherein the inverse transformed image (502) is generated from an inverse transformation model based on the first transformed training image (502); and updating (1118) the energy transformation model (716) based on the second loss function.

Images