System and method for generating noise-reduced spectral CT image from spectral CT image data acquired using spectral CT imaging system

Abstract

Various systems and methods for noise reduction of spectral CT image data are provided that include determining a noise reduction linear estimate of the spectral CT image data by maximizing or minimizing a first objective function, where at least one parameter of the noise reduction linear estimate is determined by at least one machine learning system. The noise reducer is based on a linear minimum mean square error (LMMSE) estimator. The LMMSE calculation is very fast, but is not usually used for CT image noise reduction because it cannot adapt the noise reduction amount to different parts of the image and it is difficult to derive accurate statistical attributes from CT image data. To overcome these problems, a model-based deep learning model is provided, such as a deep neural network that retains a model-based LMMSE structure.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 396,686, filed on August 10, 2022, the disclosure of which is incorporated herein by reference in its entirety. Background Art

[0003] Embodiments of the subject matter disclosed herein relate to X-ray technology and X-ray imaging and corresponding imaging reconstruction and imaging tasks. Specifically, embodiments of the subject matter disclosed herein relate to systems and methods for generating de-noised spectral computed tomography (CT) images from spectral CT image data acquired using a spectral (energy-resolved) CT imaging system.

[0004] Radiographic imaging systems, such as computed tomography (CT) imaging systems and other more general X-ray imaging systems, have been used for many years in medical applications such as medical diagnosis and treatment.

[0005] Typically, an X-ray imaging system such as a CT imaging system includes an X-ray source and an X-ray detector, which consists of a plurality of detector modules including one or more detector elements for independently measuring the intensity of the X-rays. The X-ray source emits X-rays, which pass through the subject or object to be imaged and are then received by the detector. The energy spectrum of a typical medical X-ray tube is very wide, ranging from zero up to 160 keV. Therefore, the X-ray detector generally detects X-rays with varying energy levels.

[0006] The X-ray source and X-ray detector are usually arranged to rotate around the subject or object on a rotating member of a gantry. The emitted X-rays are attenuated by the subject or object as they pass through, and the resulting transmitted X-rays are measured by the detector. The measured data can then be used to reconstruct an image of the subject or object.

[0007] The challenge facing X-ray detectors is to extract maximum information from the detected X-rays to provide input for an image of an object or subject, wherein the object or subject is characterized in terms of density, composition and structure.

[0008] refer to Figure 1AIt may be useful to briefly summarize an illustrative, general X-ray imaging system according to the prior art. In this illustrative example, the X-ray imaging system 100 includes an X-ray source 10, an X-ray detector 20, and an associated image processing system 30. Generally speaking, the X-ray detector 20 is configured to record radiation from the X-ray source 10, which radiation has optionally been focused by an optional X-ray optics or collimator and has passed through an object, subject, or portion thereof. The X-ray detector 20 may be connected to the image processing system 30 via suitable readout electronics that are at least partially integrated in the X-ray detector 20 to enable the image processing system 30 to perform image processing and / or image reconstruction.

[0009] By way of example, a conventional CT imaging system comprises an X-ray source and an X-ray detector arranged so that projection images of a subject or object can be acquired at different viewing angles covering at least 180 degrees. This is most commonly achieved by mounting the source and the detector on a support (e.g., a rotating member of a gantry) that can rotate around the subject or object. An image containing projections recorded in different detector elements for different viewing angles is called a sinogram. In the following, the collection of projections recorded in different detector elements for different viewing angles will be called a sinogram, even if the detector is two-dimensional, making the sinogram a three-dimensional image.

[0010] Figure 1B is a schematic diagram illustrating an example of an X-ray imaging system setup according to the prior art, showing projection lines from an X-ray source through an object to an X-ray detector.

[0011] A further development of X-ray imaging is energy-resolved X-ray imaging, also called spectral X-ray imaging, in which the X-ray transmission is measured for several different energy levels. This can be achieved by rapidly switching the source between two different emission spectra, by using two or more X-ray sources emitting different X-ray spectra, or by using an energy-resolving detector that measures the incident radiation at two or more energy levels. An example of such a detector is a multi-bin photon counting detector, in which each recorded photon generates a current pulse that is compared to a set of thresholds, thereby counting the number of photons incident in each of a plurality of energy bins.

[0012] Spectral X-ray projection measurement produces projection images for each energy level. A weighted sum of these projection images can be performed to optimize the contrast-to-noise ratio (CNR) for a given imaging task, as described in "SNR and DQE analysis of broadspectrum X-ray imaging", Tapiovaara and Wagner, Phys. Med. Biol. 30, 519.

[0013] Another technique enabled by energy-resolved X-ray imaging is basis material decomposition. This technique exploits the fact that all substances composed of low atomic number elements (such as human tissue) have a linear attenuation coefficient whose energy dependence can be well approximated as a linear combination of two (or more) basis functions:

[0014] μ(E)=a1f1(E)+a2f2(E)

[0015] where f1 and f2 are basis functions, and a1 and a2 are the corresponding basis coefficients. More generally, f i is a basis function, and a i are the corresponding basis coefficients, where i = 1, …, N, where N is the total number of basis functions. If one or more elements with high atomic numbers are present in the imaging volume, high enough to have a K absorption edge in the energy range used for imaging, one basis function must be added for each such element. In the field of medical imaging, such K-edge elements can typically be iodine or gadolinium, which are substances used as contrast agents.

[0016] Basis material decomposition has been described in "Energy-selective reconstructions in X-ray computerized tomography", Alvarez, Macovski, Phys Med Biol. 1976; 21(5): 733-744. In basis material decomposition, the integral of each basis coefficient in the basis coefficients (i=1,…,N where N is the number of basis functions) according to each projection ray from the source to the detector element In one embodiment, this is done by first expressing the expected number of recorded counts in each energy bin as A i The function is implemented.

[0017]

[0018] where λ i is the expected number of counts in energy bin i, E is the energy, S i is a response function that depends on the shape of the spectrum incident on the imaged object, the quantum efficiency of the detector, and the sensitivity of energy bin i to X-rays of energy E. Although the term "energy bin" is most commonly used for photon counting detectors, this formula can also describe other energy-resolving X-ray imaging systems, such as multi-slice detectors, kVp-switched X-ray sources, or multiple X-ray source systems.

[0019] Then, A can be estimated using maximum likelihood, assuming that the number of counts in each bin is a Poisson-distributed random variable. i This is achieved by minimizing the negative log-likelihood function, see, for example, “K-edge imaging in X-ray computed tomography using multi-bin photon counting detectors”, Roessland Proksa, Phys. Med. Biol. 52 (2007), 4679-4696:

[0020]

[0021] Where m i is the number of counts measured in energy bin i, and M b is the number of energy bins.

[0022] When the resulting estimated basis coefficients for each projection line are integrated When arranged into an image matrix, the result is a material-specific projection image of each basis i, also called a basis image. This basis image can be viewed directly (e.g. in projection X-ray imaging) or can be used as a basis coefficient a for forming the interior of an object i The input to a reconstruction algorithm of the map (e.g. in CT imaging). In either case, the result of the basis decomposition can be viewed as one or more basis image representations, such as basis coefficient line integrals or the basis coefficients themselves.

[0023] Standard data management procedures for X-ray imaging systems propose different approaches to optimize data acquisition, but possibly at the expense of, for example, spatial resolution, noise level and / or system complexity.

[0024] Mapping of the basis coefficients inside the object a i It is called base material image, base image, material image, material specific image, material map or foundation map.

[0025] However, a well-known limitation of this technique is that the variance of the estimated line integral generally increases with the number of bases used in the basis decomposition. This results in an unfortunate trade-off between improved tissue quantification and increased image noise, among other things.

[0026] In addition, an accurate basis decomposition with more than two basis functions may be difficult to perform in practice and may result in artifacts, biases, or excess noise. Such a basis decomposition may also require extensive calibration measurements and data preprocessing to produce accurate results.

[0027] Due to the inherent complexity in many image reconstruction tasks, artificial intelligence (AI) and deep learning have begun to be used in general image reconstruction with satisfactory results. It is expected that AI and deep learning can be used for X-ray imaging tasks including spectral CT. However, improved noise reduction methods in spectral CT are generally needed.

[0028] Another current challenge in deep learning image reconstruction is its limited interpretability. An image may appear to have a very low noise level, but actually contain errors due to biases in the neural network estimator. Explainable AI techniques would be able to provide some information about why the output image has specific characteristics based on the input image and training data.

[0029] Another drawback of existing AI techniques is that they are generally non-tunable and therefore provide only a single output image for a given input image, which means that it is impossible to adjust the characteristics of the output image without retraining the network.

[0030] Therefore, there is a need for improved noise reduction methods for spectral computed tomography (CT) in general, and in particular for noise reduction methods that have a certain degree of interpretability and tunability. Summary of the invention

[0031] This summary introduces concepts that are described in more detail in the detailed description. It should not be used to identify essential features of the claimed subject matter, nor should it be used to limit the scope of the claimed subject matter.

[0032] The present inventors have recognized that image reconstruction in spectral CT imaging is more challenging due to two main reasons: 1) the multiple bins and materials in analysis and the improved resolution significantly increase the amount of processed data; and 2) effective material decomposition and image reconstruction methods tend to generate noisy images that do not meet the expected image quality. Therefore, there is a need to reduce the noise of material images.

[0033] In this disclosure, a fast denoiser is proposed, which is based on deep learning and linear minimum mean square error (LMMSE), combined in a model-based deep learning approach. In this way, prior knowledge is incorporated into the denoiser, but the linear estimator structure is retained to provide interpretability to the results. The architecture of the linear estimator with matrices and vectors estimated using deep neural networks provides a lot of flexibility that can outperform conventional deep learning denoisers.

[0034] This interpretability allows images with desired properties to be generated by adjusting the coefficients of the estimated linear estimator. For example, one or more coefficients in the linear estimator can be increased or decreased to reduce large area deviations or improve the preservation of fine details in the image.

[0035] Such adjustments may, for example, be made prior to image reconstruction, such as when developing a reconstruction method, or in real time while the image is displayed to an end user, thereby allowing the user to adjust the image to obtain desired image properties.

[0036] According to a first aspect, a method for denoising spectral CT image data is provided, the method comprising determining a denoised linear estimate of the spectral CT image data by maximizing or minimizing a first objective function, wherein at least one parameter of the denoised linear estimate is determined by at least one machine learning system.

[0037] According to a second aspect, a CT imaging system is provided, which includes: an X-ray source configured to emit X-rays; an X-ray detector configured to generate spectral CT image data; and a processor configured to determine a denoised linear estimate of the generated spectral CT image data based on maximizing or minimizing a first objective function, wherein the processor is also configured to determine at least one parameter of the linear estimate through at least one machine learning system.

[0038] Various systems and methods for denoising spectral CT image data are provided, the systems and methods comprising determining a denoised linear estimate for the spectral CT image data by maximizing or minimizing a first objective function, wherein at least one parameter of the denoised linear estimate is determined by at least one machine learning system. The denoiser is based on a linear minimum mean square error (LMMSE) estimator. The LMMSE is very fast to calculate, but is not typically used for CT image denoising because it does not adapt the amount of denoising to different parts of the image and it is difficult to derive accurate statistical properties from the CT image data. To overcome these challenges, a model-based deep learning model is provided, such as a deep neural network that retains the model-based LMMSE structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The embodiments, together with further objects and advantages thereof, may best be understood by reference to the following description taken in conjunction with the accompanying drawings, in which:

[0040] Figure 1A and Figure 1B is a schematic diagram illustrating an example of an X-ray imaging system.

[0041] Figure 2 is a schematic diagram illustrating another example of an X-ray imaging system such as a CT imaging system.

[0042] Figure 3 is a schematic block diagram of a CT imaging system as an illustrative example of an X-ray imaging system.

[0043] Figure 4is a schematic diagram illustrating another example of relevant components of an X-ray imaging system, such as a CT imaging system.

[0044] Figure 5 is a schematic diagram of a photon counting circuit and / or device according to an exemplary embodiment.

[0045] Figure 6 is a schematic diagram illustrating an example of a semiconductor detector submodule according to an exemplary embodiment.

[0046] Figure 7 is a schematic diagram illustrating an example of a semiconductor detector submodule according to another exemplary embodiment.

[0047] Fig. 8A is a schematic diagram illustrating an example of a semiconductor detector sub-module according to yet another exemplary embodiment.

[0048] Figure 8B is a schematic diagram showing an example of a set of tiled detector submodules, wherein each detector submodule is a depth-segmented detector submodule and an application-specific integrated circuit (ASIC) or corresponding circuit system is arranged below the detector submodule as can be seen from the direction of the incident X-ray.

[0049] Fig. 9 A schematic diagram of a denoising technique based on a conventional convolutional neural network (CNN) is shown, where the CNN directly maps a noisy image to a clean image.

[0050] Fig.10 A schematic diagram of the proposed denoising technique is shown, where a CNN is used to map the linear model parameters W and b for generating a clean image.

[0051] Fig.11 An example of the interpretability of the proposed model is shown, where three parameters σ, β and λ are used to control different components of the output image.

[0052] Fig.12 An example of the interpretation provided by the two components of a=Wx+b is shown.

[0053] Fig.13 is an empirical demonstration of the parallelism of our technique and LMMSE: the first row shows the estimator components of a conventional learned linear minimum mean square error (LMMSE) for a particular phantom example, and the second row shows the estimator components equivalent to our learned approach.

[0054] Fig.14 is a schematic diagram illustrating an example of a computer implementation according to an embodiment. DETAILED DESCRIPTION

[0055] Embodiments of the present disclosure will now be described by way of example with reference to the accompanying drawings.

[0056] For a better understanding, it may be useful to continue with the introductory description of a non-limiting example of an overall X-ray imaging system in which data processing and transmission according to the inventive concept may be implemented.

[0057] Figure 2 1 is a schematic diagram showing an example of an X-ray imaging system 100, such as a CT imaging system, which includes: an X-ray source 10 that emits X-rays; an X-ray detector 20 that has an X-ray detector that detects the X-rays after they have passed through an object; an analog processing circuit system 25 that processes and digitizes the raw electrical signals from the X-ray detector; a digital processing circuit system 40 that can perform further processing operations on the measurement data, such as applying corrections, temporary storage, or filtering; and a computer 50 that stores the processed data and can perform further post-processing and / or image reconstruction. The digital processing circuit system 40 may include a digital processor. According to an exemplary embodiment, all or part of the analog processing circuit system 25 may be implemented in the X-ray detector 20. The X-ray source and the X-ray detector may be coupled to a rotating member of a gantry 11 of the CT imaging system 100.

[0058] The entire X-ray detector may be considered an X-ray detector system 20 , or an X-ray detector 20 in combination with associated analog processing circuitry 25 .

[0059] The image processing system 30 communicates with and is electrically coupled to the analog processing circuit system 25, and may include a digital processing circuit system 40 and / or a computer 50, which may be configured to perform image reconstruction based on image data from the X-ray detector. Thus, the image processing system 30 may be viewed as the computer 50, or alternatively as a combination of the digital processing circuit system 40 and the computer 50, or, if the digital processing circuit system 40 is further specialized for image processing and / or reconstruction, it may be viewed as the digital processing circuit system itself.

[0060] An example of a commonly used X-ray imaging system is a CT imaging system, which may include an X-ray source or X-ray tube that generates a fan beam or cone beam of X-rays and an array of opposing X-ray detectors that measure the fraction of X-rays that pass through a patient or object. The X-ray source or X-ray tube and the X-ray detectors are mounted in a gantry 11 that can rotate around the imaging object.

[0061] Figure 3A CT imaging system 100 is schematically shown as an illustrative example of an X-ray imaging system. The CT imaging system includes a computer 50 that receives commands and scanning parameters from an operator via an operator console 60, which may have a display 62 and some form of operator interface, such as a keyboard, mouse, joystick, touch screen, or other input device. The commands and parameters provided by the operator are then used by the computer 50 to provide control signals to an X-ray controller 41, a gantry controller 42, and a table controller 43. Specifically, the X-ray controller 41 provides power and timing signals to the X-ray source 10 to control the emission of X-rays to an object or patient located on the table 12. The gantry controller 42 controls the rotation speed and position of the gantry 11 including the X-ray source 10 and the X-ray detector 20. For example, the X-ray detector 20 may be a photon counting X-ray detector. The table controller 43 controls and determines the position of the patient table 12 and the scan coverage of the patient. There is also a detector controller 44, which is configured to control and / or receive data from the X-ray detector 20.

[0062] In one embodiment, the computer 50 also performs post-processing and image reconstruction on the image data output from the X-ray detector 20. Thus, the computer 50 corresponds to FIG. 1 and FIG. Figure 2 The image processing system 30 is shown. An associated display 62 allows an operator to observe the reconstructed image and other data from the computer 50.

[0063] An X-ray source 10 arranged in a gantry 11 emits X-rays. An X-ray detector 20 (which may be in the form of a photon counting X-ray detector) detects the X-rays after they have passed through an object or patient. The X-ray detector 20 may be formed, for example, of a plurality of pixels (also referred to as sensors or detector elements) and associated processing circuitry (such as an application specific integrated circuit (ASIC)) arranged in a detector module. A portion of the analog processing may be implemented in the pixels, while any remaining processing is implemented, for example, in the ASIC. In one embodiment, the processing circuitry (ASIC) digitizes the analog signals from the pixels. The processing circuitry (ASIC) may also include digital processing that may perform further processing operations on the measurement data, such as applying corrections, temporary storage and / or filtering. During scanning to acquire X-ray projection data, the gantry and components mounted thereon rotate about an isocenter 13.

[0064] Modern X-ray detectors generally require the conversion of incident X-rays into electrons, which often occurs via the photoelectric effect or Compton interaction, and the resulting electrons generally produce secondary visible light until their energy is lost and this light is in turn detected by a photosensitive material. Semiconductor-based detectors also exist, and in this case the electrons generated by the X-rays generate a charge from electron-hole pairs that are collected by an applied electric field.

[0065] There are detectors that work in energy integrating mode, in the sense that they provide an integrated signal from a large number of X-rays. The output signal is proportional to the total energy deposited by the detected X-rays.

[0066] X-ray detectors with photon counting and energy resolution capabilities are increasingly being used in medical X-ray applications. Photon counting detectors have the advantage that in principle the energy of each X-ray can be measured, which yields additional information about the composition of the object. This information can be used to improve image quality and / or reduce radiation dose.

[0067] Generally speaking, photon counting X-ray detectors determine the energy of a photon by comparing the height of an electrical pulse generated by the interaction of a photon in the detector material with a set of comparator voltages. These comparator voltages are also called energy thresholds. Generally speaking, the analog voltage in the comparator is set by a digital-to-analog converter (DAC). The DAC converts the digital setting sent by the controller into an analog voltage to which the height of the photon pulse can be compared.

[0068] Photon counting detectors count the number of photons that have interacted in the detector during the measurement time. New photons are generally identified by the fact that the height of the electrical pulse exceeds the comparator voltage of at least one comparator. When a photon is identified, the event is stored by incrementing a digital counter associated with the channel.

[0069] When several different thresholds are used, an energy-resolving photon counting detector is obtained, in which the detected photons can be classified into energy bins corresponding to various thresholds. Sometimes, this type of photon counting detector is also called a multi-bin detector. In general, energy information allows the creation of new kinds of images, in which new information is available and image artifacts inherent to conventional techniques can be removed. In other words, for an energy-resolving photon counting detector, the pulse height is compared with multiple programmable thresholds (T1-TN) in a comparator and classified according to the pulse height, which is in turn proportional to the energy. In other words, a photon counting detector comprising more than one comparator is referred to herein as a multi-bin photon counting detector. In the case of a multi-bin photon counting detector, the photon counts are stored in a set of counters, typically each counter corresponding to an energy threshold. For example, a count can be assigned to the highest energy threshold that a photon pulse has exceeded. In another example, a counter tracks the number of times a photon pulse crosses each energy threshold.

[0070] For example, "side-on facing" is a special non-limiting design of a photon counting detector in which the X-ray sensors (such as X-ray detector elements or pixels) are oriented sideways toward incoming X-rays.

[0071] For example, such a photon counting detector may have pixels in at least two directions, wherein one of the two directions facing the sideways photon counting detector has a component in the direction of the X-ray. Such a sideways photon counting detector is sometimes referred to as a depth segmented photon counting detector having two or more pixel depth segments in the direction of the incoming X-ray. It should be noted that one detector element may correspond to one pixel, and / or multiple detector elements may correspond to one pixel, and / or data signals from multiple detector elements may be used for one pixel.

[0072] Alternatively, the pixels may be arranged in an array (non-depth segmented) in a direction substantially orthogonal to the incident X-rays, and each pixel may be oriented sideways toward the incident X-rays. In other words, the photon counting detector may be non-depth segmented while still being arranged sideways toward the incoming X-rays.

[0073] The absorption efficiency can be improved by arranging the side-facing photon counting detector to face sideways, in which case the absorption depth can be chosen to be any length and the side-facing photon counting detector can still be fully depleted without reaching very high voltages.

[0074] The conventional mechanism for detecting X-ray photons by direct semiconductor detectors works essentially as follows. The energy of the X-ray interaction in the detector material is converted into electron-hole pairs inside the semiconductor detector, where the number of electron-hole pairs is generally proportional to the photon energy. The electrons and holes drift toward the detector electrodes and back (or vice versa). During this drift, the electrons and holes induce a current in the electrodes, which can be measured.

[0075] like Figure 4 As shown in , the signals are routed via wiring paths 26 of the detector elements 22 of the X-ray detector to the input of an analog processing circuit system (e.g., an ASIC) 25. It should be understood that the term "application specific integrated circuit (ASIC)" should be broadly interpreted as any general purpose circuit used and configured for a particular application. The ASIC processes the charge generated from each X-ray and converts it into digital data, which can be used to obtain measurement data, such as photon counts and / or estimated energy. The ASIC is configured to be connected to the digital processing circuit system so that the digital data can be sent to the digital processing circuit system 40 and / or one or more memory circuits or components 45, and ultimately the data will be provided for Figure 2 The image processing circuitry 30 or computer 50 in generates input for reconstructing the image.

[0076] Since the number of electrons and holes from one X-ray event is proportional to the energy of the X-ray photon, the total charge in one induced current pulse is proportional to this energy. After the filtering step in the ASIC, the pulse amplitude is proportional to the total charge in the current pulse and therefore proportional to the X-ray energy. The pulse amplitude can then be measured by comparing the value of the pulse amplitude with one or more thresholds (THR) in one or more comparators (COMP) and introducing a counter by which the number of cases where the pulse is greater than the threshold can be recorded. In this way, the number of X-ray photons whose energy exceeds the energy corresponding to the corresponding threshold (THR) that has been detected within a certain time frame can be counted and / or recorded.

[0077] The ASIC typically samples the analog photon pulse once per clock cycle and records the output of the comparator. Depending on whether the analog signal is above or below the comparator voltage, the comparator (threshold) outputs a one or zero. The available information at each sample is, for example, a one or zero for each comparator, which indicates whether the comparator has been triggered (the photon pulse is above the threshold) or not triggered.

[0078] In a photon counting detector, there is typically a photon counting logic component that determines whether a new photon has been recorded and records the photon in a counter. In the case of a multi-bin photon counting detector, there are typically several counters, such as one counter for each comparator, and the photon counts are recorded in these counters based on an estimate of the photon energy. This logic component can be implemented in several different ways. Two of the most common categories of photon counting logic components are non-paralyzable counting mode and paralyzable counting mode. Other photon counting logic components include, for example, local maximum detection, which counts the local maximum values ​​detected in the voltage pulse and may also record its pulse height.

[0079] Photon counting detectors have many benefits, including but not limited to: high spatial resolution, low sensitivity to electronic noise, good energy resolution; and material separation capabilities (spectral imaging capabilities). However, energy integrating detectors have the advantage of high count rate tolerance. The count rate tolerance comes from the fact / recognition that since the total energy of the photons is measured, adding one additional photon will always increase the output signal (within reasonable limits), regardless of the amount of photons currently being recorded by the detector. This advantage is one of the main reasons why energy integrating detectors have become the standard for medical CT today.

[0080] Figure 5 A schematic diagram of a photon counting circuit and / or device according to an exemplary embodiment is shown.

[0081] When photons interact in a semiconductor material, a cloud of electron-hole pairs is generated. The charge carriers are collected by electrodes attached to the detector material by applying an electric field across the detector material. The signal is routed from the detector element to the input of a parallel processing circuit (e.g., an ASIC). In one example, the ASIC can process the charge so that a voltage pulse is generated whose maximum height is proportional to the amount of energy deposited by the photon in the detector material.

[0082] The ASIC may include a set of comparators 302, each of which compares the magnitude of the voltage pulse to a reference voltage. The comparator output is typically zero or one (0 / 1), depending on which of the two voltages being compared is larger. Here, we assume that if the voltage pulse is higher than the reference voltage, the comparator output is one (1); if the reference voltage is higher than the voltage pulse, the comparator output is zero (0). A digital-to-analog converter (DAC) 301 can be used to convert a digital setting that may be provided by a user or a control program into a reference voltage that may be used by the comparators 302. If the height of the voltage pulse exceeds the reference voltage of a particular comparator, we will refer to that comparator as being "triggered." Each comparator is typically associated with a digital counter 303, which increments based on the comparator output according to the photon counting logic component.

[0083] As mentioned earlier, when the resulting estimated basis coefficients for each projection ray are integrated When arranged into an image matrix, the result is a material-specific projection image of each basis i, also called a basis image. This basis image can be viewed directly (e.g. in projection X-ray imaging) or can be used as a basis coefficient a for forming the interior of an object i Input to a reconstruction algorithm of a map (eg in CT). In any case, the result of the basis decomposition can be viewed as one or more basis image representations, such as basis coefficient line integrals or the basis coefficients themselves.

[0084] It will be appreciated that the mechanisms and arrangements described herein can be implemented, combined and rearranged in various ways.

[0085] For example, embodiments may be implemented in hardware, or at least partly in software executed by appropriate processing circuitry, or a combination of the above.

[0086] The steps, functions, processes and / or blocks described herein may be implemented in hardware (including both general purpose electronic circuitry and application specific circuitry) using any conventional technology such as discrete circuitry or integrated circuit technology.

[0087] Alternatively, or in addition, at least some of the steps, functions, processes and / or blocks described herein can be implemented in software, such as a computer program executed by appropriate processing circuitry (such as one or more processors or processing units).

[0088] In the following, non-limiting examples of specific detector module implementations will be discussed. More specifically, these examples refer to side-facing oriented detector modules and depth-segmented detector modules. Other types of detectors and detector modules are also possible.

[0089] Figure 6 is a schematic diagram illustrating an example of a semiconductor detector submodule according to an exemplary embodiment. This is an example of a detector module 21 in which the semiconductor sensor has a plurality of detector elements or pixels 22, wherein each detector element (or pixel) is typically based on a diode having a charge collection electrode as a key component. X-rays enter through the edge of the detector module.

[0090] Figure 7 is a schematic diagram illustrating an example of a semiconductor detector submodule according to another exemplary embodiment. In this example, again assuming that X-rays enter through the edge of the detector module, the detector module 21 with semiconductor sensors is also divided into a plurality of depth segments or detector elements 22 in the depth direction.

[0091] Typically, a detector element is a single X-ray sensitive sub-element of a detector. Generally speaking, photon interactions occur in a detector element and the charge generated thereby is collected by corresponding electrodes of the detector element.

[0092] Each detector element typically measures the incident X-ray flux as a sequence of frames. A frame is the data measured during a specified time interval, called a frame time.

[0093] Depending on the detector topology, a detector element may correspond to a pixel, especially when the detector is a flat panel detector. A depth segmented detector may be viewed as having multiple detector strips, each strip having multiple depth segments. For such a depth segmented detector, each depth segment may be viewed as a separate detector element, especially if each of the depth segments is associated with its own separate charge collection electrode.

[0094] The detector strips of a depth segmented detector usually correspond to pixels of a common flat panel detector and are therefore sometimes also called pixel strips. However, a depth segmented detector can also be viewed as a three-dimensional pixel array, where each pixel corresponds to a separate depth segmenter / detector element.

[0095] The semiconductor sensor can be implemented as a so-called multi-chip module (MCM), in the sense that it serves as a base substrate for electrical wiring and multiple ASICs, which are preferably attached by so-called flip-chip technology. The wiring will include signal connections from each pixel or detector element to the ASIC input and connections from the ASIC to external memory and / or digital data processing. The power to the ASIC can be supplied by similar wiring, but also by separate connections, taking into account the increase in the cross-section required for the high currents in these connections. The ASIC can be positioned to the side of the active sensor, which means that it can be protected from incident X-rays if an absorber cover is placed on top, and it can also be protected from scattered X-rays coming from the side by positioning the absorber in this direction as well.

[0096] Fig. 8A8,183,535 is a schematic diagram showing a detector module implemented as an MCM similar to the embodiment in U.S. Patent No. 8,183,535. In this example, it is shown how the semiconductor sensor 21 can also have the function of the substrate in the MCM. The signal is routed from the detector element 22 to the input of the parallel processing circuit 24 (e.g., ASIC) positioned next to the active sensor area via the wiring path 23. The ASIC processes the charge generated from each X-ray and converts it into digital data, which can be used to detect photons and / or estimate the energy of the photons. The ASIC can have their own digital processing circuit system and memory for small tasks. In addition, the ASIC can be configured to connect to the digital processing circuit system and / or memory circuit or component located outside the MCM, and ultimately use the data as input for reconstructing the image.

[0097] However, the adoption of deep segmentation also brings two notable challenges to silicon-based photon counting detectors. First, a large number of ASIC channels must be employed to process the data fed from the associated detector segments. In addition to the increased number of channels due to both smaller pixel size and deep segmentation, multiple energy bins also increase the data size. Second, since a given X-ray input count is divided into smaller pixels, segments, and energy bins, each with a much lower signal, detector calibration / correction requires several orders of magnitude more calibration data to minimize statistical uncertainty.

[0098] Naturally, in addition to requiring larger computing resources, hard drives, memory, and central processing units (CPUs) or graphics processing units (GPUs), the data size that is several orders of magnitude larger also slows down both data processing and preprocessing. For example, when the data size is 10GB instead of 10MB, the processing time for reading and writing data may be as long as 1000 times.

[0099] A problem in any detector that counts X-ray photons is the pile-up problem. When the flux rate of X-ray photons is high, problems may arise in distinguishing two subsequent charge pulses. As mentioned above, the pulse length after the filter depends on the shaping time. If this pulse length is greater than the time between two X-ray photon induced charge pulses, the pulses will grow together and the two photons will be indistinguishable and can be counted as one pulse. This is called pile-up. Therefore, one way to avoid pile-up at high photon fluxes is to use a small shaping time, or to use deep segmentation.

[0100] For pile-up calibration vector generation, the pile-up calibration data needs to be pre-processed for jet correction. For material decomposition vector generation, the material decomposition data should preferably be pre-processed for both jet correction and pile-up correction. For patient scan data, the data needs to be pre-processed for jet, pile-up, and material decomposition before image reconstruction ensue. These are simplified examples to explain "pre-processing" as the actual pre-processing steps may include several other calibration steps such as reference normalization and air calibration as needed. The term "processing" may simply indicate the last step of each calibration vector generation or patient scan, but may be used interchangeably in some cases.

[0101] Figure 8B is a schematic diagram showing an example of a set of tiled detector submodules, wherein each detector submodule is a deeply segmented detector submodule and an ASIC or corresponding circuit system 24 is arranged below the detector elements 22 as can be seen from the direction of the incoming X-rays, thereby allowing wiring paths 23 from the detector elements 22 to parallel processing circuits 24 (e.g., ASICs) in the space between the detector elements.

[0102] Artificial intelligence (AI) and deep learning have begun to be used in general image reconstruction with some promising results. However, a current challenge in deep learning image reconstruction is its limited interpretability. An image may appear to have a very low noise level but actually contain errors due to biases in the neural network estimator.

[0103] Typically, deep learning involves machine learning methods based on artificial neural networks or similar architectures with representation learning. Learning can be supervised, semi-supervised, or unsupervised. Deep learning systems (such as deep neural networks, deep belief networks, recurrent neural networks, and convolutional neural networks) have been applied to a variety of technical fields, including computer vision, speech recognition, natural language processing, social network filtering, machine translation, and board game programs, where they have produced results that are comparable to and in some cases exceed the performance of human experts.

[0104] The adjective "deep" in deep learning stems from the use of multiple layers in the network. Early work showed that linear perceptrons cannot be general classifiers, and on the other hand, networks with non-polynomial activation functions with one hidden layer of unbounded width can be general classifiers. Deep learning is a modern variant involving an unlimited number of layers of bounded size, which allows practical applications and optimized implementations while maintaining theoretical generality under mild conditions. In deep learning, for efficiency, trainability, and understandability, the layers are also allowed to be heterogeneous and deviate widely from biologically understood connection models.

[0105] The present inventors have appreciated that there is a need for denoising algorithms with improved performance for spectral CT, and in particular algorithms with improved interpretability.

[0106] The proposed techniques are generally applicable to providing denoised image data in spectral CT based on neural networks and / or deep learning.

[0107] To provide an exemplary framework for facilitating understanding of the proposed techniques, a specific example of deep learning based image reconstruction in the specific context of spectral CT image reconstruction will now be given.

[0108] However, it should be understood that the proposed techniques for providing an indication of confidence in deep learning image reconstruction in spectral CT applications are generally applicable to deep learning based image reconstruction for CT and are not limited to the following specific example of deep learning based image reconstruction.

[0109] The inventors disclose a new fast denoiser based on a linear minimum mean square error (LMMSE) estimator. LMMSE is very fast to compute but is not typically used for CT image denoising, probably because it cannot adapt the amount of denoising to different parts of the image and it is difficult to derive accurate statistical properties from CT data. To overcome these challenges, the inventors propose a model-based deep learning strategy, i.e. a deep neural network that retains the LMMSE structure (model-based), thereby providing more robustness to unseen data and good interpretability of the results. In this way, the solution adapts to the anatomical structure in each point of the image and the noise properties at that specific location.

[0110] As an exemplary, non-limiting embodiment of the present disclosure, let us assume linear minimum mean square error (LMMSE) to denoise the dual-material image after FPB, i.e., x = [x1, x2]. This is the solution of the following formula:

[0111]

[0112] Where a = Wx + b, where a = [a1, a2] contains the resulting denoised image, and W and b are the parameters of the linear denoising. Therefore, the LMMSE solution is:

[0113]

[0114] where ∑ x is the covariance matrix of the noisy FBP result, x and ∑ ax is the cross covariance matrix between the noisy image and the clean image. Here, we let W and b denote the general matrix and vector used in the linear transformation a = Wx + b, where and represents a specific instance of such matrices and vectors obtained, for example, by processing spectral image data through a neural network.

[0115] Despite the discovery and This may seem simple at first, but you may run into several problems. The first is the matrix dimensionality, which is not feasible due to the very high dimensional images we are dealing with. Therefore, the cross-covariance analysis and covariance analysis will have to be restricted to the relationships between a finite number of pixels. The simplest case is to assume only the diagonals of the cross-covariance matrix and the covariance matrix, which will be computationally simple but an oversimplification and a very biased approximation. However, we will use this case as a starting point for our deep learning approach. The second problem is that these matrices as well as the means a and x are initially unknown and need to be estimated with a sufficient amount of observed data. We will perform these estimates using our training data as sample cross-covariance and covariance and sample mean.

[0116] Let's explain how we think about model-based deep learning in this scenario. We have a model-based solution (LMMSE denoiser) that we need to enhance so that we get good estimates of W and b when we only use diagonal cross-covariance and cross-covariance. Therefore, we want to preserve the mathematical structure (linear, fast) with deep learning inference (to leverage a powerful statistical model agnostic approach to estimate the LMMSE parameters). Implementing the problem structure, our goal is a neural network that requires very few training samples and is more robust to unseen datasets than a typical "black-box" network. Of course, we also expect the result to be much better than considering diagonal cross-covariance and covariance in the oversimplified LMMSE. We have already discussed this in detail. Fig.10 Our proposed deep learning solution is shown in Figure .

[0117] An additional explanation can be given to the results. The goal of the network is to obtain W and b, not to denoise the image. Therefore, if one wants to manipulate and understand the solution, instead of changing or accessing the millions of parameters within the CNN, one can consider the parameters in W and b, which are significantly fewer and also more "interpretable" (related to LMMSE).

[0118] The proposed deep learning method requires a training database. To show the proof of concept of the proposed disclosure, we have trained a learned LMMSE estimator with simulated photon counting data. We have simulated a set of 1200 cases where PCCT measurements were calculated using an eight-bin silicon detector and then dual material decomposition and FBP were performed to obtain material images. We have used the KiTS19 database consisting mainly of abdominal scans. 1000 samples were used for training and 200 for testing. To evaluate the robustness of the technique to unseen data, we have also simulated 200 additional scans from a different database (NSCLS) that also contains whole body scans and therefore contains more variability in anatomical structures than the training database.

[0119] In this example, the neural network has been trained using PyTorch and an NVIDIA GPU GeForce RTX 2080Ti GPU board. To perform a comparative study, we consider the following competing solutions: (1) the original simplistic LMMSE as described in the previous section; and (2) a “black-box” CNN based on the UNet architecture.

[0120] Fig. 9 A schematic diagram of a conventional CNN-based denoising technique 90 is shown, where a black-box CNN 94 maps directly from a noisy CT image 92 to a clean CT image 96 .

[0121] Fig.10 A schematic diagram of the proposed denoising technique 1000 is shown, wherein a CNN 1004 accepts a noisy CT image 1002 as input, the CNN finds a linear estimator 104, the CNN is used to map the linear model parameters W and b, and the linear estimator is used to perform denoising 1006 and generate a clean CT image. Thus, a linear structure is implemented in the learning process.

[0122] Fig.11 An example of its interpretability is shown: the learned linear components W and b can be manipulated with only a few parameters. Three parameters σ, β, and λ are used to control w ii (the “variance” of a single material component of a linear model), w ij (the "cross covariance" of the opposite material components) and b (the mean or "deviation" of a single material component). When the deviation is zero (λ=0) 1010, an enhanced structure is shown in the results. If the "cross covariance" w ij is zero (β=0) 1012, cross contamination between materials may be less corrected. Additionally, if the "variance" component is zero (σ=0) 1014, it may be a single material noise variance that is not reduced.

[0123] Fig.12An example of the interpretation of the results provided by the two components in a=Wx+b is shown. The first term (Wx) provides details about the structure of the anatomical structure (finer edges, local small structures), as shown in CT image 1016 (structural image), but does not give the exact CT number expressed in Hounsfield units (HU). The second independent term b generally corrects the HU value of the result (mean or deviation), as shown in CT image 1018 (deviation image) with very little details of the anatomical structure (very smooth image).

[0124] Fig.13 An empirical demonstration of the parallelism of this technique and LMMSE is shown: the first row 1020 represents the estimator components of the conventional LMMSE for a specific phantom example, and the second row 1030 represents the estimator components equivalent to our learned method. It can be seen that LMMSE is not specific enough and the results are too vague and do not accurately represent the denoised example. However, our method is more specific and accurate, but retains the similarity with the mentioned LMMSE. Therefore, it can be interpreted as a DNN-enhanced LMMSE (which is the best linear estimator that minimizes MSE).

[0125] The present disclosure relates to spectrally or energy-resolved image data consisting of image data comprising at least two spectral components. In this case, the image data may be, for example, two-dimensional, three-dimensional or time-resolved and refers to a reconstructed image or an intermediate representation of image data such as a sinogram. The different spectral components may be, for example, synthetic monoenergetic images, wide spectrum images acquired at different tube acceleration voltages, or material selective images such as basis images. The different spectral components may also be a combination of the above images.

[0126] The above description should be understood as exemplary and non-limiting, and several variations of the method can be envisioned. For example, several different architectures of convolutional neural networks are possible, such as UNet, ResNet, or unfolding iterative networks, such as unfolding gradient descent networks or unfolding original dual networks. In addition, it may or may not be desirable to include batch normalization, jump connections, and in network training, and different pooling layers such as maximum pooling, average pooling, or softmax pooling may be included in the network. Different loss functions such as L1 loss, L2 loss, perceptual loss, and adversarial loss can be minimized when training the network. Different discriminator networks can be used to implement perceptual loss, and different layers of such networks can be used to obtain different image characteristics.

[0127] The inventors have realized that it is impractical to have W as a full matrix and use a neural network to obtain all its elements, as this would require having approximately 10 12outputs of a neural network. It is therefore desirable to impose some structure on the matrix, for example by letting W be a sparse matrix, i.e. a matrix with a small number of non-zero elements. For example, the matrix W may be a diagonal matrix, in which case each pixel value in the image dataset will be multiplied by the value by which the matrix is ​​applied. Another option is to let W be block diagonal. For example, if the spectral data consists of N spectral components, then W may, for example, consist of blocks of N×N elements along its diagonals, such that applying W to the vectors causes the values ​​corresponding to different spectral components in one particular pixel to be transformed by the N×N block to a new set of spectral components in the corresponding transformed set of spectral components in the image.

[0128] Another example is to let W act on each of the different spectral components separately, with entries corresponding to crosstalk between the different components set to zero. In both the case where W acts on each of the different spectral components separately and in the more general case where W includes corresponding cross-component entries, they can be considered to correspond to those of a certain maximum distance in pixels between the input pixel and the output pixel. Alternatively, W can be expressed as a transform W=F in the Fourier domain -1 W F F, where F is the Fourier transform operator and only W F The elements of corresponding to specific frequencies (such as low frequency or high frequency) are non-zero.

[0129] Other examples include letting W be an element of a range of linear transformations or non-linear transformations, such as, for example, an artificial neural network or a convolutional neural network.

[0130] Vector b may also be selected as, for example, a full vector without any restrictions or a sparse vector in which only certain elements are non-zero. In another exemplary embodiment of the present disclosure, b may be restricted to the range of linear transformation or nonlinear transformation, such as an artificial neural network or a convolutional neural network, or restricted to a linear combination of Fourier components b=F -1 b F , where b F is a vector of Fourier components of b, which may, for example, be restricted to contain high or low spatial frequencies.

[0131] In implementation, such restrictions on W and b can be imposed by having the convolutional neural network output only those elements of W and b that should be non-zero and setting the other components to zero. In another embodiment of the present disclosure, the convolutional neural network can generate a feature vector that is then transformed into W and b, for example, by a linear transformation or by an artificial neural network or a convolutional neural network.

[0132] For example, multiple Fourier components can be generated by a neural network and transformed into one or more diagonals forming, for example, b or W. In another embodiment of the present disclosure, different components of b and / or W or eigenvectors associated with b and / or W are given different weights in the loss function used to train the neural network to generate these components, or are penalized by a penalty term so that these components are unlikely to obtain values ​​of a large number of values. For example, the components of b and / or W can be regularized in a way that high spatial frequencies are penalized, which means that these components will mainly contain low frequencies. In this way, it is possible to avoid too large changes between the transformations applied to adjacent pixels, so that the denoising method is more robust to the differences in noise characteristics and image appearance compared to the training data set. For example, low frequencies can be penalized, thereby providing a denoiser that is particularly suitable for retaining fine details.

[0133] The inventors have recognized that the linear structure of the denoiser can provide both interpretability and tunability. The learned LMMSE denoiser is similar in its mathematical structure to a conventional LMMSE denoiser based on a hand-crafted noise model. By comparing the coefficients of the learned LMMSE denoiser with the coefficients of a conventional LMMSE denoiser, information about how the denoiser acts on an image can be obtained. For example, such a comparison can show that the action of the learned LMMSE denoiser in a limited region of an image is similar to a conventional LMMSE denoiser built based on a specific model of the signal and noise. This information can prove useful when seeking to analyze image quality and robustness properties and improve the learned LMMSE denoiser, for example by adjusting the structure or training parameters of b and / or W.

[0134] The structure of the linear LMMSE denoiser also provides tunability to the model. For example, one can adjust and / or To obtain an image with the desired properties, you can adjust the individual entries or groups of entries in and / or The entries that control the pixel values ​​in a specific area of ​​the image are adapted to the image properties in a specific area of ​​interest. For example, you can adjust The value of the diagonal or the block diagonal The values ​​of the diagonal blocks of to obtain specific image properties.

[0135] Such coefficient manipulation can be performed by multiplying a selected set of coefficients by a constant factor. Alternatively, it can be performed by and the unit transformation corresponding to setting W equal to the unit matrix and b to zero. In this way, a new learned linear transformation a = With the previous transformation In contrast, the selected components are more similar to unit transformations.

[0136] By way of example, the inventors have recognized that tends to be related to the structure of the image, where b is related to large area deviations. and The relative weights of , so that the desired trade-off between structure and bias can be obtained. This can be done, for example, by or Multiplies the selected elements of with a scalar value and or This is accomplished by multiplying the selected elements of by another scalar value. For example, You can multiply by values ​​between 0 and 1 to enhance the representation of structures in the image while accepting higher bias. For example, Can be multiplied by a value between 0 and 1 to reduce image bias in cases where detailed structure is less important.

[0137] In another embodiment of the present disclosure, by training a single neural network based on the tuning parameter t to generate the matrix and vector Tunability is achieved by using a family of t such that a varying t gives an image a=Wx+b with different properties. For example, images with different resolutions or deviation properties can be obtained. As another example, different t can give images with different noise textures. This can be achieved by using a training dataset, where each training sample consists of a spectral input image dataset and multiple spectral output image datasets. The loss function used to train the neural network can then incorporate a term for each output image dataset, thereby penalizing the difference between the network output for different values ​​of t and each of the output image datasets.

[0138] As another example, t may be replaced by multiple tuning parameters that allow tuning of several different properties of the image.

[0139] In another embodiment of the present disclosure, tunability may be achieved in real time while the image is displayed to the end user, allowing the user to adjust the image to obtain desired image attributes.

[0140] In an exemplary embodiment of the present disclosure, a convolutional neural network is trained by minimizing an L1 loss function, an L2 loss function, a perceptual loss function, an adversarial loss function, or a combination of these loss functions.

[0141] The goal of the network is to obtain W and b such that a = Wx + b, where the denoised image a = [a1, a2] corresponds to the material image x = [x1, x2]. Although the examples here are described for the case of two spectral components, this is a non-limiting example and the vectors a and x can generally have any number of components greater than or equal to two. This goal is achieved by using the L2 loss function Train the network to achieve and and is the output from the network. You can also use L1 loss The L2 and L1 losses are pixel-wise loss functions that are known to result in over-smoothing and loss of fine details that may be important to the perceived quality and clinical usefulness of the resulting images.

[0142] One possible solution is to use a feature-based perceptual loss, which does not compare the output and the truth pixel by pixel, but compares the feature representations corresponding to the output and the truth. The feature representation is obtained by passing the target and the output through a pre-trained convolutional neural network (CNN). For example, VGG16 / 19 (CNN from the Visual Geometry Group of the University of Oxford) is often used as a feature extractor. Perceptual loss has been used in a variety of computer vision problems, such as image denoising and super-resolution. Let φ j represents the jth layer of the pre-trained CNN, then the perceptual loss is defined as

[0143] Another possibility is to minimize some notion of the distance between the distribution of true and output images. This can be done using an adversarial loss based on Generative Adversarial Networks (GANs). In this setting, we pit the network against another CNN in a minimax game that, through successive improvements, will encourage the distribution of the outputs to be indistinguishable from the true distribution. This prevents over-noising and over-smoothing associated with pixel-wise losses such as the L2 and L1 losses. Let As the distribution of real material images, as the distribution of noisy material images, and As by An implicitly defined distribution where and is the output G from the network. Let’s denote D against the discriminator by the network that G is adversarial against. The job of the discriminator is to discriminate (classify) between the real output and the generated output. The original version of GAN solves the following minimax game:

[0144]

[0145] For the best discriminator, the purpose of the generator is equivalent to minimizing and . Despite being able to produce amazing results, GANs are notoriously difficult to train. One version that mitigates the common problems of vanishing gradients and mode collapse is the Wasserstein GAN with Gradient Penalty (WGAN-GP). WGAN-GP strives to minimize and The Bulldozer distance or Wasserstein distance between , rather than the Jensen-Shannon divergence. The discriminator is now called the critic, which we denote as C,, because it now outputs any real number, rather than numbers in [0,1], and thus no longer discriminates. The minimax game is:

[0146]

[0147] The last term is a gradient penalty that softly enforces that the critic is 1-Lipschitz continuous, and is through (where ∈~U[0,1]) implicitly defines the distribution. Using a and This linear combination of , rather than checking the gradient anywhere, which would be intractable. The 1-Lipschitz continuity condition on the critic is necessary to obtain a tractable version of the Wasserstein distance. In contrast to standard GANs that take random inputs to produce some realistic but random outputs, this setting can be used to train the learned LMMSE by sampling a pair of noisy material images x instead of random noise vectors (as is usually done in GANs). Furthermore, the adversarial loss can be advantageously combined with a reconstruction loss such as the perceptual loss.

[0148] WGAN-GP is not necessarily the best performing GAN, however it is one of the most stable GANs to train. Previous publications have demonstrated the stability of WGAN-GP for several different tasks and datasets without experiencing the common problems of vanishing gradients and mode collapse.

[0149] To weigh the pros and cons of these loss functions, one can consider a weighted sum of the previously mentioned loss functions.

[0150] In an exemplary embodiment of the present disclosure, a convolutional neural network is trained as part of a pair of cycle-consistent generative adversarial networks.

[0151] The data required for the present disclosure are paired samples of noisy material images and their true (low-noise) counterparts. However, in many cases, such paired datasets are not available. Instead, we may have a bunch of noisy material images and a bunch of denoised / low-noise material images. In order to extend the learned LMMSE to unpaired data, a so-called cycle-consistent GAN can be applied. The key insight that makes this possible is the cycle-consistent loss. The goal is to find a mapping from a source domain X to a target domain A. Let G:X→A be a mapping that takes a pair of noisy material images x, passes them through our network and forms a denoised material image a=Wx+b. Using an adversarial loss, we can push the distribution induced by G(X) so that it is indistinguishable from the distribution of A. However, this mapping is highly underconstrained and the space of possible mappings is huge. To reduce the space of possible mappings, one can consider an inverse mapping F:A→X and enforce cycle consistency via a cycle-consistent loss. If and Our mapping is cycle consistent. Cycle consistency can be enforced via a cycle consistent loss:

[0152]

[0153] This is combined with a GAN for mapping G and inverse mapping F, each with their own discriminator D A and D X ,. Therefore, we have the following goals:

[0154]

[0155] And for is similar. Putting it all together gives us the minimax game:

[0156]

[0157] As with the original GAN, this formulation will have training stability issues. To circumvent this, the negative log-likelihood loss is replaced by the L2 loss. In other words, the generator is trained to minimize And the discriminator is trained to minimize Furthermore, the output history (e.g., 50) is used to update the discriminator.

[0158] The method proposed by the inventors comprises the following steps: (1) acquiring energy-resolved CT image data; (2) processing the energy-resolved CT image data based on at least one convolutional neural network so as to obtain a matrix and vector and (3) based on the matrix W, the denoised energy-resolved CT image data a is formed into a linear denoiser Where x is a representation of spectral CT image data comprising at least two spectral components.

[0159] In an exemplary embodiment of the present disclosure, adjusting or to improve at least one element of the image quality metric.

[0160] In an exemplary embodiment of the present disclosure, the measure of image quality is mean square error, structural similarity, deviation, fidelity of fine detail, numerical observer detectability, visual grading score, or observer performance.

[0161] In an exemplary embodiment of the present disclosure, the matrix is a diagonal matrix.

[0162] In another exemplary embodiment of the present disclosure, the matrix is a block diagonal matrix where its nonzero off-diagonal entries correspond to the cross terms between the spectral components in each pixel.

[0163] In another exemplary embodiment of the present disclosure, the matrix is a sparse matrix where the non-zero elements correspond to pixels that are located near each other.

[0164] In an exemplary embodiment of the present disclosure, the convolutional neural network has a ResNet architecture, a UNet architecture, an unfolded iterative architecture, or a combination of these architectures.

[0165] In an exemplary embodiment of the present disclosure, a convolutional neural network is trained by minimizing an L1 loss function, an L2 loss function, a perceptual loss function, an adversarial loss function, or a combination of these loss functions.

[0166] In an exemplary embodiment of the present disclosure, a convolutional neural network is trained as a generator in a generative adversarial network.

[0167] In an exemplary embodiment of the present disclosure, a convolutional neural network is trained as part of a pair of cycle-consistent generative adversarial networks.

[0168] In an exemplary embodiment of the present disclosure, the energy-resolved image data x is a set of sinusoidal patterns.

[0169] In another exemplary embodiment of the present disclosure, the energy-resolved image data x is a set of reconstructed images.

[0170] In an exemplary embodiment of the present disclosure, different components of the energy-resolved image data x contain monoenergetic image data of different monochromatic energies, or image data corresponding to different measured energy levels or energy bins, or different basis images.

[0171] In an exemplary embodiment of the present disclosure, the end user is given an adjustment matrix and vector The possibility of weight.

[0172] In another exemplary embodiment of the present disclosure, a convolutional neural network is trained based on a data set comprising a plurality of low noise images having different image characteristics for each high noise image, and the neural network is trained to generate a low noise image having different characteristics for each setting of at least one tuning parameter.

[0173] Fig.14 2 is a schematic diagram illustrating an example of a computer-specific implementation according to an embodiment. In this particular example, the system 200 includes a processor 210 and a memory 220, the memory including instructions that can be executed by the processor, whereby the processor is operable to perform the steps and / or actions described herein. The instructions are typically organized as a computer program 225, 235, which may be pre-configured in the memory 220 or downloaded from an external memory device 230. Optionally, the system 200 includes an input / output interface 240, which may be interconnected to the processor 210 and / or the memory 220 to enable input and / or output of relevant data (such as input parameters and / or resulting output parameters).

[0174] The term "processor" should be interpreted in a general sense as any system or device capable of executing program code or computer program instructions to perform specific processing, determination or computing tasks.

[0175] Thus, processing circuitry including one or more processors is configured to perform well-defined processing tasks (such as those described herein) when executing a computer program.

[0176] The processing circuitry need not be dedicated only to executing the steps, functions, processes and / or blocks described above, but may also perform other tasks.

[0177] The proposed technology also provides a computer program product comprising a computer readable medium 220, 230 having such a computer program stored thereon.

[0178] By way of example, the software or computer program 225, 235 may be implemented as a computer program product, which is typically carried or stored on a computer readable medium 220, 230, specifically a non-volatile medium. The computer readable medium may include one or more removable or non-removable memory devices, including but not limited to: read-only memory (ROM), random access memory (RAM), compact disc (CD), digital versatile disc (DVD), Blu-ray disc, universal serial bus (USB) memory, hard disk drive (HDD) storage device, flash memory, magnetic tape, or any other conventional memory device. Thus, the computer program may be loaded into the operating memory of a computer or equivalent processing device for execution by its processing circuit system.

[0179] The computer program resident in the memory may therefore be organized into appropriate functional modules that are configured to perform at least part of the steps and / or tasks described herein when the computer program is executed by a processor.

[0180] As mentioned, at least some of the steps, functions, processes and / or blocks described herein may be implemented in software, such as a computer program, for execution by appropriate processing circuitry, such as one or more processors or processing units.

[0181] The method flow, when executed by one or more processors, can be regarded as a computer action flow. The corresponding device, system and / or apparatus can be defined as a set of functional modules, wherein each step executed by the processor corresponds to a functional module. In this case, the functional module is implemented as a computer program running on the processor. Therefore, the device, system and / or apparatus can alternatively be defined as a set of functional modules, wherein these functional modules are implemented as computer programs running on at least one processor.

[0182] The computer program resident in the memory may therefore be organized into appropriate functional modules that are configured to perform at least part of the steps and / or tasks described herein when the computer program is executed by a processor.

[0183] Alternatively, these modules may be implemented primarily through hardware modules, or alternatively through hardware. The degree of software versus hardware is purely a choice of the particular implementation.

[0184] As used herein, the element or step that is listed in the singular and begins with the word "one" or "a kind of" should be understood as not excluding multiple elements or steps, unless such exclusion is explicitly stated. In addition, the reference to "an embodiment" of the present disclosure is not intended to be interpreted as excluding the existence of additional embodiments that also include the described features. In addition, unless explicitly stated to the contrary, the embodiment of "including", "including" or "having" an element or multiple elements with a specific characteristic may include additional such elements that do not have the characteristic. The terms "including" and "in..." are used as the concise language equivalents of the corresponding terms "including" and "wherein". In addition, the terms "first", "second" and "third" etc. are only used as marks, and are not intended to impose numerical requirements or specific position order on their objects.

[0185] The embodiments of the present disclosure shown in the drawings and described above are exemplary embodiments only and are not intended to limit the scope of the appended claims, including any equivalents included within the scope of the claims. Those skilled in the art will appreciate that various modifications, combinations and changes may be made to these embodiments without departing from the scope of the invention as defined by the appended claims. It is intended that any combination of non-mutually exclusive features described herein is within the scope of the invention. That is, the features of the embodiments may be combined with any appropriate aspect described above, and the optional features of any one aspect may be combined with any other appropriate aspect. Similarly, the features listed in the dependent claims may be combined with non-mutually exclusive features of other dependent claims, particularly where the dependent claims are dependent on the same independent claim. In some jurisdictions that require single claim dependencies, these dependencies may have been used as practice, but this should not be taken to mean that the features in the dependent claims are mutually exclusive.

[0186] It should also be noted that, unless explicitly stated otherwise, the concept of the present invention relates to all possible feature combinations. In particular, where technically possible, different partial solutions in different embodiments can be combined in other configurations.

Claims

1. A method for denoising spectral CT image data, the method comprising: A denoised linear estimate of the spectral CT image data is determined by maximizing or minimizing a first objective function, wherein at least one parameter of the denoised linear estimate is determined by at least one machine learning system.

2. The method of claim 1 , wherein determining the denoised linear estimate of the spectral CT image data comprises: receiving spectral CT image data; Processing the spectral CT image data based on the at least one machine learning system so as to obtain a matrix W and a vector b; as well as De-noised spectral CT image data a are formed according to said linear estimation as a=Wx+b, wherein x is a representation of the spectral CT image data comprising at least two spectral components.

3. The method of claim 2, wherein at least one of the matrix W and the vector b is adjustable for optimizing at least one image quality indicator of the CT image data by maximizing or minimizing the first objective function.

4. The method of claim 2, wherein the first objective function is at least one of mean square error, structural similarity, bias, fidelity of fine details, numerical observer detectability, visual grading score, and observer performance. The method according to claim 2 , wherein the matrix W is a diagonal matrix.

6. The method of claim 2, wherein the matrix W is a block diagonal matrix, and the non-zero off-diagonal entries of the matrix W correspond to cross terms between the at least two spectral components in each pixel of the spectral CT image data. 7 . The method of claim 2 , wherein the matrix W is a sparse matrix, and non-zero elements of the matrix W correspond to pixels of the spectral CT image data that are located near each other.

8. The method of claim 2, wherein the at least one machine learning system is trained by minimizing at least one of an L1 loss function, an L2 loss function, a perceptual loss function, and an adversarial loss function.

9. The method of claim 2, wherein the spectral CT image data x comprises at least one of a set of sinusoidal graphs and a set of reconstructed CT images.

10. The method of claim 2, wherein the at least two spectral components of the spectral CT image data x comprise at least one of monoenergetic image data of different monochromatic energies, image data corresponding to different measured energy levels or energy bins, and different basis images.

11. The method of claim 2, wherein at least one of the matrix W and the vector b of the denoised spectral CT image data a is adjusted by an end user.

12. The method of claim 2, wherein the at least one machine learning system comprises at least one convolutional neural network (CNN).

13. The method of claim 12, wherein the at least one CNN comprises at least one of a ResNet architecture, a UNet architecture, and an unfolding iterative architecture.

14. The method of claim 12, wherein the at least one convolutional neural network is trained as a generator in a generative adversarial network (GAN).

15. The method of claim 12, wherein the at least one convolutional neural network is trained as part of a pair of cycle-consistent generative adversarial networks (GANs).

16. The method of claim 12, wherein the at least one convolutional neural network is trained based on a data set comprising a plurality of low noise images, the plurality of low noise images having different image characteristics for each high noise image and trained to generate a low noise image having different characteristics for each setting of at least one tuning parameter.

17. A CT imaging system, comprising: An X-ray source configured to emit X-rays; an X-ray detector configured to generate spectral CT image data; and A processor, the processor being configured to: determining a denoised linear estimate of the generated spectral CT image data based on maximizing or minimizing a first objective function; The processor is further configured to determine at least one parameter of the linear estimate by at least one machine learning system.

18. The CT imaging system of claim 17, wherein the processor is configured to: Processing the spectral CT image data based on the at least one machine learning system so as to obtain a matrix W and a vector b; and De-noised spectral CT image data a are formed according to said linear estimation as a=Wx+b, wherein x is a representation of the spectral CT image data comprising at least two spectral components.

19. The CT imaging system of claim 18, wherein at least one of the matrix W and the vector b is adjustable so as to optimize at least one image quality indicator of the CT image data based on maximizing or minimizing a second objective function.

20. The CT imaging system of claim 18, wherein the at least one second objective function is at least one of mean square error, structural similarity, bias, fidelity of fine detail, numerical observer detectability, visual grading score, and observer performance.

21. The CT imaging system of claim 18, wherein the matrix W is a diagonal matrix.

22. The CT imaging system of claim 18, wherein the spectral CT image data comprises at least one of a set of sinusoidal graphs and a set of reconstructed images.

23. The CT imaging system of claim 18, wherein at least one of the matrix W and the vector b of the denoised spectral CT image data a is adjustable by an end user.

Images