Determination of Reliability Display of Deep Learning Image Reconstruction in Computed Tomography

Abstract

A method and system for determining one or more confidence indications for machine learning image reconstruction in computed tomography (CT) is provided. The method comprises (S1) acquiring energy resolved X-ray data and (S2) processing the energy resolved X-ray data based on at least one machine learning system to generate a representation of a posterior probability distribution of at least one reconstructed basis image or image features thereof. The method further includes (S3) generating one or more confidence indications for the at least one reconstructed basis image, or at least one derived image derived from the at least one reconstructed basis image, or image features of the at least one reconstructed basis image or the at least one derived image based on the representation of the posterior probability distribution.

Description

Technical Field

[0001] The proposed technology relates to X-ray technology, X-ray imaging, and corresponding imaging reconstruction and imaging tasks. In particular, the proposed technology relates to methods and systems for determining confidence indications for deep learning image reconstruction in computed tomography (CT), methods and systems for generating uncertainty maps for deep learning image reconstruction in spectral CT, and corresponding image reconstruction systems and X-ray imaging systems, as well as related computer programs and computer program products.

Background Art

[0002] Radiation images such as X-ray images have been used for many years in medical applications and non-destructive inspections.

[0003] Typically, an X-ray imaging system includes an X-ray source and an X-ray detector array including a plurality of detectors including one or a number of detector elements (independent means for measuring X-ray intensity / fluence). The X-ray source emits X-rays, which are registered by the detector array after passing through the subject or object to be imaged. Since substances that absorb X-rays are more abundant than other substances, an image of the subject or object is formed.

[0004] The problem of an X-ray image detector is to extract the maximum amount of information from the detected X-rays and input it into the image of the object or subject.

[0005] In a general medical X-ray image system, X-rays are generated by an X-ray tube. The energy spectrum of a general medical X-ray tube is wide, ranging from zero to 180 keV. Therefore, the detector usually detects X-rays of various energies.

[0006] Referring to FIG. 1, it will be useful to briefly describe the overview of an exemplary overall X-ray imaging system. In this exemplary but non-limiting embodiment, the X-ray imaging system 100 basically consists of an X-ray source 10, an X-ray detector system 20, and a related image processing system or device 30. Generally, the X-ray detector system 20 is configured to register the radiation from the X-ray source 10, optionally focused by an X-ray optical system, and register the radiation that has passed through an object, subject, or a part thereof. The X-ray detector system 20 can be connected to the image processing system 30 via appropriate analog and readout electronics that are at least partially integrated into the X-ray detector system 20 to enable image processing and / or image reconstruction by the image processing system 30.

[0007] As an example, an X-ray computed tomography (CT) system includes an X-ray source and an X-ray detector arranged such that projection images of a subject or object can be acquired at different view angles that cover at least 180 degrees. This is most commonly achieved by mounting the source and detector on a support that can rotate around the subject or object. An image containing projections registered on different detector elements for different view angles is called a sinogram. Hereinafter, even if the detector is two-dimensional, a set of projections registered on different detector elements for different view angles will be called a sinogram, and the sinogram is used to form a three-dimensional image.

[0008] A further development of X-ray imaging is energy-resolved X-ray imaging, also known as spectral X-ray imaging. This can be achieved by rapidly switching the radiation source between two different emission spectra, using two or more X-ray sources that emit different X-ray spectra, or, more notably, by using an energy-discriminating detector that measures the incident radiation at two or more energy levels. An example of such a detector is a multi-bin photon-counting detector, in which each registered photon generates a current pulse that is compared with a set of thresholds, thereby counting the number of photons incident on each of a number of energy bins.

[0009] In spectral X-ray projection measurements, projection images at each energy level are typically obtained. As described by Tapsovaara and Wagner, “SNR and DQE analysis of broad-spectrum X-ray Imaging”, Phys. Med. Biol. 30, 519, these weighted sums can be created to optimize the contrast-to-noise ratio (CNR) for a given imaging task.

[0010] Another technique enabled by energy-resolved X-ray imaging is basis material decomposition. This approach exploits the fact that all materials composed of low atomic number elements, such as human tissue, have a linear attenuation coefficient μ(E) that can be approximately represented as a linear combination of two basis functions that are energy-dependent.

Number

[0011] Here, f1 and f2 are basis functions, and a1 and a2 are the corresponding basis coefficients. More generally, f1 is a basis function and a1 is the corresponding basis coefficient. If there is one or more elements with a sufficiently high atomic number such that the imaging volume has a K-absorption edge in the energy range used for imaging, then one basis function must be added for each such element. In the field of medical imaging, such K-absorption edge elements can typically be iodine or gadolinium, substances used as contrast agents.

[0012] Regarding basis material decomposition, it is described in Alvarez and Macovski, "Energy-selective reconstructions in X-ray computerised tomography", Phys. Med. Biol. 21, 733. In basis material decomposition, for i = 1…N (N is the number of basis functions), the integral of each basis coefficient Ai = ∫ l a i dl is estimated from the measurement data on each projection line l from the radiation source to the detector element. In one implementation, this is first achieved by expressing the expected registered counts in each energy bin as a function of A i .

Number

[0013] Here, λ i is the count expected in energy bin i, E is the energy, S i is the spectral shape incident on the subject, the quantum efficiency of the detector, and a response function that depends on the sensitivity of energy bin i to X-rays with energy E. The term "energy bin" is most commonly used for photon counting detectors, but this equation can also represent other energy-resolved X-ray systems such as multi-layer detectors or kVp switching radiation sources.

[0014] Next, under the assumption that the counts in each bin are Poisson-distributed random variables, Ai can be estimated using the maximum likelihood method. This is achieved by minimizing the negative log-likelihood function. See Roessl and Proksa, "K-edge imaging in X-ray computed tomography using multi-bin photon counting detectors", Phys. Med. Biol. 52(2007), 4679-4696. [Number] Here, m i is the measured count in energy bin i, and M b is the number of energy bins.

[0015] As a result, when the estimated basis coefficient line integral Ŵi (Ai with an upper ring) for each projection line is placed in the image matrix, a material-specific projection image (also called a basis image) for each basis i is obtained. This basis image can be viewed directly (such as in projection X-ray imaging), or it can be taken as an input to a reconstruction algorithm that forms a map of the basis coefficient a i inside the object (such as in CT). In any case, the result of basis decomposition can be regarded as one or more basis image representations such as the line integral of the basis coefficient or the basis coefficient itself.

[0016] The map of the basis coefficient a i inside the object is called a basis material image, a basis image, a material image, a material-specific image, a material map or a basis map.

[0017] However, a well-known limitation of this and other methods is that the variance of the estimated line integrals typically increases with the number of bases used in the basis decomposition. In particular, this results in an unfortunate trade-off between improving the quantification of tissue and increasing image noise.

[0018] Furthermore, it is actually difficult to perform accurate basis decomposition using two or more basis functions, and artifacts, bias, or excessive noise may occur. Also, such basis decomposition may require large-scale calibration measurements and data preprocessing to obtain accurate results.

[0019] Since many image reconstruction tasks are inherently complex, machine learning such as artificial intelligence (AI) and deep learning has begun to be used in general image reconstruction and satisfactory results have been obtained. However, the current problem in image reconstruction by machine learning such as deep learning is its low explainability. Even if an image appears to have a very low noise level at first glance, it actually contains errors due to the bias of the neural network estimator.

[0020] Therefore, there is a need to improve reliability and / or explainability in machine learning image reconstruction such as deep learning image reconstruction for computed tomography (CT).

Prior Art Documents

Patent Documents

[0021]

Non-Patent Document 1

Non-Patent Document 2

[0022] Generally, it is desirable to provide improvements related to image reconstruction for X-ray image applications.

[0023] An object of the present invention is to provide a method for determining a reliability display of machine learning image reconstruction such as deep learning image reconstruction in computed tomography (CT).

[0024] The present invention aims to provide a method for generating an uncertainty map for machine learning image reconstruction such as deep learning image reconstruction in spectral CT.

[0025] Another object is to provide a system for determining a reliability display of machine learning image reconstruction such as deep learning image reconstruction in computed tomography (CT).

[0026] Another object is to provide a system for generating an uncertainty map for machine learning image reconstruction such as deep learning image reconstruction in spectral CT.

[0027] Yet another object is to provide a corresponding image reconstruction system.

[0028] Yet another object is to provide an overall X-ray image system.

[0029] It is also an object to provide a corresponding computer program and a computer program product.

[0030] These and other objects can be achieved by one or more embodiments of the proposed technology.

[0031] The inventors have realized that in order to make the images obtained from machine learning image reconstruction such as deep learning image reconstruction reliable, it is highly desirable to quantify the degree of confidence in the reconstructed image (values), or otherwise determine an indication or representation of confidence. This is particularly important in photon counting spectral CT, where, although it is theoretically possible to generate a quantitatively accurate map of the material composition, the noise level is high, especially in three-basis decomposition, and machine learning image reconstruction such as deep learning reconstruction methods may have to be used as an important component of the image reconstruction chain.

[0032] The basic idea of the present invention is to provide a radiologist with an indication of confidence such as an uncertainty map or a reliability map for each image generated by machine learning image reconstruction such as deep learning image reconstruction.

[0033] A set of training data, e.g., a measured energy-resolved X-ray data set, and a corresponding set of ground truth or reconstructed basis material maps specially selected for training a machine learning system such as a neural network can be used to specify or approximate the probability distribution of one or more reconstructed basis material images. Such a distribution prior to the new measurements being evaluated is called a prior distribution. When one or more measurements of the representation of the X-ray image data are further performed, the probability distribution of the possible basis material images due to this additional knowledge of the measurement is known as the posterior probability distribution.

[0034] According to a first aspect, a method is provided for determining one or more confidence indications for machine learning image reconstruction in computed tomography (CT). The method comprises obtaining energy-resolved X-ray data, processing the energy-resolved X-ray data based on at least one machine learning system to generate a representation of the posterior probability distribution of at least one reconstructed basis image or image features thereof, and generating one or more confidence indications for at least one reconstructed basis image, or at least one derived image derived from the at least one reconstructed basis image, or image features of the at least one reconstructed basis image or the at least one derived image based on the representation of the posterior probability distribution.

[0035] As an example, the confidence indication can include one or more uncertainty maps or reliability maps. Such uncertainty maps or reliability maps may be presented with the relevant images or image features in various ways to provide additional useful information to a radiologist.

[0036] According to a second aspect, a system for determining one or more confidence displays for machine learning image reconstruction in computed tomography (CT) is provided. The system is configured to acquire energy-resolved X-ray data. The system is further configured to process the energy-resolved X-ray data based on at least one machine learning system to obtain a representation of a posterior probability distribution of at least one reconstructed basis image or its image features. The system is also configured to generate one or more confidence displays for the at least one reconstructed basis image, or for at least one derived image derived from the at least one reconstructed basis image, or for the image features of the at least one reconstructed basis image or the at least one derived image, based on the representation of the posterior probability distribution.

[0037] According to a third aspect, a corresponding image reconstruction system including such a system for determining confidence displays is provided.

[0038] According to a fourth aspect, an overall X-ray imaging system including such an image reconstruction system is provided.

[0039] According to a fifth aspect, a corresponding computer program and computer program product are provided.

[0040] In this way, it may be possible to improve the reliability and / or explainability in machine learning image reconstruction for computed tomography (CT).

[0041] This embodiment will be best understood by reference to the following description in conjunction with the accompanying drawings, with its further objects and advantages.

Brief Description of the Drawings

[0042]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Best Mode for Carrying Out the Invention

[0043] For better understanding, it will be useful to continue with an introductory explanation regarding a non-limiting example of an overall X-ray image processing system.

[0044] Figure 2 is a schematic diagram showing an example of an X-ray imaging system 100 composed of an X-ray source 10 that irradiates X-rays and an X-ray detector system 20 having an X-ray detector that detects X-rays. Figure 2 is a schematic diagram showing an example of an X-ray imaging system 100 including an X-ray source 10 that emits X-rays, an X-ray detector system 20 that detects X-rays after passing through an object, and an analog processing circuit 25 that processes and digitizes raw electrical signals from the X-ray detector, and a digital processing circuit 40 that can perform further processing operations on the measured data, such as applying corrections, temporary storage, or filtering, and a computer 50 that stores the processed data and can perform further post-processing and / or image reconstruction. According to the present invention, all or part of the analog processing circuit 25 can be implemented in the X-ray detector system 20.

[0045] The overall X-ray detector can be regarded as the X-ray detector system 20, or the X-ray detector system 20 combined with the related analog processing circuit 25.

[0046] The digital part including the digital processing circuit 40 and / or the computer 50 can be regarded as an image processing system 30 that performs image reconstruction based on the image data from the X-ray detector. The image processing system 30 may be defined as the computer 50, or the image processing system 35 (digital image processing) may be defined as a combined system of the digital processing circuit 40 and the computer 50, or when the digital processing circuit is further specialized for image processing and / or reconstruction, it may be defined as the digital processing circuit 40 alone.

[0047] As an example of a commonly used X-ray imaging system, there is an X-ray computed tomography (CT) system, which includes an X-ray tube that generates a fan, cone, or beam of X-rays, and an opposing array of X-ray detectors that measure the proportion of X-rays that have passed through a patient or object. The X-ray tube and the detector array are attached to a gantry that rotates around the subject.

[0048] FIG. 3 is a schematic block diagram of a CT system as an example of an X-ray imaging system. The CT system is composed of a computer 50 that receives commands and scanning parameters from an operator via a display and some form of operator interface, such as an operator console 60 having a keyboard and a mouse. The commands and parameters supplied from the operator are used by the computer 50 to supply control signals to an X-ray control device 41, a gantry control device 42, and a table control device 43. Specifically, the X-ray controller 41 supplies power and timing signals to the X-ray source 10 and controls the emission of X-rays to an object or patient lying on the table 12. The gantry control device 42 controls the rotation speed and position of the gantry 11 that constitutes the X-ray source 10 and the X-ray detector 20. The table control device 43 controls and determines the position of the patient table 12 and the scanning range of the patient. There is also a detector controller 44, which is configured to control the detector 20 and / or receive data from the detector 20.

[0049] In one embodiment, the computer 50 also performs post - processing and image reconstruction of the image data output from the X - ray detector. As a result, the computer corresponds to the image processing system 30 shown in FIGS. 1 and 2. The operator can observe the reconstructed image and other data from the computer on the associated display.

[0050] The X - ray source 10 arranged on the gantry 11 emits X - rays. For example, an X - ray detector 20 in the form of a photon - counting detector detects the X - rays after passing through the patient. The X - ray detector 20 is formed, for example, by a plurality of pixels, also called sensor or detector elements, and associated processing circuits such as an ASIC arranged in the detector module. A part of the analog processing part is implemented in the pixels, and the remaining processing part is implemented, for example, in the ASIC. In one embodiment, the processing circuit (ASIC) digitizes the analog signal from the pixels. The processing circuit (ASIC) can also constitute a digital processing unit that can perform further processing operations on the measurement data, such as the application of corrections, temporary storage, and / or filtering. During the scan to acquire X - ray projection data, the gantry and the components mounted thereon rotate about the iso - center.

[0051] Modern X - ray detectors usually need to convert the incident X - rays into electrons. This is usually done by the photoelectric effect or Compton interaction, and the resulting electrons usually create secondary visible light until their energy is lost, and this light is detected by a photosensitive material. There are also semiconductor - based detectors, in which case the electrons generated by the X - rays create charges in the form of electron - hole pairs that are collected by the applied electric field.

[0052] Detectors operating in the energy - integrating mode exist in the sense that they provide a signal integrated from a large number of X - rays. The output signal is proportional to the total energy accumulated by the detected X - rays.

[0053] In medical X-ray applications, X-ray detectors with photon counting and energy resolution capabilities are becoming increasingly common. Photon counting X-ray detectors basically have the advantage of being able to measure the energy of each X-ray, so additional information about the composition of the subject can be obtained, and this information can be used to improve image quality and reduce radiation dose.

[0054] Generally, a photon counting X-ray detector determines the energy of a photon by comparing the height of the electrical pulse generated by the interaction of the photon in the detector material with a series of comparator voltages (comparator voltages). These comparator voltages are also called energy thresholds. Generally, the analog voltage of the comparator is set by a digital-to-analog converter (DAC). The DAC converts the digital setting transmitted from the controller into an analog voltage that can compare the height of the photon pulse.

[0055] A photon counting detector counts the number of photons that interacted within the detector during the measurement time. New photons are generally identified when the height of the electrical pulse exceeds the comparator voltage of at least one comparator. When a photon is identified, an event is stored by incrementing the digital counter associated with the channel.

[0056] When using multiple different thresholds, a so-called energy-discriminating photon counting detector can be obtained, which can sort the detected photons into energy bins corresponding to various thresholds. This type of photon counting detector is sometimes also called a multi-bin detector. Generally, energy information enables the creation of a new type of image where new information is available and image artifacts inherent in the prior art can be removed. In other words, in an energy-discriminating photon counting detector, the height of the pulse is compared with several programmable thresholds (T1 - TN) in the comparator and classified according to the height of the pulse. In other words, a photon counting detector composed of two or more comparators is called a multi-bin photon counting detector. In the case of a multi-bin photon counting detector, the photon counts are typically stored in a set of counters, one for each energy threshold. For example, the counters can be assigned to correspond to the highest energy threshold exceeded by the photon pulse. In another example, the counters record the number of times the photon pulse crosses each energy threshold.

[0057] As an example, edge-on is a special and non-limiting design for a photon counting detector in which an X-ray sensor such as an X-ray detection element or pixel is edge-on with respect to the incident X-ray.

[0058] For example, such a photon counting detector can have pixels in at least two directions, and one of the directions of the edge-on photon counting detector has a component in the direction of the X-ray. Such an edge-on photon counting detector is sometimes called a depth-segmented photon counting detector having pixels of two or more depth segments in the direction of the incident X-ray.

[0059] Alternatively, the pixels may be arranged as an array (non-depth-segmented) in a direction substantially orthogonal to the direction of the incident X-ray, and each of the pixels may be oriented edge-on with respect to the incident X-ray. In other words, the photon counting detector may be non-depth-segmented while remaining edge-on with respect to the incident X-ray.

[0060] To increase the absorption efficiency, it is also possible to arrange the edge-on type photon counting detector edge-on. In this case, the absorption depth can be selected to any length, and the edge-on type photon counting detector can still be fully depleted without applying a very high voltage.

[0061] The conventional mechanism for detecting X-ray photons through a direct semiconductor detector basically operates as follows. The energy of the X-ray interaction in the detector material is converted into electron-hole pairs inside the semiconductor defect, and the number of electron-hole pairs generally scales with the photon energy. The electrons and holes drift towards the electrodes and the back surface of the detector (or vice versa). During this drift, the electrons and holes induce a current in the electrodes, and this current can be measured.

[0062] As shown in FIG. 4, one or more signals are sent from the detector element 21 of the X-ray detector to the inputs 27 of an analog processing circuit (e.g., ASIC) 25. The term Application Specific Integrated Circuit (ASIC) should be understood to be broadly interpreted as a general circuit used and configured for a specific application. The ASIC can process the charge generated from each X-ray, convert it into digital data, and can be used to obtain measurement data such as the number of photons and / or the estimated energy. Since the ASIC is configured to connect to a digital processing circuit, the digital data is sent to a further digital processing circuit 40 and / or one or more memories 45, and finally the data serves as an input to an image processing circuit 30 for generating a reconstructed image.

[0063] Since the number of electrons and holes generated from one X-ray event is proportional to the energy of the X-ray photon, the total charge contained in one induced current pulse is proportional to this energy. After the filtering step of the ASIC, the pulse amplitude is proportional to the total charge of the current pulse and thus proportional to the X-ray energy. The amplitude of the pulse can be measured by comparing its value with one or more threshold values (THR) of one or more comparators (COMP), and a counter is introduced to record the number of times the pulse is greater than the threshold. In this way, it is possible to count and / or record the number of X-ray photons having an energy exceeding the energy corresponding to each threshold value (THR) detected within a certain time frame.

[0064] The ASIC typically samples the analog photon pulse once per clock cycle and registers the output of the comparator. The comparator (threshold) outputs a 1 or 0 depending on whether the analog signal exceeds or falls below the comparator voltage. The information available at each sample is, for example, the 1 or 0 of each comparator indicating whether the comparator was triggered (the photon pulse was higher than the threshold).

[0065] In a photon counting detector, there is typically photon counting logic that determines whether a new photon has been registered and registers the photon in a counter. In the case of a multi-bin photon counting detector, there are typically multiple counters, for example one for each comparator, and the photon count is registered in the counter according to an estimated value of the photon energy. The logic can be implemented in several different ways. The two most common categories of photon counting logic are the so-called non-paralyzable counting modes and paralyzable counting modes. Other photon counting logic includes, for example, local maxima detection that counts the detected local maxima in the voltage pulse and, in some cases, also registers the height of the pulse.

[0066] Photon counting detectors have many advantages, such as high spatial resolution, low electronic noise, energy resolution, material separation capability, spectral imaging ability, etc., but are not limited to these. However, energy integrating detectors have the advantage of high count-rate tolerance. Count-rate tolerance is derived from the fact / recognition that since the total energy of a photon is measured, adding one more photon always increases the output signal (within a reasonable range) regardless of the amount of photons currently registered in the detector. This decisive advantage is one of the main reasons why energy integrating detectors have become the standard in today's medical CT.

[0067] To better understand, it would be useful to start with an overview of a simple system and / or an analysis of some technical problems. For this purpose, refer to FIG. 5 which provides a schematic diagram of a prior art photon counting circuit and / or device.

[0068] When a photon interacts in a semiconductor material, a cloud of electron-hole pairs is generated. Applying an electric field to the detector material causes the charge carriers to be collected at the electrodes attached to the detector material. The signal is sent from the detector element to the input of an analog processing circuit (such as an ASIC). It should be understood that the term "application-specific integrated circuit (ASIC)" is widely interpreted as a general circuit used and configured for a specific application. The ASIC can process the charge generated from each X-ray, convert it into digital data, and be used to obtain measurement data such as the number of photons and / or the estimated energy. In one example, the ASIC can process the charge such that a voltage pulse with a maximum height proportional to the amount of energy deposited by the photon in the detector material is generated.

[0069] The ASIC can include a set of comparators 302, and each comparator 302 compares the magnitude of a voltage pulse with a reference voltage. The comparator output typically becomes zero or one (0 / 1) depending on which of the two compared voltages is greater. Here, it is assumed that if the voltage pulse is higher than the reference voltage, the comparator output is 1, and if the reference voltage is higher than the voltage pulse, it is 0. A digital - to - analog converter, DAC, 301 can be used to convert digital settings that can be supplied by a user or a control program into reference voltages that can be used by the comparators 302. If the height of the voltage pulse exceeds the reference voltage of a particular comparator, that comparator is said to be triggered. Each comparator is generally associated with a digital counter 303, and this counter is incremented based on the comparator output according to photon - counting logic.

[0070] Generally, basis - material decomposition utilizes the fact that all materials composed of low - atomic - number elements such as human tissue have a linear attenuation coefficient μ(E) that can be approximately expressed as a linear combination of two (or more) basis functions with energy dependence.

Number

[0071] Here, f1 and f2 are basis functions, and a1 and a2 are the corresponding basis coefficients. More generally, f i is a basis function, and a i is the corresponding basis coefficient. If there is one or more elements with a high enough atomic number such that an absorption edge k exists in the energy range used for imaging within the imaging volume, it is necessary to add one basis function for each such element. In the field of medical imaging, such k - absorption - edge elements are typically iodine or gadolinium, substances used as contrast agents.

[0072] As described above, the line integral A i of each basis coefficient a1 is inferred from the measured data of each projection ray l from the radiation source to the detector element. The line integral A iIt can be expressed as follows.

Number

[0073] Here, N is the number of basis functions. In one embodiment, the decomposition of the base substance is first to predict the number of expected registrations of counts in each energy bin as a function of A i is achieved by expressing it. Usually, such a function can take the following form.

Number

[0074] Here, λ i is the expected count of energy bin t, E is the energy, s i is a response function that depends on the spectral shape incident on the imaging object, the quantum efficiency of the detector, and the sensitivity to X-rays of the energy if of energy bin i. The term "energy bin" is most commonly used for photon counting detectors, but this equation can also describe other energy-resolved X-ray systems such as multi-layer detectors and kVp switching sources.

[0075] Next, using the maximum likelihood method, A can be estimated under the assumption that the count of each bin is a Poisson distribution random variable. This is achieved by minimizing the negative log-likelihood function. See Roessl and Proksa, "K-edge imaging in X-ray computed tomography using multi-bin photon counting detectors", Phys. Biol. 52 (2007), 4679-4696. i

Number

[0076] Here, m i$N_i$ is the measured count of energy bin $i$, and $M_b$ is the number of energy bins.

[0077] From the line integral $A$, the basis coefficient $a$ i Tomographic reconstruction can be performed to obtain this. This step may be regarded as another tomographic reconstruction or as part of the overall basis decomposition.

[0078] As described above, when the estimated basis coefficient line integral $A$ for each projection line i is placed in the image matrix, the result is the projection image specific to the substance for each basis $i$ (also called the basis image). This basis image can be directly seen (e.g., in projection X-ray imaging) or can be taken as an input to a reconstruction algorithm for forming a map of the basis coefficient $a$ i inside the object (e.g., in CT). In any case, the result of the basis decomposition can be regarded as one or more basis image representations such as the line integral of the basis coefficient or the basis coefficient itself.

[0079] In the field of X-ray imaging, the representation of image data is composed of, for example, a sinogram, a projection image, or a reconstructed CT image. Such a representation of image data may be energy-decomposed when the data in different channels is related to the measured X-ray data in different energy intervals, i.e., when it is composed of a plurality of channels, so-called multi-channel or multi-bin energy information.

[0080] Through the process of material decomposition that receives as input the representation of energy-decomposed X-ray image data, a set of basis image representations can be generated. Such a set is a collection of a large number of basis image representations, and each basis image representation is related to the contribution of a specific basis function to the total X-ray attenuation. Such a set of basis image representations may be a set of basis sinograms, a set of reconstructed basis CT images, or a set of projection images. It will be understood that "image" in this context may mean, for example, a two-dimensional image, a three-dimensional image, or a time-resolved image sequence.

[0081] For example, the representation of energy-resolved X-ray image data can be constructed from a collection of energy bin sinograms, where each energy sinogram contains the counts (count numbers) measured in one energy bin. By using this collection of energy bin sinograms as an input to a material decomposition algorithm, a set of basis sinograms can be generated. Such basis sinograms can be taken as an input to a reconstruction algorithm to generate a reconstructed basis image.

[0082] In two-basis decomposition, two basis image representations are generated. This is based on the approximation that the attenuation of any material within the imaged object can be represented as a linear combination over two basis functions. In three-basis decomposition, three basis image representations are generated based on the approximation that the attenuation of any material within the imaged object can be represented as a linear combination over three basis images. Similarly, four-basis decomposition, five-basis decomposition, and similar higher-order decompositions can be defined. Also, it is possible to perform one-basis decomposition by approximating all materials within the imaged object as having X-ray attenuation coefficients with similar energy dependencies up to a density scale factor.

[0083] Two-basis decomposition results in a set of basis sinograms that includes, for example, a water sinogram and an iodine sinogram, corresponding to basis functions given by the linear attenuation coefficients of water and iodine, respectively. Alternatively, the basis functions may represent the attenuation of water and calcium, calcium and iodine, polyvinyl chloride and polyethylene. Three-basis decomposition results in a set of basis sinograms that includes, for example, a water sinogram, a calcium sinogram, and an iodine sinogram. Alternatively, the basis functions represent the attenuation of water, iodine, gadolinium, or polyvinyl chloride, polyethylene, iodine.

[0084] As described above, machine learning such as artificial intelligence (AI) and deep learning has begun to be used for general image reconstruction and has achieved somewhat satisfactory results. However, the current problem with image reconstruction by machine learning such as image reconstruction by deep learning is its limited empiricism. Even if an image appears to have little noise at first glance, it actually contains errors due to the bias of the neural network estimator.

[0085] Generally speaking, deep learning is a machine learning method using representation learning based on artificial neural networks and similar architectures. There are supervised, semi-supervised, and unsupervised learning. Deep learning systems such as deep neural networks, deep belief networks, recurrent neural networks, and convolutional neural networks have been applied to various technical fields such as computer vision, speech recognition, natural language processing, social network filtering, machine translation, and board game programs, producing results comparable to or in some cases exceeding those of human experts.

[0086] The adjective "deep" in deep learning (deep neural network) comes from the use of multiple layers in the network. In early research, it was shown that a linear perceptron could not be a universal classifier, but a network with one hidden layer with a non-polynomial activation function and an unbounded width could be a universal classifier. Deep learning is the latest variation that, theoretically, has an unlimited number of layers and is of finite size, enabling practical applications and optimized implementations, while maintaining theoretical universality under mild conditions. In deep learning, for efficiency, ease of training, and ease of understanding, the layers are heterogeneous and are also allowed to deviate significantly from biologically informed connectionist models.

[0087] The inventors have realized that there is a need to improve the reliability and / or explainability in machine learning image reconstruction such as deep learning image reconstruction, particularly in computer tomography (CT).

[0088] The proposed technique is generally applicable to provide an indicator of the reliability of reconstructed images and / or image features based on machine learning such as neural networks and deep learning.

[0089] As described above, the inventors have noticed that in order to make an image (such as those described above) obtained from machine learning image reconstruction, such as deep learning image reconstruction, reliable, it is highly desirable to quantify the degree of confidence in the reconstructed image (value), or otherwise determine the display or representation of confidence. This is particularly important in photon counting spectral CT because, although it is theoretically possible to generate a quantitatively accurate map of the material composition, especially in 3-basis decomposition, the noise level is high, meaning that machine learning such as deep learning image reconstruction must or should be used as an important component of the image reconstruction chain.

[0090] In a sense, the basic idea of the present invention is to provide a radiologist with a reliability display, such as an uncertainty map, for each image or image feature generated by machine learning image reconstruction, such as deep learning image reconstruction.

[0091] According to a first main aspect, a non-limiting example of a method for determining a confidence display for machine learning image reconstruction, such as deep learning image reconstruction, in computed tomography (CT) is provided.

[0092] FIG. 6A is a schematic flow diagram showing an example of a method for determining a confidence display for machine learning image reconstruction, such as deep learning image reconstruction, in computed tomography (CT).

[0093] Basically, this method includes the following steps. Acquire energy-resolved X-ray data (S1). Process the energy-resolved X-ray data based on at least one machine learning system, such as a neural network, to generate a representation of the posterior probability distribution of at least one reconstructed basis image or its image features (S2). Based on the representation of the posterior probability distribution, generate one or more reliability displays for the at least one reconstructed image, or at least one derivative image derived from the at least one reconstructed basis image, or the image features of the at least one reconstructed basis image or the at least one derivative image (S3).

[0094] In other words, this can be expressed as processing energy-resolved X-ray data based on at least one neural network or similar machine learning system to obtain a representation of the posterior probability distribution of at least one basis image or at least one of its image features. Then, this representation can be processed to form a reliability display of one or more images or image features.

[0095] It is understood that a set of training data, for example, a measured energy-resolved X-ray data set and a corresponding set of ground truth or reconstructed basis material maps specially selected for training a machine learning system such as a neural network, can be used to specify or approximate the probability distribution of one or more reconstructed basis material images. Such a distribution prior to the new measurement being evaluated is called a prior distribution. When one or more measurements of the representation of the X-ray image data are further performed, the probability distribution of the possible basis material images due to this additional knowledge of the measurement is known as a posterior probability distribution.

[0096] In other words, the prior information about how the CT image is likely to look is typically specified by a training data set that includes pairs of training input image data and output image data. Such input image data and output image data can take the form of images with different contents such as a sinogram or a bin image or a sinogram or a base image or a sinogram. By training a mapping to map the input data in each pair to output image data that is as similar as possible to the corresponding output image data in the input-output training pair, a mapping can be obtained that can perform noise removal, decomposition into a base image, or image reconstruction from the measured image data. The training output image data in each pair is also called a label. In a preferred embodiment, such a mapping can take the form of a convolutional neural network (CNN), but there are also other embodiments such as support vector machines or decision trees, and this mapping can be realized / constructed. To find the mapping that gives the best match between the network output and the training output image data, a data mismatch function, also called a loss function, is typically used to calculate the data mismatch between the network output and the training output image data. In a preferred embodiment of the present invention, the mapping may be probabilistic, which means that it gives different outputs when applied multiple times to the same input data. In this embodiment, the loss function can take the form of, for example, the Kullback-Leibler distance or the Wasserstein distance between the distribution of the output image data generated by the network and the distribution of the training output image data.

[0097] As an example, the training of the convolutional neural network is performed by minimizing this data discrepancy using an optimization method such as ADAM. Once the mapping is trained, it can be applied at runtime by mapping the measured image data to generate output image data. For example, the probabilistic mapping can be applied multiple times to the input image data to generate a set (ensemble) of output image data. The mean and standard deviation of the output image data can be calculated over this set, and the mean output image can be used as an estimate of the denoised image, the decomposed image, or the reconstructed image, and the standard deviation can be used as an estimate of the uncertainty of the denoised image, the decomposed image, or the reconstructed image.

[0098] In another embodiment of the present invention, two separate neural networks are used. One network is trained to generate an estimate of the output image data, for example, the reconstructed basis image, and the second network is trained to generate an estimate of the uncertainty of the output image data, for example, a map of the uncertainty of the reconstructed basis image. For example, one way to train such networks is to first train a single probabilistic neural network to generate samples from the posterior distribution of the output image data as described above, and then train two neural networks to predict the mean and standard deviation of the posterior distribution.

[0099] In yet another embodiment of the present invention, the network that predicts the mean and standard deviation of the output image data can be directly learned. This is achieved by assuming an output probability distribution parameterized by the mean and standard deviation and minimizing a data discrepancy measure such as the Kullback-Leibler difference or the Wasserstein difference between the output probability distribution with the parameters predicted by the network and the approximation of the posterior distribution of the output image data based on the training dataset.

[0100] In yet another embodiment of the present invention, a neural network estimator implemented according to one of the above methods is trained to predict the uncertainty of a non-neural-network-based CT data processing method, such as a reconstruction, decomposition, or noise removal method. For this purpose, the uncertainty of the CT data processing method can be predicted by repeatedly applying the method to noisy data, such as simulated data or measured data, and the neural network can be trained to predict such noisy data.

[0101] In an exemplary embodiment of the present invention, the energy-resolved X-ray data is obtained using a photon-counting X-ray detector or obtained from an intermediate memory storing the energy-resolved X-ray data.

[0102] The fact that energy-resolved X-ray data is used means that multi-channel energy information is employed. Further, the fact that one or more basis images, also referred to as basis material images or material-specific images or material-selective images, are considered means that multiple materials (i.e., at least two basis materials) are involved in the overall analysis. This leads to a higher-dimensional context.

[0103] The confidence indication may be any suitable indication of the confidence of the finally reconstructed image(s) or image feature(s), such as by machine learning image reconstruction such as deep learning image reconstruction, for example, a relevant quantification of the confidence and / or trust of the reconstructed image(s) and / or image feature(s). The confidence indication may also be a complex representation of confidence, such as an uncertainty map, as will be illustrated in detail later.

[0104] An example of the reliability display is a map showing the uncertainty of the base image estimate, such as the standard deviation of the estimated iodine concentration, for example. Thereby, an image is obtained that emphasizes regions with high uncertainty in iodine concentration in the reconstructed iodine base image.

[0105] Another example of the reliability display is a reliability map showing the reliability that a certain substance exists in various locations. As an example, such a map can highlight regions where iodine is surely present in the image, while leaving regions where it can be said with high certainty that iodine is not contained dark. Such a reliability map can be calculated, for example, by dividing the estimated iodine concentration by the estimated standard deviation of the iodine concentration. In another example, such a map can be calculated by calculating the posterior probability that the map contains iodine at a specific position. Yet another example of the reliability display is a confidence interval for the concentration of one or more base substances at each point in the image.

[0106] As an example, the machine learning image reconstruction is deep learning image reconstruction, and the at least one machine learning system includes at least one neural network.

[0107] In a specific example, the representation of the posterior probability distribution can include at least one of a mean variance, a covariance, a standard deviation, a skewness, and a kurtosis.

[0108] Optionally, one or more reliability displays can include an error estimate value or a measure of statistical uncertainty for at least one point in the at least one reconstructed base image, and / or an error estimate value or a measure of statistical uncertainty for at least one image measurement value derivable from the at least one reconstructed base image.

[0109] For example, the error estimate value or the measure of statistical uncertainty can include at least one of an upper bound for an error, a lower bound for an error, a standard deviation, a variance or a mean absolute error.

[0110] As an example, the at least one image measurement can include at least one of a dimensional measure of a feature, an area, a volume, a degree of inhomogeneity, a measure of shape or irregularity, a measure of composition, and a measure of concentration of a sub-stance.

[0111] As will be illustrated later, one or more reliability displays can include one or more uncertainty maps for the at least one reconstructed basis image, or at least one derived image derived from the at least one reconstructed basis image, or its image features.

[0112] In a particular embodiment, step S3 of generating one or more reliability displays includes generating a reliability map of a reconstructed material-selective X-ray image for Computed Tomography (CT).

[0113] As an example, the reliability map can be generated to highlight portions of the reconstructed material-selective X-ray image that can be determined by machine learning image reconstruction with a reliability above a predetermined threshold, i.e., with high reliability.

[0114] For example, step S3 of generating one or more reliability displays may include generating one or more reliability maps by a neural network that takes as input a substance concentration map obtained from substance decomposition based on deep learning.

[0115] In a specific embodiment schematically shown in FIG. 6B, the method further includes performing S2a substance decomposition-based image reconstruction and / or machine learning image reconstruction to generate the at least one reconstructed basis image or its image features based on the acquired energy-dispersive X-ray data.

[0116] As an example, step S2a of performing image reconstruction based on substance decomposition and / or image reconstruction by machine learning may include generating the at least one reconstructed basis image or image features by a neural network that takes as input an energy bin sinogram.

[0117] In an optional embodiment, step S3 of generating one or more reliability indicators may include determining the uncertainty or reliability map of individual basis substance images and the covariance between different basis substance images. Thereby, using a formula or algorithm for the propagation of uncertainty, the uncertainty or reliability map can be propagated to generate an uncertainty map of the derived image.

[0118] In a specific embodiment, the at least one basis substance image may be generated together with at least one uncertainty map, the uncertainty map being a representation of the uncertainty or error estimate of the at least one basis substance image, and the at least one basis substance image and the at least one uncertainty map can be presented to the user as separate images / maps or in combination.

[0119] For example, the at least one uncertainty map may be presented as an overlay on the at least one substrate image, or the at least one uncertainty map may be presented by a distorting filter on the at least one substrate image.

[0120] As an example, step S2 (FIG. 6A) or S2b (FIG. 6B) of processing energy-dispersive X-ray data based on at least one machine learning system to generate a representation of a posterior probability distribution consists of generating, by a neural network, samples of a posterior probability distribution given the acquired energy-dispersive X-ray data, and step S3 of generating one or more reliability displays includes generating an uncertainty map as a standard deviation over a plurality of samples.

[0121] In any embodiment, step S2 or S2b of processing energy-dispersive X-ray data based on at least one machine learning system to generate a representation of a posterior probability distribution applies a neural network implemented as a variational autoencoder to encode an input data vector into parameters of a probability distribution of a latent random variable, and to extract, for subsequent processing by a corresponding decoder to obtain posterior observations, a collection of posterior samples of the latent random variable from this probability distribution.

[0122] In a particular example, step S3 of generating one or more reliability displays includes generating at least one map of a variance or standard deviation of at least one basis coefficient, and / or at least one map of a covariance or correlation coefficient of at least one set of basis functions associated with the at least one reconstructed basis image.

[0123] In an exemplary embodiment of the present invention, the representation of the posterior probability distribution is defined by the mean and variance of a plurality of image features.

[0124] In an exemplary embodiment of the present invention, the representation of the posterior probability distribution can be given by a number of Monte Carlo samples from the distribution.

[0125] In an exemplary embodiment of the present invention, the neural network is a convolutional neural network (CNN) having at least five layers.

[0126] In an exemplary embodiment of the present invention, the processing based on the neural network may include processing by a probabilistic neural network.

[0127] In an exemplary embodiment of the present invention, the neural network is configured to operate based on random dropout, noise insertion, a variational autoencoder, or noisy stochastic gradient descent.

[0128] In an exemplary embodiment of the present invention, the processing based on the neural network may include processing by a deterministic neural network that provides a measure of the posterior probability distribution.

[0129] In an exemplary embodiment of the present invention, the processing based on the neural network can include processing by a deterministic neural network that provides a measure of uncertainty of a reconstructed image or image feature.

[0130] In an exemplary embodiment of the present invention, the processing based on the neural network is based on a neural network based on one or more inputs calculated from at least one physical model of data acquisition.

[0131] In an exemplary embodiment of the present invention, the at least one input calculated from the physical model of data acquisition is a gradient of a data discrepancy function, an estimate of a scattered photon distribution, or a representation of cross-talk between detector pixels, or a representation of pile-up.

[0132] In an exemplary embodiment of the present invention, the processing is based on a neural network including an unrolled optimization neural network architecture.

[0133] In an exemplary embodiment of the present invention, the processing is based on a neural network that takes as input at least one standard deviation, variance, or covariance map in the image space or sinogram space based on the Cramer-Rao lower bound.

[0134] In an exemplary embodiment of the present invention, the processing can be based on a neural network that performs the following steps. performing at least two basis material decompositions on at least one representation of the energy-resolved X-ray image data, resulting in at least two original basis image representation sets; and obtaining or selecting at least two basis image representations from at least two of the original basis image representation sets; and processing the obtained or selected basis image representations by data processing based on the neural network, resulting in representing a posterior probability distribution of the basis image representation set.

[0135] In an exemplary embodiment of the present invention, the processing is based on a neural network trained by minimizing a loss function calculated as a measure of mismatch in the image space or sinogram space between a label, i.e., a predetermined output corresponding to the network input in the training set, and the network output.

[0136] In an exemplary embodiment of the present invention, the loss function is based on a weighted mean squared error, a Kullback-Leibler distance, or a Wasserstein distance.

[0137] In an exemplary embodiment of the present invention, the loss function incorporates at least two different basis material components incorporated with different weight coefficients.

[0138] In an exemplary embodiment of the present invention, the loss function is calculated based on a set of basis coefficients in a basis transformed with respect to the original basis.

[0139] In an exemplary embodiment of the present invention, the algorithm is trained with image data generated by intentionally introducing a model error. Thereby, the neural network estimator can be made more robust to model errors and model uncertainties. Further, by this method, the uncertainty of the image due to unknown model errors can be incorporated into the probabilistic neural network.

[0140] According to a complementary aspect, a non-limiting example of a method for generating an uncertainty map for machine learning image reconstruction, such as deep learning image reconstruction in spectral CT, is provided.

[0141] FIG. 7 is a schematic flowchart showing an example of a method for generating an uncertainty map for machine learning image reconstruction, such as deep learning image reconstruction in spectral CT.

[0142] This method includes the following steps. A step of acquiring energy-resolved X-ray data (S11). A step of processing the energy-resolved X-ray data based on at least one neural network so that a representation of the posterior probability distribution of at least one reconstructed basis image or its image features can be obtained (S12). A step of generating one or more uncertainty maps for at least one reconstructed image, or a derived image, or its image features based on the representation of the posterior probability distribution (S13).

[0143] In an exemplary embodiment of the present invention, the step of generating one or more uncertainty maps includes generating at least one map of the variance or standard deviation of at least one basis coefficient and / or at least one map of the covariance or correlation coefficient of at least one set of basis functions.

[0144] As an example, the energy-resolved X-ray data is acquired from a photon-counting type X-ray detector, acquired by a photon-counting type X-ray detector, or acquired from an intermediate memory storing the energy-resolved X-ray data.

[0145] In an exemplary embodiment of the present invention, the neural network is a convolutional neural network (CNN) having at least five layers.

[0146] In an exemplary embodiment of the present invention, the representation of the posterior probability distribution is defined by the mean and variance of a plurality of image features.

[0147] In an exemplary embodiment of the present invention, the representation of the posterior probability distribution can be given by a number of Monte Carlo samples from the distribution.

[0148] In an exemplary embodiment of the present invention, the processing based on the neural network may include processing by a stochastic neural network.

[0149] In an exemplary embodiment of the present invention, the neural network is configured to operate based on random dropout, noise insertion, a variational autoencoder or noisy stochastic gradient descent.

[0150] In an exemplary embodiment of the present invention, the processing based on the neural network may include processing by a deterministic neural network that provides a scale of a probability distribution.

[0151] In an exemplary embodiment of the present invention, the processing based on the neural network can include processing by a deterministic neural network that provides a measure of the uncertainty of a reconstructed image or image features.

[0152] In an exemplary embodiment of the present invention, the neural network-based processing is based on a neural network based on one or more inputs calculated from at least one physical model of data acquisition.

[0153] In an exemplary embodiment of the present invention, the at least one input calculated from the physical model of data acquisition is a gradient of a data discrepancy function, an estimate of a scattered photon distribution, or a representation of cross-talk between detector pixels, or a representation of pile-up.

[0154] In an exemplary embodiment of the present invention, the processing is based on a neural network including an unrolled optimization neural network architecture.

[0155] In an exemplary embodiment of the present invention, the processing is based on a neural network that takes as input at least one standard deviation, variance, or covariance map in the image space or sinogram space based on the Cramer-Rao lower bound.

[0156] In an exemplary embodiment of the present invention, the processing is based on a neural network trained by minimizing a loss function calculated as a discrepancy measure in the image space or sinogram space between a label and a network output.

[0157] In an exemplary embodiment of the present invention, the loss function is based on a weighted mean square error in which at least two different base substance components are incorporated with different weight coefficients.

[0158] To provide an exemplary framework for facilitating understanding of the proposed technology, a specific example of image reconstruction based on deep learning in the specific context of CT image reconstruction is shown next.

[0159] However, it should be understood that the proposed technology for providing an indication of the reliability of deep learning image reconstruction in CT applications is generally applicable to image reconstruction based on deep learning for CT and is not limited to the following specific examples of image reconstruction based on deep learning.

[0160] As an example, the disclosed invention can provide a reliability map of a reconstructed material-selective X-ray CT image. Such a reliability map can highlight portions of the image that the reconstruction algorithm was able to determine with high confidence.

[0161] In particular, such an image can be provided for an image of the distribution of a contrast agent such as iodine. It is understood that quantifying iodine by three-base decomposition is very sensitive to noise, and thus a reconstruction algorithm such as a deep learning algorithm may need to make extensive use of prior information to obtain this image. Therefore, a reliability map for the iodine concentration is useful for enabling an observer such as a radiologist to interpret the image, as schematically shown in FIG. 8, for example.

[0162] Furthermore, it is understood that the noise in the decomposed basis images and sinograms typically has a high correlation between different basis material images. Additionally, if the image reconstruction algorithm is incomplete, it is understood that one feature of a basis image, such as a region containing iodine, may appear as an artifact in another basis image, such as a basis image of bone. Therefore, it is important to predict not only the uncertainty of individual material maps but also the covariance between different material maps. This allows for propagating the reliability map using a formula or algorithm for uncertainty propagation and obtaining a derived image (e.g., a virtual non-contrast image, a virtual non-calcium image, a virtual monoenergetic image, or a synthetic Hounsfield unit image uncertainty map).

[0163] In non-limiting embodiments of the disclosed invention, a method is provided for generating at least one basis material image together with at least one uncertainty map, where the uncertainty map is a representation of the uncertainty or error estimate of the basis material image. Such at least one basis material image together with at least one uncertainty map can be presented to the user as separate images or combined, for example, as a color overlay. As another possibility, the at least one uncertainty map can be presented in the form of a distortion filter of the basis material map, for example, by a blur filter.

[0164] FIG. 9 shows a non-limiting example of a probabilistic neural network that generates an average material image together with a variance map or uncertainty map.

[0165] FIG. 10 shows a deep neural network that maps a substance concentration map to an uncertainty map. Such maps can be presented together with the underlying substance map or separately (Bone: bone, Soft Tissue: soft tissue, Iodine: iodine).

[0166] It is also understood that in order to generate high-precision CT images, a detector with excellent energy resolution such as a photon-counting detector is required. It is also understood that an accurate physical model is beneficial for generating high-precision CT images from the energy-resolved measurement data. Such a physical model can be incorporated into deep learning image processing or reconstruction algorithms, for example, by unrolling an iterative optimization loop.

[0167] FIG. 11 shows an exemplary embodiment of such an unrolled iterative loop for sinogram spatial basis material decomposition. Thereby, an input sinogram is processed through a series of neural network blocks, each block may include one or more neural network layers. As described above, the energy sinogram may include, for example, the counts (count numbers) measured in a particular energy bin. Each block takes as input the output from the previous block, along with an estimate of the gradient of an objective function, such as a likelihood function or log-likelihood function. In one embodiment of the present invention, this likelihood function can be a likelihood function for detecting a particular combination of measured counts given an estimate of the basis material sinogram. In a preferred embodiment, the layers of the neural network can be convolutional layers of a convolutional neural network. In a preferred embodiment, the neural network is implemented using a graphics processing unit (GPU). This is a non-limiting example, and it should be understood that through projection and back-projection operations within the neural network, the network can convert a material image or energy bin count images into a material image, or convert bin counts (bin counts) or a material sinogram into a material image.

[0168] Separate from the unrolled gradient descent algorithm described above, other iterative algorithms such as a Newton method, a conjugate gradient method, a Nesterov-accelerated method or a primal-dual method can be unrolled, resulting in different network architectures or different functions of the image estimates as inputs to each network layer.

[0169] In an exemplary embodiment of the present invention, the neural network architecture can be based on a physical model based on models of the focal spot shape, the x-ray spectrum shape, charge sharing, scatter in the patient, scatter inside the detector or pile-up; or on a material decomposition method that takes into account correction terms. By these means, different functions can be applied to the estimated output at one or more steps within the neural network.

[0170] A combination of a photon counting detector and careful physics modeling can generate high-precision photon counting images. It is understood that this advantage of high precision is enhanced by providing reliable error estimation. The proposed technique is based on the finding that a high-precision quantitative image can be generated together with error estimation by using spectral CT and neural network-based error estimation in combination. Furthermore, it is understood that both image estimation and uncertainty estimation can be further improved by incorporating at least one model related to the physics of image acquisition.

[0171] As an example, a method for generating an uncertainty map is disclosed. Using a probabilistic neural network such as a Bayesian neural network, samples can be generated from the posterior probability distribution of one or more images given observed / measured data and a training set. As an example, the training set can include a set of input-output pairs, each training input being a set of bin sinograms and the training output (label) being a set of basis images. Such a set can be generated, for example, through simulation of CT imaging of a numerical phantom or through measurement of physical phantoms having known compositions. In another example, such training pairs are generated by CT imaging of a patient, the training output is obtained as a reconstructed image, and the training input can be obtained as a measured sinogram, as a modified sinogram with additional noise, or as a sinogram from another CT scan of the same subject acquired at a lower dose. By using training inputs with increased noise compared to the training output, the resulting trained neural network can achieve the ability to reduce noise. In another embodiment of the present invention, the input data may be a set of basis sinograms, a set of reconstructed bin images, or a set of basis images. In yet another embodiment of the present invention, the input data may be a set of basis sinograms, a set of reconstructed bin images, or a set of basis images. In this way, a neural network can be constructed that operates in either the sinogram region or the image region and performs basis or bin image or basis decomposition or noise removal of the sinogram.

[0172] Once trained, the neural network is ready to process the observed / measured data to generate a reliability indication such as an uncertainty map or a confidence map for each image considered, for example, during "run-time" in a clinical setting. This type of neural network provides the network with an output that is an input-dependent random variable. By feeding the same data into this network, the posterior probability distribution can be sampled. For example, the uncertainty map can be generated as the standard deviation of such a large number of samples.

[0173] Such a neural network that generates an uncertainty map of the substrate image needs to be specially designed to process multi-energy channel images and / or sinogram data. Specifically, such a neural network can take as input at least two representations of the energy-resolved measurement data, for example, two energy-bin images or two pre-decomposed substrate images. Also, such a neural network can generate at least one uncertainty map of at least one substrate image. Such a neural network may process different material maps jointly, separately, or separately for some layers and then jointly for at least one layer.

[0174] A neural network estimator for generating a substrate image or an uncertainty map, or both, can incorporate a smoothing filter with an adjustable filter size to adjust the spatial resolution of the resulting image. It is understood that such an adjustable filter or filters can take the form of Gaussian smoothing, for example, in at least one layer. The neural network can be trained to generate a series of images with varying resolution characteristics when changing one or more parameters of the adjustable filter or filters. After training, the neural network can be used to generate images of varying resolution by adjusting at least one parameter of the adjustable filter. Such adjustable filters can be applied with different filter characteristics, such as kernel size, to different substrate images to achieve the desired spatial resolution and noise characteristics in each material image.

[0175] A Bayesian neural network can be trained by minimizing the discrepancy between the output distribution when given a training input image and the distribution of the training output images (also called training labels). Such a discrepancy can be measured by a mean squared error, a Kullback-Leibler divergence, or a Wasserstein distance. The concepts of "training input images" and "training output images" are non-limiting and should be understood to refer to representations of image data that can be, for example, bin images or sinograms, or base images or sinograms.

[0176] Such a probabilistic neural network can be based, for example, on random dropout where the network connections are made with a probability that can be fixed or learned from data (Figure 13). In another embodiment of the present invention, the probabilistic neural network can be based on additive noise insertion after at least one network layer (Figure 14). This additive noise insertion can also be replaced with multiplicative noise insertion or other types of noise insertion.

[0177] In another embodiment of the present invention, the probabilistic neural network is implemented as a variational autoencoder (Figure 15). This variational autoencoder first encodes the data input vector into the parameters of a probability distribution of latent probabilistic variables such as normal distribution parameters. Next, a set of posterior samples of the latent variables are drawn from the latent distribution and processed by the decoder to obtain the resulting posterior observations. These posterior observations are used to calculate the posterior mean (final result) and the posterior variance (uncertainty map of the result).

[0178] In particular, this discrepancy or loss function used to train the neural network is optimally adapted to the context of spectral CT material decomposition by treating different basis components differently, reflecting their different noise levels and potentially different clinical importance. As an example, the basis images may be transformed by a basis change before the data discrepancy is calculated. For example, the data discrepancy can be calculated by comparing the basis images from the training set with the basis projections generated as the output from the network, and in another example, the data discrepancy can be calculated by converting the basis images into a set of monoenergetic images and comparing these between the training set and the network output. Depending on which type of image is used to calculate the data discrepancy, the performance of the neural network's noise removal method can be optimized for the types of images that are of interest to show to the end user. In another example, the mathematical function that calculates the data discrepancy can weight different linear combinations of the basis images differently to obtain greater noise suppression in types of images where low noise is more important compared to types of images where unbiasedness is more important. For example, to accurately evaluate the material composition of a sample, it may be more important to achieve unbiasedness in a map of the effective atomic number, while it may be preferable to minimize noise in a 70 keV monoenergetic image.

[0179] As part of or in addition to generating an uncertainty map, it is possible to generate uncertainty estimates of one or more derived image features, such as radiographic image features, using the disclosed method. Examples of such features include the volume of a lesion, the average density of a region, the average effective atomic number of a region, or the standard deviation or another measure of heterogeneity across a region. To generate error estimates of such derived features, a set of image realizations can be generated using a probabilistic neural network. And the uncertainty of the feature can be determined, for example, as the standard deviation of these realizations.

[0180] For example, it is actually difficult to perform accurate basis decomposition using two or more basis functions, and artifacts, bias, and excessive noise may occur. Also, such basis decomposition may require large-scale calibration measurements and data preprocessing to obtain accurate results. In general, basis decomposition into more basis functions may be technically more difficult than decomposition into fewer basis functions.

[0181] For example, it may be difficult to perform calibration with sufficient accuracy to perform a three-basis decomposition with less image bias and artifacts compared to a two-basis decomposition. Also, it may be difficult to find a material decomposition algorithm that can perform a three-basis decomposition on noisy data without generating overly noisy basis images. That is, while it may be difficult to achieve the theoretical lower limit of basis image noise given by the Cramér-Rao lower bound, it may be easy to achieve this lower limit when performing a two-basis decomposition.

[0182] As an example, the amount of information required to generate a larger number of basis image representations may be extractable from a set of multiple basis image representations, each having a smaller number of basis image representations. For example, the information required to generate three basis decompositions into the sinograms of water, calcium, and iodine may be extractable from a set of three two-basis decompositions (water-calcium decomposition, water-iodine decomposition, calcium-iodine decomposition).

[0183] It may be easier to accurately perform a plurality of two-basis decompositions than to accurately perform a single three-basis decomposition. This observation can be used, for example, to solve the problem of performing an accurate three-basis decomposition. As an example, energy-decomposed image data is first used to perform water-calcium decomposition, water-iodine decomposition, and calcium-iodine decomposition. Next, a convolutional neural network can be used to map the six resulting basis images, or a subset thereof, to a set of three output images including water, calcium, and iodine images. Such a network can be trained using, as input data, a set of multiple two-basis image representations and, as output data, a set of three-basis image representations, where the set of two-basis image representations and the set of three-basis image representations are generated from measured patient image data or phantom image data, or from simulated image data based on a numerical phantom.

[0184] By the method described above, the bias, artifacts, or noise of the set of three-basis image representations can be significantly reduced compared to a three-basis decomposition performed directly on the energy-decomposed image data. Alternatively, a higher-resolution image can be generated.

[0185] As an alternative or complement to the neural network, a machine learning system or method applied to the original basis images can include another machine learning system or method, such as a support vector machine or a decision tree-based system or method.

[0186] The basis material decomposition step used to generate the original basis image representation can include prior information such as volume or mass preservation constraints or nonnegativity constraint. Alternatively, such prior information can take the form of a prior image representation such as an image from a previous examination or a reconstructed image from the total count of all energy bins, and the algorithm can penalize the deviation of the decomposed basis image representation from this prior image representation. Another option is to use prior information learned from a set of learning images represented as, for example, a learned dictionary or a pre-trained convolutional neural network, or a learned subspace, i.e., a subspace of the vector space of possible images where the reconstructed image is expected to exist.

[0187] Material decomposition can be performed on projection image data or sinogram data, for example, by independently processing each measured projection ray. This processing can take the form of maximum likelihood decomposition, or maximum a posteriori decomposition where a prior probability distribution regarding the material composition within the imaged object is assumed. Also, a linear or affine transformation from a set of input counts to a set of output counts, exemplified by Alvarez (Med Phys:2324-2334), a low-order polynomial approximation such as exemplified by Lee et al. (IEEE Transactions on Medical Imaging (Volume:36, Issue:2, Feb.2017: 560-573)), a neural network estimator or a lookup table such as exemplified by Alvarez (https: / / arxiv.org / abs / 1702.01006). Alternatively, the material decomposition method may jointly process multiple rays or may constitute a one-step or two-step reconstruction algorithm.

[0188] In the paper by Chen and Li published in Optical Engineering 58(1), 013104, a method for multi - material decomposition of spectral CT data using deep neural networks is disclosed.

[0189] The paper by Poirot et al. published in Scientific Reports, Volume 9, Article number: 17709 (2019) discloses a method for generating non - contrast single - energy CT images from dual - energy CT images using convolutional neural networks.

[0190] Figure 8 is a schematic diagram showing an example of an uncertainty map according to an embodiment. While the iodine map shows the estimated value of iodine concentration, the iodine reliability map shows the reliability that the algorithm can predict the presence of iodine at each given pixel of the image. The resulting image emphasizes regions where iodine is likely to be present with a high probability, and the dark regions are areas where iodine is likely not to be present.

[0191] Figure 9 is a schematic diagram showing an example of a Bayesian or probabilistic neural network that can be used to solve the material decomposition problem. This exemplary neural network takes eight - energy - bin sinograms as input and generates the number T of material sinograms as output. The neural network is represented by the mapping Ψθ, where θ is a random parameter vector. Since this mapping depends on random parameters, applying the network to the same input data multiple times will result in different outputs. The average of such a set of outputs is used as an estimate of the material map, while the variance is used as an estimate of the uncertainty map.

[0192] Figure 10 is a schematic diagram showing an example of a neural network estimator that takes in a substance concentration map obtained from deep learning-based substance decomposition and generates a reliability map. These reliability maps highlight regions of the image where bone, soft tissue, and iodine are likely to be present respectively. As can be seen from the image, the iodine reliability map highlights the regions where there are tumors taking up iodine, but since the algorithm cannot completely rule out the presence of iodine in this region, a low but non-zero value is assigned to the spinal region as well. The application of the neural network to the substance concentration map obtained from deep learning-based substance decomposition is exemplary, and in another embodiment of the present invention, the neural network can be applied to the reconstructed image using other methods such as filtered backprojection.

[0193] Figure 11 is a schematic diagram showing a neural network for sinogram spatial substance decomposition based on the unrolled iterative substance decomposition method. This method is based on a predefined number of iterative noise removal methods, and the update step in each iteration is replaced by a neural network. In this exemplary embodiment, the gradient descent algorithm is unrolled, which means that the gradient is calculated at each iteration step and taken as the additional input to the next network, thereby providing the network with information about the physics and statistical models underlying noise removal.

[0194] Figure 12 is a schematic diagram showing an example of a neural network that takes in an energy bin sinogram as input and generates a reconstructed basis substance image. As an example, a detector that generates 8 energy bin sinograms can be used, and these are fed into the neural network as 8 input channels. In this example, 3 output channels correspond to 3 basis images (bone, soft tissue, iodine).

[0195] FIG. 13 is a schematic diagram showing an example of a probabilistic neural network that takes an energy binogram as input and generates a reconstructed base material image based on random dropout. Each time the network is applied to a set of input binograms, a random selection of the network's weights is randomly set to zero, giving a random network output.

[0196] FIG. 14 is a schematic diagram showing an example of a probabilistic neural network that takes an energy binogram as input and generates a reconstructed base material image based on the insertion of additive noise. By adding noise values to the nodes of each layer of the network, the output base image becomes a random function of the input binogram. In this way, by applying the network to the same input image multiple times, a random distribution of the output images can be given, and the network can be trained so that this distribution matches the posterior distribution of the image given the input data.

[0197] FIG. 15 is a schematic diagram showing an example of a probabilistic neural network that takes an energy binogram as input and generates a reconstructed base material image based on a variational autoencoder, which is composed of an encoder, a random feature generator, and a decoder. The encoder converts the energy binogram into a feature mean vector and a variance vector. These vectors are used as parameters for randomly generating a random feature vector. This vector can be selected, for example, as a sample from a multivariate normal distribution having a mean and a variance given by the output vector from the encoder section. This random feature vector is used as the input to the decoder network to generate bone, soft tissue, and iodine base images. In this way, the entire variational autoencoder functions as a probabilistic neural network that maps the input binogram to a set of output base images, and these output images are not deterministic but are sampled from a statistical distribution. The network can be trained so that this distribution matches the posterior distribution of the image when the input data is given.

[0198] Non-limiting, exemplary functions of mapping using deep neural networks include the following. a. The neural network learns to approximate the posterior probability distribution of the solution. b. The neural network functions as a random function. That is, for the same observation "y" (network input), the network can provide K different solutions (network outputs, X1, X2,..., X K ). · In a Bayesian network, this is achieved, for example, by random dropout. · In the post-processing of a variational encoder, there are random latent parameters Z. · In a generative network, there are random input parameters Z. c. To learn the posterior probability distribution of the solution, a statistical distance (such as KL divergence, Wasserstein distance, etc.) is used in the learning loss of the neural network.

[0199] According to a second main aspect, a non-limiting example of a corresponding system for determining a reliability indication for machine learning image reconstruction, such as deep learning image reconstruction in computed tomography (CT), is provided. The system for determining the reliability indication is configured to acquire energy-resolved X-ray data. The system is further configured to process the energy-resolved X-ray data based on at least one machine learning system, such as one or more neural networks, to obtain a representation of the posterior probability distribution of at least one reconstructed basis image or its image features. The system is also configured to generate one or more reliability indications for the at least one reconstructed image, or at least one derived image derived from the at least one reconstructed basis image, or the image features of the at least one reconstructed basis image or the at least one derived image, based on the representation of the posterior probability distribution.

[0200] As described above, the machine learning image reconstruction may be, for example, deep learning image reconstruction, and the at least one machine learning system may include at least one neural network.

[0201] As an example, one or more reliability displays may include an error estimate value or a measure of statistical uncertainty for at least one point in the at least one reconstructed basis image, and / or an error estimate value or a measure of statistical uncertainty for at least one image measurement value derivable from the at least one reconstructed basis image.

[0202] Optionally, the system may be configured to generate the one or more reliability displays in the form of one or more uncertainty maps for the at least one reconstructed basis image, or at least one derivative image derived from the at least one reconstructed basis image, or its image features.

[0203] In a particular embodiment, the system may be configured to generate the one or more reliability displays in the form of a reliability map for a reconstructed material-selective X-ray image for computed tomography (CT).

[0204] As an example, the system is further configured to perform image reconstruction based on material decomposition and / or image reconstruction by machine learning based on an energy-bin sinogram as input to generate the at least one reconstructed basis image or its image features.

[0205] Optionally, the system may be configured to generate a reliability map to highlight portions of the reconstructed material-selective X-ray image that can be determined with a reliability above a threshold by the machine learning image reconstruction.

[0206] According to a complementary aspect, a non-limiting example of a corresponding system for generating an uncertainty map for machine learning image reconstruction, such as deep learning image reconstruction in spectral CT, is provided. A system for generating an uncertainty map is configured to acquire energy-resolved X-ray data. The system is further configured to process the energy-resolved X-ray data based on at least one machine learning system, such as one or more neural networks, so that an expression of a posterior probability distribution of at least one basis image or its image features can be obtained. The system is also configured to generate one or more uncertainty maps for at least one reconstructed image, or derived image, or its image features based on the expression of the posterior probability distribution.

[0207] According to an additional aspect, a corresponding image reconstruction system is provided that includes such a system for determining a confidence indication and / or such a system for generating an uncertainty map for deep learning image reconstruction.

[0208] According to another aspect, an overall X-ray imaging system including such an image reconstruction system is provided.

[0209] According to yet another aspect, a corresponding computer program and computer program product are provided.

[0210] In an exemplary embodiment of the present invention, the step or configuration of acquiring or obtaining energy-resolved X-ray (image) data is performed by a CT imaging system.

[0211] In an exemplary embodiment of the present invention, the step or configuration of acquiring or obtaining energy-resolved X-ray (image) data is performed by an energy-resolved photon counting detector, also referred to as a multi-bin photon counting X-ray detector.

[0212] Alternatively, the step or configuration for obtaining energy-dispersive X-ray (image) data is performed by multi X-ray tube acquisition, low-speed or high-speed kV switching acquisition, multi-layer detector or split filter acquisition.

[0213] In an exemplary embodiment of the present invention, the machine learning can include a machine learning architecture and / or algorithm based on a convolutional neural network. Alternatively, the machine learning architecture and / or algorithm may be based on a support vector machine or a decision tree-based method.

[0214] In an exemplary embodiment of the present invention, the convolutional neural network may be based on a residual network (ResNet), a residual encoder-decoder, a U-Net, an AlexNet, or a LeNet architecture. Alternatively, the machine learning algorithm based on the convolutional neural network may be based on an unrolled optimization method based on a gradient descent algorithm, a primal-dual algorithm, or an alternating direction method of multipliers (ADMM) algorithm.

[0215] In an exemplary embodiment of the present invention, the convolutional neural network includes at least one forward projection or at least one back projection as part of the network architecture.

[0216] For better understanding, next, exemplary and non-limiting examples of the proposed technology are described.

[0217] As an example, for instance, by introducing a separate machine learning-based estimator to generate estimates of the bias, variance, and / or covariance of different reconstructed basis images, it is possible to determine a reliability indication such as an uncertainty map or a confidence map. These estimates are propagated to generate uncertainty maps of derivative images such as virtual monochromatic images and virtual non-contrast images.

[0218] There are various ways to generate these maps. One method is based on bootstrapping, which trains a neural network on a resampled training dataset. For example, a random set of training samples including input and output training data can be sampled with replacement and used for training the neural network. By repeating this procedure, an ensemble of neural networks can be obtained, and by processing the input image data using each of these networks, an ensemble of output images or output image data representations can be obtained. The variation or uncertainty within this ensemble of output images can be measured, for example, as the standard deviation per pixel across the distribution of the image. Next, a second neural network can be trained to map the measured image data to the resulting uncertainty or the distribution of the resulting image values. However, as a method with a lower computational load, there is the variational autoencoder. This neural network architecture that maps data to a low-dimensional feature space in the intermediate layer can be trained to sample the posterior probability distribution of the image results of machine learning image reconstruction procedures such as deep learning reconstruction methods under study. The posterior probability distribution of the low-dimensional intermediate latent feature quantities of the variational autoencoder can be obtained from the encoder function. Next, using the decoder, the corresponding posterior probability distribution of the reconstructed image can be obtained.

[0219] One way to represent a probability distribution is to provide random samples, also called Monte Carlo samples, from the distribution.

[0220] One way to process an image representation to obtain a representation of a probability distribution is to apply a stochastic neural network to the image representation. A stochastic neural network is a neural network that includes random elements or components such that the probability distribution outputs a random function that depends on the input.

[0221] As an example, the stochastic neural network can provide one or more Monte Carlo samples of the probability distribution.

[0222] In another exemplary embodiment, a deterministic neural network can be trained to provide a measure of the probability distribution of a posterior random variable, such as an image or image feature.

[0223] For example, the measure of the probability distribution of the posterior random variable can be a mean variance, a covariance, a standard deviation, a skewness, a kurtosis, or a combination thereof.

[0224] For example, it can be used to first create a statistical estimator of the uncertainty of an image or the posterior probability distribution, such as a Monte Carlo estimator, a Markov Chain Monte Carlo estimator, a bootstrap estimator or a stochastic neural network estimator, and then generate training data for training a deterministic neural network that predicts one or more measures of the posterior probability distribution.

[0225] As an example, the base image can be a map of the density of physical substances such as water, soft tissue, calcium, iodine, gadolinium or gold. Also, the base image can be, for example, a map of a virtual or imaginary substance representing physical properties such as a map of the Compton scatter cross-section, photoelectric absorption cross-section, density or effective atomic number.

[0226] For example, the reliability indication can be an error estimate value or a measure of statistical uncertainty for at least one point in one or more reconstructed images. Also, the reliability indication can be an error estimate value or a measure of statistical uncertainty for at least one image measurement value derivable from at least one image.

[0227] For example, the error estimate value or the measure of statistical uncertainty can be the upper limit of the error, the tower limit of the error, the standard deviation, the variance, the mean absolute error.

[0228] For example, the image measurement derivable from at least one image can be a measurement of the dimension of a feature, area, volume, degree of non-uniformity, shape or irregularity measurement, composition measurement, or measurement of the concentration of a substance.

[0229] For example, the image measurement derivable from at least one image can be a radiological feature, for example a standardized radiological feature.

[0230] In an exemplary embodiment of the present invention, the processing of the energy-resolved X-ray data includes forming at least one base sinogram or reconstructed base image and processing the image based on a neural network.

[0231] For example, the neural network can be a convolutional neural network.

[0232] For example, in order to have sufficient flexibility to fit the training data, the neural network can be a deep neural network with at least five layers.

[0233] As an example, methods for estimating the error map include the Markov chain Monte Carlo method and the approximate Bayesian estimation method based on variational dropout.

[0234] The paper "Uncertainty modelling in deep learning for safer neuroimage enhancement: Demonstration in diffusion MRI" by Tanno et al., Neuroimage 225, October 9, 2020, is about a method using a Bayesian neural network with variational dropout to generate uncertainty maps for diffusion magnetic resonance image enhancement.

[0235] The paper "Uncertainty Quantification in Deep MRI Reconstruction" by Edupuganti et al., IEEE Transactions on Medical Imaging, Vol. 40, No. 1, January 2021, is about a method that uses a variational autoencoder as a post - processing step to generate posterior samples of the results and construct an uncertainty map using them for magnetic resonance image (MRI) reconstruction from undersampled data. However, the authors require a pre - processed reconstruction calculated without deep learning and are further related to MRI.

[0236] The paper "Deep posterior sampling: Uncertainty quantification for large scale inverse problems" by Adler and Oktem in Medical Imaging with Deep Learning 2019 is about a method for quantifying the uncertainty of X-ray computed tomography images, considering a generative neural network for post-processing that samples the posterior probability distribution of the results. However, the authors require pre-processed reconstructions, and this paper does not disclose a method for quantifying the error of a specific material density map. This paper is further about MRI.

[0237] The specification of US20200294284A1 is about a method for generating uncertainty information regarding a reconstructed image. However, this paper does not disclose a method for quantifying the error of a specific material density map and considers energy-integrated CT without using any energy decomposition data at all. This further means that effective material basis decomposition cannot be performed to generate a basis image.

[0238] The approach of this application mainly considers energy decomposition (spectrum) CT with results of multi-energy and multi-material.

[0239] For example, photon-counting CT represents a high-dimensional problem, requires appropriate scaling, and cross-material and cross-energy information affects posterior sampling.

[0240] The above three articles are based on post-processing neural networks and do not solve the reconstruction problem with deep learning.

[0241] With the proposed technology, basis image reconstruction and uncertainty mapping can be solved with machine learning such as deep learning.

[0242] It will be understood that the mechanisms and arrangements described herein can be implemented, combined, and rearranged in various ways.

[0243] For example, an embodiment may be implemented in hardware, at least partially implemented in software for execution by appropriate processing circuitry, or a combination thereof.

[0244] The steps, functions, procedures, and / or blocks described herein can be implemented in hardware using conventional techniques including both general-purpose and application-specific circuits such as discrete circuits and integrated circuit technology.

[0245] Alternatively, or in addition, at least some of the steps, functions, procedures, and / or blocks described herein may be implemented in software, such as a computer program, for execution by an appropriate processing circuit such as one or more processors or processing units.

[0246] This can be implemented, for example, as part of a computer-based image reconstruction system.

[0247] FIG. 16 is a schematic diagram showing an example of computer implementation according to an embodiment. In this particular example, system 200 includes a processor 210 and a memory 220, the memory containing instructions executable by the processor, whereby the processor is operable to execute the steps and / or actions described herein. The instructions are typically configured as computer programs 225, 235 and may be pre-configured in the memory 220 or downloaded from an external memory device 230. Optionally, system 200 may include an input / output interface 240 interconnected to one or more processors 210 and / or memory 220 to enable input and / or output of associated data such as input parameters and / or result output parameters.

[0248] In certain embodiments, the memory includes such a set of instructions executable by a processor, whereby the processor is operable to generate a reliability indication, such as an uncertainty map, for deep learning-based image reconstruction in CT imaging.

[0249] The term "processor" should be interpreted in a general sense as any system or device capable of executing program code or computer program instructions to perform a particular processing, decision-making, or computational task.

[0250] Thus, a processing circuit including one or more processors is configured to perform a well-defined processing task as described herein when executing a computer program.

[0251] The processing circuit need not be specialized only for the execution of the steps, functions, procedures, and / or blocks described above, and may perform other tasks.

[0252] The technology also provides a computer program product including computer-readable media 220, 230 storing such a computer program.

[0253] As an example, software or computer programs 225, 235 can be implemented as computer program products and are typically carried or stored on computer-readable media 220, 230, particularly non-volatile media. The computer-readable media can include one or more removable or non-removable memory devices, including but not limited to read-only memory (ROM), random access memory (RAM), compact disc (CD), digital versatile disc (DVD), Blu-ray disc, universal serial bus (USB) memory, hard disk drive (HDD) storage devices, flash memory, magnetic tape, or any other conventional memory device. Thus, a computer program can be loaded into the operating memory of a computer or equivalent processing device for execution by its processing circuitry.

[0254] When the processing flow is executed by one or more processors, it can be regarded as a computer action flow. The corresponding device, system, and / or apparatus can be defined as a group of functional modules, and each step executed by the processor corresponds to a functional module. In this case, the functional module is implemented as a computer program executed on the processor. Thus, the device, system, and / or apparatus can alternatively be defined as a group of functional modules, and the functional module is implemented as a computer program executed on at least one processor.

[0255] Thus, a computer program resident in the memory can be organized as appropriate functional modules configured to execute at least a portion of the steps and / or tasks described herein when executed by the processor.

[0256] Alternatively, most of the modules can be realized as hardware modules or replaced with hardware. Whether to use software or hardware is purely a matter of implementation choice.

Description of Symbols

[0257] 10: X-ray source 11: Gantry 12: Table 13: Isocenter 15: CT 20: X-ray detector system 21: Detector element 25: Analog processing circuit 30: Image processing system or device 35: Image processing system 40: Digital processing circuit 41: X-ray control device 42: Gantry control device 43: Table control device 44: Detector controller 45: Memory 50: Computer 60: Operator console 100: X-ray imaging system 200: System 210: Processor 220: Memory 225, 235: Computer program 230: External memory device 240: Input / output interface 301: Digital-to-analog converter 302: Comparator 303: Digital counter

Claims

1. A method for determining one or more reliability displays for machine learning image reconstruction in computed tomography (CT), comprising: obtaining energy-resolved X-ray data (step S1); performing material decomposition-based image reconstruction by machine learning image reconstruction to generate at least one reconstructed basis image or its image features based on the obtained energy-resolved X-ray data (step S2a); processing the energy-resolved X-ray data based on at least one machine learning system to generate a representation of the posterior probability distribution of the at least one reconstructed basis image or its image features (steps S2, S2b); generating one or more reliability displays for (1) the at least one reconstructed basis image, or (2) at least one derived image derived from the at least one reconstructed basis image, or (3) image features of the at least one reconstructed basis image or the at least one derived image, based on the representation of the posterior probability distribution (step S3), wherein the step (S3) of generating the one or more reliability displays includes determining an uncertainty or reliability map of individual basis material images and a covariance between different basis material images, and propagating the uncertainty or the reliability map using an equation or algorithm for propagation of the uncertainty to obtain an uncertainty map of the derived image.

2. The method according to claim 1, wherein the machine learning image reconstruction is deep learning image reconstruction and the at least one machine learning system includes at least one neural network.

3. The method according to claim 1, wherein the representation of the posterior probability distribution includes at least one of mean variance, covariance, standard deviation, skewness, and kurtosis.

4. The method according to claim 1, wherein the one or more reliability displays include an error estimate value or a measure of statistical uncertainty for at least one point in the at least one reconstructed basis image, and / or an error estimate value or a measure of statistical uncertainty for at least one image measurement value derivable from the at least one reconstructed basis image.

5. The method according to claim 4, wherein the error estimate value or the measure of statistical uncertainty includes at least one of an upper limit of error, a lower limit of error, standard deviation, variance, or mean absolute error.

6. The method according to claim 4, wherein the at least one image measurement value includes at least one selected from dimensional measurement of features, area measurement, volume measurement, measurement value of degree of non-uniformity, measurement value of shape or irregularity, measurement value of composition, and measurement value of concentration of substances.

7. The method according to claim 1, wherein the one or more reliability displays include one or more uncertainty maps for the at least one reconstructed basis image, or at least one derived image derived from the at least one reconstructed basis image, or image features thereof.

8. The method according to claim 1, wherein the step (S3) of generating one or more reliability displays includes generating a reliability map of a reconstructed material selection X-ray image for computed tomography (CT).

9. The method according to claim 8, wherein the reliability map is generated to emphasize a portion of the reconstructed material selection X-ray image for which the machine learning image reconstruction can be determined with a reliability equal to or higher than a predetermined threshold.

10. The method according to claim 1, wherein the step (S3) of generating one or more reliability displays includes generating one or more reliability maps by a neural network that takes as input a substance concentration map obtained from substance decomposition based on deep learning.

11. The method according to claim 10, wherein the step (S2a) of performing image reconstruction based on substance decomposition and / or machine learning image reconstruction includes generating the at least one reconstructed basis image or image feature by a neural network that receives an energy bin sinogram as input.

12. The method according to claim 1, wherein at least one basis substance image is generated together with at least one uncertainty map, the uncertainty map being a representation of the uncertainty or error estimate of the at least one basis substance image, and the at least one basis substance image and the at least one uncertainty map can be presented to the user as separate images or in combination.

13. The method according to claim 12, wherein the at least one uncertainty map can be presented as an overlay on the at least one basis substance image, or the at least one uncertainty map can be presented by a distortion filter for the at least one basis substance image.

14. The step (S2; S2b) of processing the energy-dispersive X-ray data based on at least one machine learning system to generate a representation of a posterior probability distribution includes, by a neural network, generating samples of a posterior probability distribution given the acquired energy-dispersive X-ray data, and the step (S3) of generating one or more reliability indicators includes generating an uncertainty map as a standard deviation over a plurality of samples, the method according to claim 1.

15. The step (S2; S2b) of processing the energy-dispersive X-ray data based on at least one machine learning system to generate a representation of a posterior probability distribution applies a neural network implemented as a variational autoencoder to encode an input data vector into parameters of a probability distribution of latent probability variables, and extracting a set of posterior samples of latent probability variables from this probability distribution for processing by a corresponding decoder to obtain posterior observations, the method according to claim 1.

16. The step (S3) of generating the one or more reliability indicators includes generating at least one map of a variance or standard deviation of at least one basis coefficient associated with the at least one reconstructed basis image, and / or at least one map of a covariance or correlation coefficient of at least one set of basis functions associated with the at least one reconstructed basis image, the method according to claim 1.

17. The representation of the posterior probability distribution is specified by the mean and variance of a plurality of image features, the method according to claim 1.

18. A system (30, 40, 50, 200) for determining one or more reliability displays for machine learning image reconstruction in computed tomography (CT), wherein the system is configured to acquire energy-resolved X-ray data, and the system (30, 40, 50, 200) is configured to perform material decomposition-based image reconstruction by machine learning image reconstruction based on the energy-resolved X-ray data including an energy-bin sinogram as an input, and to generate at least one reconstructed basis image or image features thereof, the system (30, 40, 50, 200) is further configured to process the energy-resolved X-ray data based on at least one machine learning system to obtain a representation of the posterior probability distribution of at least one reconstructed basis image or image features thereof, the system (30, 40, 50, 200) is also configured to generate one or more reliability displays for the at least one reconstructed basis image, or at least one derived image derived from the at least one reconstructed basis image, or image features of the at least one reconstructed basis image or the at least one derived image based on the representation of the posterior probability distribution, the system is configured to determine an uncertainty or reliability map of individual basis material images and a covariance between different basis material images, and to enable propagation of the uncertainty or the reliability map to generate an uncertainty map of a derived image, system (30, 40, 50, 200).

19. The system (30, 40, 50, 200) according to claim 18, wherein the machine learning image reconstruction is deep learning image reconstruction and the at least one machine learning system includes at least one neural network.

20. The system (30, 40, 50, 200) according to claim 18, wherein the one or more reliability displays include an error estimate value or a measure of statistical uncertainty for at least one point in the at least one reconstructed basis image and / or an error estimate value or a measure of statistical uncertainty for at least one image measurement value derivable from the at least one reconstructed basis image.

21. The system (30, 40, 50, 200) according to claim 18, is configured to generate the one or more reliability displays in the form of one or more uncertainty maps for the at least one reconstructed basis image, or at least one derivative image derived from the at least one reconstructed basis image, or image features thereof.

22. The system (30, 40, 50, 200) according to claim 18, is configured to generate the one or more reliability displays in the form of a reliability map for a reconstructed material selection X-ray image for computed tomography (CT).

23. The system (30, 40, 50, 200) according to claim 18, is further configured to perform image reconstruction based on material decomposition and / or machine learning image reconstruction based on an energy bin sinogram as input, to generate the at least one reconstructed basis image or image features thereof.

24. An image reconstruction system comprising the system (30, 40, 50, 200) according to any one of claims 18 to 23, for determining one or more reliability displays for machine learning image reconstruction in CT.

25. An X-ray imaging system (100) comprising the image reconstruction system according to claim 24.

26. A computer program (225, 235) which, when executed by at least one processor (30, 40, 50, 210) associated with a computed tomography system (100), includes instructions for causing the at least one processor (30, 40, 50, 210) to execute the method according to any one of claims 1 to 17.

27. A computer program product comprising a non-transitory computer-readable storage medium (220, 230) carrying the computer program (225, 235) according to claim 26.

Images