Related methods and systems for x-ray imaging

Abstract

A method and corresponding system for image reconstruction based on energy-resolved x-ray data is provided. The method includes collecting (S1) at least one representation of energy-resolved X-ray data, and performing (S2) at least two basis material decompositions based on the at least one representation of the energy-resolved X-ray data to generate at least two sets of original basis image representations. The method further includes (S3) obtaining or selecting at least two basis image representations from the at least two sets of original basis image representations, and (S4) processing the obtained or selected basis image representations using machine learning-based data processing to generate at least one representation of output image data.

Description

[Technical field]

[0001] The present disclosure relates to X-ray technology and X-ray imaging, and corresponding imaging reconstruction and imaging tasks. In particular, the present disclosure relates to methods and systems for image reconstruction, corresponding X-ray imaging systems, and corresponding computer programs and computer program products. [Background technology]

[0002] Radiation imaging, such as X-ray imaging, has been used for many years in medical applications and non-destructive testing.

[0003] Typically, an X-ray imaging device contains an X-ray source and an X-ray detector array, which consists of one or more detector elements (independent means of measuring the intensity / fluorescence of X-rays). X-rays emitted from the X-ray source pass through the object and impinge on the detector array, where the detected X-rays are recorded. Some materials absorb X-rays, while others do not, forming an image of the object.

[0004] The challenge of an X-ray imaging detector is to extract maximum information from the detected X-rays and provide an image input in which an object or subject is described in terms of density, composition, structure, etc.

[0005] In a typical medical X-ray imaging system, X-rays are generated by an X-ray tube. The energy spectrum of a typical medical X-ray tube is broad, ranging from 0 to 160 keV. Thus, a detector typically detects X-rays of varying energy. It may be useful to provide a brief overview of an overall X-ray imaging system with reference to FIG. 1. In this illustrative but non-limiting example, the X-ray imaging system 100 essentially comprises an X-ray source 10, an X-ray detector system 20, and an associated image processing system or device 30. The X-ray detector system 20 is configured to record radiation from the X-ray source 10, which is optionally focused by any X-ray optics and passes through an object, subject, or part thereof. The X-ray detector system 20 is connectable to the image processing system 30 via suitable analog and readout electronics at least partially integrated in the X-ray detector system 20, which allows image processing and / or image reconstruction by the X-ray detector.

[0006] As an example, an X-ray computed tomography (CT) system includes an X-ray source and an X-ray detector that project a projection image of a subject or object through at least 180 degrees. This is most commonly achieved by mounting the source and detector on a support that can be rotated around the subject or object. An image that includes projections recorded on different detector elements for different view angles is called a sinogram. In the following, the collection of projections recorded on different detector elements for different view angles is called a sinogram even if the detector is two-dimensional, and the sinogram is made into a three-dimensional image.

[0007] A further development of X-ray imaging is energy-resolved X-ray imaging (also called spectral X-ray imaging), in which the X-ray transmission is measured for several different energy levels. This can be achieved by rapidly switching the source between two different emission spectra, by using two or more X-ray sources that emit different X-ray spectra, or by using an energy-discriminating detector that measures the incident radiation at two or more energy levels. An example of such a detector is the multi-bin photon-counting detector, where each recorded photon produces a current pulse that is compared to a set of thresholds to count the number of incident photons in each of a number of energy bins.

[0008] Spectral X-ray projection measurements provide projection images at each energy level. The weighted sum of these projection images is described in "Tapiovaara and Wagner, "SNR and DQE analysis of broad spectrum X-ray imaging", Phys. Med. Biol. 30, 519".

[0009] Another technique enabled by energy-resolved x-ray imaging is basis material decomposition, which exploits the fact that all materials composed of elements with low atomic numbers, such as human tissue, have a linear attenuation coefficient μ(E) whose energy dependence can be approximately expressed as a linear combination of two (or more) basis functions. μ(E)=a1f1(E)+a2f2(E) where fi is the basis function and ai is the corresponding basis coefficient. If one or more elements with atomic numbers high enough that their k-edges lie in the energy range used for imaging are present in the imaged volume, one basis function needs to be added for each such element. In the field of medical imaging, such k-edge elements might typically be iodine or gadolinium, substances used as contrast agents.

[0010] Basis material decomposition is described in Alvarez and Macovski, "Energy-selective reconstructions in X-ray computerised tomography", Phys. Med. Biol. 21, 733. In basis material decomposition, the integral of each basis coefficient is estimated from the measured data for each projection ray l from the source onto a detector element.

[0011]

number

[0012]

number

[0013] Under the assumption that the counts in each bin are Poisson-distributed random variables, Ai can be estimated using maximum likelihood methods (see Roessl and Proksa, K-edge imaging in x-ray computed tomography using multi-bin photon counting detectors, Phys. Med. Biol. 52 (2007), 4679-4696).

[0014]

number

[0015] The resulting estimated basis coefficient line integrals for each projection line

[0016]

number

[0017] However, a well-known limitation of this technique is that the variance of the estimated line integrals typically increases with the number of bases used in the basis decomposition, which, among other things, results in an undesirable trade-off between improved tissue quantification and increased image noise.

[0018] Moreover, accurate decompositions using more than two basis functions can be difficult to implement in practice and can introduce artifacts, bias or excessive noise. Such decompositions may also require extensive calibration measurements and data preprocessing to obtain accurate results.

[0019] There is also a general need for improvements regarding x-ray image reconstruction. [Prior art documents] [Patent documents]

[0020] [Patent Document 1] U.S. Patent No. 9,036,885 Summary of the Invention

[0021] In general, it is desirable to provide improvements with respect to image reconstruction for x-ray imaging applications.

[0022] It is a particular object to provide improvements with respect to basis decompositions for X-ray imaging.

[0023] It is a particular object to provide an improved method of image reconstruction.

[0024] It is also an object to provide an improved system for image reconstruction.

[0025] Another object is to provide an improved overall X-ray imaging system.

[0026] It is a further object to provide corresponding computer programs and computer program products.

[0027] These and other objects may be achieved by one or more embodiments of the proposed technology.

[0028] According to a first aspect, there is provided a method for image reconstruction based on energy resolved X-ray data, comprising: collecting at least one representation of energy resolved X-ray data; performing at least two basis material decompositions based on the at least one representation of the energy resolved x-ray data to generate at least two original basis image representation sets; Obtaining or selecting at least two basis image representations from at least two of the original set of basis image representations; The obtained or selected basis image representations are subjected to data processing based on machine learning to generate at least one representation of output image data.

[0029] According to a second aspect, a system for image reconstruction based on energy resolved X-ray data is provided. The system is configured to obtain at least one representation of the energy resolved X-ray data. The system is configured to perform at least two basis material decompositions based on the at least one representation of the energy resolved X-ray data to generate at least two sets of basis image representations. The system is further configured to obtain or select at least two basis image representations from the set of at least two basis image representations. The system is also configured to perform data processing based on machine learning on the obtained or selected basis image representations to generate at least one representation of output image data.

[0030] According to a third aspect, there is provided a system for image reconstruction based on energy resolved X-ray data, the system comprising: an input for receiving at least one representation of energy resolved X-ray data; a basis material decomposition system configured to perform at least two basis material decompositions based on at least one representation of the energy resolved x-ray data to generate at least two sets of basis image representations; a trained machine learning system having a configuration determined based on input-output-paired training data, the trained machine learning system being configured to receive at least two basis image representations from at least two of the sets of basis image representations, and to perform machine learning-based data processing on the received basis image representations to generate at least one representation of output image data; an output for outputting at least one representation of the output image data; Equipped with.

[0031] According to a fourth aspect there is provided an X-ray imaging system comprising a system for image reconstruction according to the second or third aspect.

[0032] According to a fifth aspect, there is provided a computer program product comprising a non-transitory computer readable medium having stored thereon a computer program comprising instructions which, when executed by a processor, cause the processor to perform a number of steps, the number of steps being: collecting at least one representation of energy resolved X-ray data; performing at least two basis material decompositions based on said at least one representation of the energy resolved x-ray data to generate at least two sets of original basis image representations; Extracting at least two basis image representations from at least two of the original sets of basis image representations; The method includes data processing the extracted base image representations based on machine learning to generate at least one representation of output image data.

[0033] The proposed technique provides significantly improved image reconstruction by using basis image representations obtained from two or more basis material decompositions as input to machine learning-based data processing. The technical solution provided by the proposed technique can, for example, reduce bias, artifacts, and / or noise in the output image and the corresponding image representations.

[0034] The inventors have recognized that it is possible to significantly improve and facilitate the efficiency, operation and / or results of machine learning processes and / or systems in performing X-ray image reconstruction. Instead of following the prevailing trend of using machine learning on raw image data, the inventors have recognized that improved image results can be achieved when utilizing basis image representations resulting from two or more four-basis material decompositions as inputs to the machine learning process and / or system. For example, improvements may include reduced bias, artifacts and / or noise, or improved resolution. Another advantage is that the proposed technique reduces the amount of training data required to obtain reliable and quality image reconstructions. Embodiments of the present invention, together with further objects and advantages thereof, may best be understood by referring to the following description taken in conjunction with the accompanying drawings, in which: [Brief description of the drawings]

[0035] [Figure 1] FIG. 1 is a schematic diagram showing an example of an entire X-ray imaging system. [Diagram 2] FIG. 2 is a schematic diagram illustrating another example of an X-ray imaging system. [Diagram 3] 1 is a schematic block diagram of a CT system as an illustrative example of an X-ray imaging system. [Figure 4] FIG. 2 is a schematic diagram illustrating another example of relevant portions of an X-ray imaging system. [Diagram 5] 1 is a schematic diagram of a photon counting circuit and / or device according to the prior art; [Figure 6]1 is a schematic flow chart illustrating an example of a method for image reconstruction based on energy resolved X-ray measurement data according to an embodiment. [Figure 7] FIG. 13 is a schematic diagram illustrating an example of a supplementary representation of machine learning based image reconstruction according to one embodiment. [Figure 8] FIG. 1 is a schematic block diagram illustrating an example of an X-ray image reconstruction system according to one embodiment. [Figure 9] FIG. 1 is a schematic diagram illustrating an example of a proposed scheme and / or architecture for deep learning debiase, according to one embodiment. [Figure 10] FIG. 2 is a schematic diagram showing an example of residual blocks (ResBlocks). [Figure 11] FIG. 1 is a schematic diagram illustrating an example of a bias-removing deep learning scheme and / or architecture that includes a series of residual blocks (ResBlocks) following biased results from a two-material decomposition. [Figure 12] FIG. 1 is a schematic diagram showing an example of a sample simulated data set, where different parts of a Shepp Logan phantom mimic different materials. [Figure 13] FIG. 11 is a schematic diagram showing an example of qualitative results of the deep learning based bias mitigation proposed by Shep Logan on a simulated phantom. [Figure 14] FIG. 1 is a schematic diagram illustrating an exemplary embodiment of the proposed invention in which a machine learning system maps a material decomposition sinogram to a reconstructed image. [Figure 15] FIG. 1 is a schematic diagram illustrating an exemplary embodiment of the proposed invention in which a one-step image reconstruction system provides material decomposition images that are used as input to a machine learning system. [Figure 16] 1 is a schematic diagram illustrating an exemplary embodiment of the proposed invention, in which a two-step image reconstruction method including a basis material decomposition system followed by an image reconstruction system provides material decomposed images that are used as input to the machine learning system. [Figure 17]FIG. 1 is a schematic diagram illustrating an example of a computer implementation according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0036] For purposes of understanding, it may be useful to follow with an introductory description of a non-limiting example of an overall X-ray imaging system.

[0037] 2 is a schematic diagram showing an example of an X-ray imaging system 100 including an X-ray source 10 for emitting X-rays, an X-ray detector system 20 with an X-ray detector for detecting the X-rays after they have passed through an object, and an analog processing circuit 25 for processing and digitizing raw electrical signals from the X-ray detector. A digital processing circuit 40 may perform further processing operations on the measurement data, such as applying corrections, temporary storage, or filtering, and a computer 50 may store the processed data and perform further post-processing and / or image reconstruction. According to the present invention, all or part of the analog processing circuit 25 may be implemented in the X-ray detector system 20.

[0038] The entire X-ray detector may be considered as an X-ray detector system 20, or an X-ray detector system 20 combined with associated analog processing circuitry 25.

[0039] The digital part, including the digital processing circuit 40 and / or the computer 50, can be considered as an image processing system 30 that performs image reconstruction based on image data from the X-ray detector. Thus, the image processing system 30 can be considered as the computer 50, or alternatively as a combined system of the digital processing circuit 40 and the computer 50, or as the digital processing circuit 40 alone, if the digital processing circuit is further specialized for image processing and / or reconstruction as well.

[0040] One example of a commonly used x-ray imaging system is an x-ray computed tomography (CT) system, which may include an x-ray tube that produces a fan or cone beam of x-rays and an opposing array of x-ray detectors that measure the fraction of the x-rays that pass through a patient or object. The x-ray tube and detector array are mounted on a gantry that rotates around the object to be imaged.

[0041] FIG. 3 is a schematic block diagram of a CT system as an illustrative example of an X-ray imaging system. The CT system includes a computer 50 that receives commands and scanning parameters from an operator via an operator console 60, which may have a display and some form of operator interface, e.g., a keyboard and a mouse. The operator-supplied commands and parameters are then utilized by the computer 50 to provide control signals to the X-ray controller 41, the gantry controller 42, and the table controller 43. In particular, the X-ray controller 41 provides power and timing signals. The gantry controller 42 controls the rotational speed and position of the X-ray source 10 and the gantry 11 that comprises the X-ray source 10. By way of example, the X-ray detector may be a photon-counting X-ray detector. The table controller 43 controls and determines the position of the patient table 12 and the scan area of ​​the patient. The detector controller 44 is configured to control and / or receive data from the detector 20.

[0042] In one embodiment, computer 50 also performs image reconstruction and post-processing of image data output from the x-ray detector. The computer thus corresponds to image processing system 30 shown in Figures 1 and 2. An associated display allows an operator to view reconstructed images and other data from the computer.

[0043] An X-ray source 10 disposed in a gantry 11 emits X-rays. An X-ray detector 20, for example in the form of a photon counting detector, detects the X-rays after passing through a patient. The X-ray detector 20 may be formed, for example, by a number of pixels, also called sensors or detector elements, and associated processing circuits, such as an ASIC, disposed in a detector module. Part of the analog processing may be implemented in the pixels, while the remaining processing is implemented, for example, in the ASIC. In one embodiment, the processing circuit (ASIC) digitizes the analog signals from the pixels. The processing circuit (ASIC) may comprise a digital processing unit, which may perform further processing operations on the measurement data, such as applying corrections, temporary storage, and / or filtering. During a scan to acquire X-ray projection data, the gantry and the components mounted on the gantry rotate about an isocenter 13.

[0044] Modern X-ray detectors usually require the conversion of incident X-rays into electrons, which is usually done by the photoelectric effect or the Compton interaction, and the generated electrons usually produce secondary visible light until they lose energy, which is then detected by a light-sensitive material. In this case, the electrons generated by the X-rays create a charge in terms of electron-hole pairs that are collected by an applied electric field.

[0045] Some detectors operate in energy-integrated mode, meaning that they provide a signal integrated from many x-rays: the output signal is proportional to the total energy deposited by the detected x-rays.

[0046] X-ray detectors with photon counting and energy resolving capabilities are becoming more common in medical X-ray applications. Photon counting detectors have the advantage that in principle the energy of each X-ray can be measured, yielding additional information about the composition of the object. This information can be used to improve image quality and / or reduce radiation dose.

[0047] Typically, photon counting X-ray detectors determine the energy of a photon by comparing the height of the electrical pulse generated by the photon interaction in the detector material to a series of comparator voltages. These comparator voltages are also called energy thresholds. Typically, the analog voltage of the comparator is set by a digital-to-analog converter (DAC). The DAC converts the digital setting sent from the controller into an analog voltage against which the height of the photon pulse can be compared.

[0048] Photon counting detectors count the number of photons that have interacted in the detector during a measurement time. Typically, a new photon is identified by the fact that the height of the electrical pulse exceeds the comparator voltage of at least one comparator. Once a photon is identified, the event is stored by incrementing a digital counter associated with the channel.

[0049] Using several different thresholds, so-called energy discriminating photon counting detectors are obtained, which can sort the detected photons into energy bins corresponding to the various thresholds. This type of photon counting detector is sometimes called a multi-bin detector. In general, the energy information allows new kinds of images to be created, new information becomes available, and image artifacts inherent in conventional techniques can be removed. In other words, for an energy discriminating photon counting detector, the pulse height is compared to a number of programmable thresholds (T1-TN) of comparators and sorted according to the pulse height, which is proportional to the energy. That is, a photon counting detector consisting of several comparators can be referred to here as a multi-bin photon counting detector. For a multi-bin photon counting detector, the photon counts are stored in a set of counters, typically one per energy threshold. For example, a counter can be assigned to correspond to the highest energy threshold crossed by the photon pulse. In another example, a counter records the number of times the photon pulse crossed each energy threshold.

[0050] As an example, edge-on is a special non-limiting design for photon counting detectors where the x-ray sensors, such as x-ray detecting elements or pixels, are oriented edge-on with respect to the incident x-rays.

[0051] For example, such a photon counting detector may have pixels in at least two directions, one of the directions of the edge-on photon counting detector having a component in the direction of the x-rays. Such an edge-on photon counting detector may be referred to as a depth-segmented photon counting detector, having two or more depth segments of pixels in the direction of the incident x-rays.

[0052] Alternatively, the pixels can be arranged as an array (not depth resolved) in a direction substantially perpendicular to the direction of the incident x-rays, with each of the pixels oriented edge-on to the incident x-rays. In other words, the photon-counting detector can be non-depth resolved while still being positioned edge-on to the incident x-rays.

[0053] The absorption efficiency can be increased by placing the edge-on photon counting detector on edge, in which case the absorption depth can be chosen to be arbitrarily long and the edge-on photon counting detector can be fully depleted without requiring very high voltages.

[0054] The conventional scheme for detecting X-ray photons directly via semiconductor detectors basically works as follows: the energy of the X-ray interaction with the detector material is converted into electron-hole pairs inside the semiconductor detector, where the number of electron-hole pairs is generally proportional to the photon energy. The electrons and holes drift towards the electrodes and backside of the detector (or vice versa). During this drift, the electrons and holes induce a current in the electrodes, which can be measured.

[0055] As shown in FIG. 4, the signal(s) are sent from the detector elements 21 of the X-ray detector to the input 27 of a parallel processing circuit (e.g., ASIC) 25. The term application specific integrated circuit (ASIC) is broadly interpreted as a general circuit used and configured for a specific application. The ASIC processes the charge generated from each X-ray and converts it into digital data that can be used to obtain measurement data such as photon count and / or estimated energy. The ASIC is configured to connect to the digital data processing circuit so that the digital data can be sent to further digital data processing 40 and / or one or more memories 45, and finally the data will be the input of image processing 30.50 to generate a reconstructed image.

[0056] The number of electrons and holes generated from one X-ray event is proportional to the energy of the X-ray photon, so the total charge of one induced current pulse is proportional to this energy. After a filtering process in the ASIC, the pulse amplitude is proportional to the total charge of the current pulse, and therefore to the X-ray energy. The pulse amplitude can then be measured by comparing its value with one or more thresholds (THR) in one or more comparators (COMP). Also, a counter is introduced that can record the number of cases where the pulse is greater than the threshold. In this way, it is possible to count and / or record the number of X-ray photons with energies above the energy corresponding to the respective thresholds (THR) detected within a certain time frame.

[0057] The ASIC typically samples the analog photon pulse once per clock cycle and records the output of a comparator. The comparator (threshold) outputs a 1 or 0 depending on whether the analog signal is above or below the comparator voltage. The information available at each sample is, for example, a 1 or 0 for each comparator representing whether the comparator was triggered (the photon pulse was above the threshold) or not.

[0058] In a photon counting detector, there is usually a photon counting logic that determines if a new photon has been registered and registers the photon in a counter. In the case of a multi-bin photon counting detector, there are usually several counters, for example one for each comparator, and the photon count is registered in the counter according to an estimate of the photon energy. This logic can be implemented in several different ways. The two most common categories of photon counting logic are the so-called non-paralyzable counting modes and paralyzable counting modes. Other photon counting logic includes, for example, local maxima detection, which counts the local maxima detected in the voltage pulse and sometimes also records the pulse height.

[0059] Photon-counting detectors have many advantages, including but not limited to high spatial resolution, low electronic noise, energy resolution, and material separation capabilities (spectral imaging capabilities). However, energy-integrating detectors have the advantage of high count-rate tolerance. The count-rate tolerance comes from the fact / realization that since they measure the total energy of the photons, adding one additional photon will always increase the output signal (within reasonable limits), regardless of the amount of photons currently registered by the detector. This advantage is one of the main reasons why energy-integrating detectors are the standard in medical CT today.

[0060] For a better understanding of the proposed measurement method, it may be useful to start with a brief system overview and / or an analysis of the technical problems. To this end, reference is made to Figure 5, which provides a schematic diagram of a photon counting circuit and / or device according to the prior art.

[0061] When photons interact in the semiconductor material, a cloud of electron-hole pairs is generated. When an electric field is applied to the detector material, the charge carriers are collected at electrodes attached to the detector material. A signal is sent from the detector to the input of a parallel processing circuit (such as an ASIC). It should be understood that the term Application Specific Integrated Circuit (ASIC) is to be broadly interpreted as a general circuit used and configured for a specific application. The ASIC processes the charge generated from each X-ray and converts it into digital data, which can be used to obtain measurement data such as the number of photons and / or estimated energy. As an example, the ASIC can process the charge such that a voltage pulse is generated with a maximum height proportional to the amount of energy deposited by the photon in the detector material.

[0062] The ASIC may include a set of comparators 302, each of which compares the magnitude of the voltage pulse to a reference voltage. The output of the comparator will be either zero or one (0 / 1) depending on which of the two voltages compared is greater. Assume here that if the voltage pulse is higher than the reference voltage, the comparator output is (1) and if the reference voltage is higher than the voltage pulse, it is (0). A digital-to-analog converter (DAC) 301 may be used to convert a digital setting, which may be provided by a user or a control program, into a reference voltage that may be used by the comparators 302. If the height of the voltage pulse exceeds the reference voltage of a particular comparator, that comparator will be said to be triggered. Each comparator is typically associated with a digital counter 303 that is incremented based on the comparator output according to photon counting logic.

[0063] In general, basis material decompositions exploit the fact that all materials composed of elements with low atomic numbers, such as human tissue, have a linear attenuation coefficient μ(E), whose energy dependence can be approximately expressed as a linear combination of two (or more) basis functions. μ(E)=a1f1(E)+a2f2(E) where fi is the basis function and ai is the corresponding basis coefficient. If one or more elements with atomic numbers high enough that their k-edges lie in the energy range used for imaging are present in the imaged volume, one basis function needs to be added for each such element. In the field of medical imaging, such k-edge elements might typically be iodine or gadolinium, substances used as contrast agents.

[0064] As mentioned above, the line integrals Ai of each of the basis coefficients ai are estimated from the measurement data for each projection ray l from the source to the detector element. The line integrals Ai can be expressed as follows:

[0065]

number

[0066]

number

[0067] The maximum likelihood method can then be used to estimate Ai under the assumption that the counts in each bin are Poisson distributed random variables, which is accomplished by minimizing the negative log-likelihood function (see Roessl and Proksa, K-edge imaging in x-ray computed tomography using multi-bin photon counting detectors, Phys. Med. Biol. 52 (2007), 4679-4696).

[0068]

number

[0069] From the line integrals A, a tomographic reconstruction ai may be performed to obtain the basis coefficients. This procedural step can be considered as a separate tomographic reconstruction or as part of the overall basis decomposition.

[0070] As previously mentioned, the result is obtained for each projection line.

[0071]

number

[0072] The proposed technique aims to significantly improve image reconstruction by using basis image representations resulting from two or more basis material decompositions as input to machine learning-based data processing. The technical solution provided by the proposed technique can, by way of example, reduce bias, artifacts and / or noise in the output image and the corresponding image representations.

[0073] FIG. 6 is a schematic flow diagram showing an example of an image reconstruction method based on energy-resolved X-ray measurement data according to an embodiment. Essentially, the method includes the following steps: S1: Collect at least one representation of energy resolved X-ray data. S2: Perform at least two basis material decompositions based on the at least one representation of the energy-resolved X-ray data to generate at least two sets of original basis image representations. S3: Obtain or select at least two basis image representations from at least two of the original sets of basis image representations. S4: Processing the obtained or selected basis image representations by machine learning based data processing to generate at least one representation of output image data. As an example, the machine learning based data processing may be configured to map the obtained or selected base image representation to at least one representation of the output image data.

[0074] In certain examples, the machine learning based data processing may be configured to map the obtained or selected basis image representations to at least one representation of the output image data by implementations based on trained neural networks, support vector machines, or decision tree based means.

[0075] For example, implementations based on trained neural networks, support vector machines, or decision trees can be trained on at least one of measured patient image data or phantom image data, or on simulated image data based on a numerical phantom.

[0076] Optionally, a neural network, support vector machine, or decision tree based implementation may be trained on images of a first type and adapted for use with images of a second type using transfer learning.

[0077] As an example, implementations based on neural networks, support vector machines, or decision trees may be implemented on non-energy-resolved CT images and adapted for use on energy-resolved photon-counting CT images using transfer learning.

[0078] Typically, at least one representation of the energy resolved X-ray data may originate from the X-ray detector or intermediate storage and may include energy resolved X-ray image data and / or energy resolved X-ray measurement data.

[0079] For example, the X-ray detector may be an energy-resolving photon counting detector, also called a multi-bin photon counting X-ray detector.

[0080] Alternatively, energy resolved x-ray data can be obtained from multiple x-ray tube acquisitions, slow or fast kV switching acquisitions, multi-layer detectors, or by split filter acquisition.

[0081] 7 is a schematic diagram illustrating an example of a complementary representation of machine learning based image reconstruction according to one embodiment. Essentially, energy resolved x-ray data is provided as input to a basis material decomposition to generate at least two sets of basis image representations, at least a subset of which are forwarded to a machine learning process and / or system that, once properly trained, maps the basis image representations to output image data (usually by a process called inference).

[0082] In a particular embodiment, the at least one representation of the output image data includes a set of basis image representations, referred to as an output set of basis image representations, wherein the number of basis image representations in the output set is greater than the maximum number of basis image representations in any of the at least two original sets of basis image representations.

[0083] As an example, the output set of basis image representations may have a number of basis functions, each of the original basis image representation sets may have a respective number of basis functions, and a basis function corresponding to the output set of basis image representations may be a subset of or equal to the sum of the basis functions corresponding to the at least two original basis image representation sets.

[0084] For example, the output set of basis image representations has a number of basis functions, each of the original basis image representation sets having a respective number of basis functions, and the number of the obtained or selected basis image representations is the same as or greater than the number of basis functions corresponding to the output set of basis image representations.

[0085] In a particular example, the original set of basis image representations is based on two-basis decompositions and the output set of basis image representations is based on three-basis decompositions.

[0086] In such a case, the machine learning based data processing may be performed, for example, by a neural network, which may be trained using a plurality of 2-based image representation sets as input data and a 3-based image representation set as output data.

[0087] As an example, the 2-basis image representation sets and the 3-basis image representation sets may be generated from measured patient image data or phantom image data, or from simulated image data based on a numerical phantom.

[0088] Optionally, the step of processing the obtained or selected basis image representations may include: - combining the obtained or selected basis image representation with at least one representation of non-basis image data. - Processing the acquired or selected basis image representation together with a representation of at least one non-basis image data by machine learning based data processing.

[0089] For example, the non-basis image data can include at least one of bin counts sinogram data, log-normalized bin counts sinogram data, sum-over-bins sinogram or log-normalized data, reconstructed CT image data, and segmentation mask data.

[0090] Optionally, the step of processing the obtained or selected base image representation may include: - combining said obtained or selected basis image representations with at least one representation of prior information. - Processing the obtained or selected basis image representations using the at least one representation of prior information by machine learning based data processing.

[0091] Prior information is generally defined as a distinct set of available information that is assumed to be known, for example, based on physical laws and principles and previous measurements, images and / or simulations.

[0092] Optionally, the step of processing the obtained or selected basis image representations may include: - combining the acquired or selected basis image representations with at least one computational representation of a physical process in an acquisition system, - Processing the obtained or selected basis image representations with said at least one computational representation of the physical process by machine learning based data processing.

[0093] Optionally, the step of processing the obtained or selected basis image representations may include: - combining the acquired or selected basis image representations with at least one computational representation of a statistical model of the measurement data, - processing the acquired or selected basis image representations with said at least one computational representation of a statistical model by machine learning based data processing.

[0094] In certain examples, the at least one representation of output image data includes an output set of basis image representations and / or material decomposition reconstructed images.

[0095] As an example, the machine learning based data processing can include deep learning post-processing configured based on the training data, where the deep learning post-processing may be configured to receive biased material-decomposed sinograms as input and provide de-biased material-decomposed sinograms or debiased material-decomposition reconstructed images as output.

[0096] As an example, a set of sinograms can be provided as input for a 2-basis decomposition to generate two or more basis image representations, and a deep learning neural network can be used to convert the resulting basis image representations into a 3-material decomposition.

[0097] According to a second aspect, a system for image reconstruction based on energy resolved X-ray data is provided. The system is configured to obtain at least one representation of the energy resolved X-ray data. The system is configured to perform at least two basis material decompositions based on the at least one representation of the energy resolved X-ray data to generate at least two sets of basis image representations. The system is further configured to obtain or select at least two basis image representations from the set of at least two basis image representations. The system is also configured to process the obtained or selected basis image representations by data processing based on a machine learning system to generate at least one representation of output image data.

[0098] According to a third aspect, there is provided a system for image reconstruction based on energy resolved X-ray data, the system comprising: an input for receiving at least one representation of energy resolved X-ray data; a basis material decomposition system configured to perform at least two basis material decompositions based on said at least one representation of the energy resolved X-ray data to generate at least two sets of basis image representations; - a trained machine learning system having a configuration determined based on input / output pair training data and adapted to receive at least two base image representations from at least two of the sets of base image representations and process the received base image representations by machine learning; and a base data processing to generate at least one representation of output image data. an output for outputting said at least one representation of output image data.

[0099] 8 is a schematic block diagram showing an example of an X-ray image reconstruction system according to an embodiment. The X-ray image reconstruction system 30 in FIG. 8 basically includes an input 31, a basis material decomposition system 32, a machine learning system 33, and an output 34.

[0100] As mentioned above, the machine learning system 33 can be based on, for example, a neural network, a support vector machine or a decision-tree based implementation.

[0101] As an example, the machine learning system 33 may be trained on at least one of measured patient image data or phantom image data, or from simulated image data based on a numerical phantom.

[0102] In a particular example, the machine learning system 33 is trained on images of a first type and adapted for use with images of a second type using transfer learning.

[0103] For example, the machine learning system 33 may be trained on non-energy resolved CT images and adapted for use on energy resolved photon counting CT images using transfer learning.

[0104] In a particular example, the machine learning system 33 is configured to map the acquired or selected basis image representations to said at least one representation of the output image data.

[0105] Typically, said at least one representation of energy resolved X-ray data may originate from an X-ray detector or an intermediate storage device and may comprise energy resolved X-ray image data and / or energy resolved X-ray measurement data.

[0106] By way of example, the X-ray detector may be an energy-resolving photon counting detector, also called a multi-bin photon counting X-ray detector.

[0107] As described above, the at least one representation of the output image data may include a set of base image representations, referred to as the output set of base image representations, where the number of base image representations in the output set is greater than the maximum number of base image representations in any of the at least two original sets of base image representations.

[0108] For example, the output set of basis image representations has a number of basis functions, each original basis image representation set has a respective number of basis functions, and a basis function corresponding to the output set of basis image representations is a subset of or equal to the sum of the basis functions corresponding to the at least two original basis image representation sets.

[0109] For example, the at least one representation of output image data may include an output set of basis image representations and / or material decomposition reconstructed images.

[0110] In a particular example, an image reconstruction system comprises at least one processor and a memory, the memory comprising instructions that, when executed by the at least one processor, cause the at least one processor to perform image reconstruction.

[0111] By way of example, the basis material decomposition system may comprise processing circuitry configured to generate at least two sets of basis image representations, and the trained machine learning system may comprise processing circuitry configured to generate at least one representation of the output image data.

[0112] For a better understanding, the proposed technique will now be described with reference to a non-limiting example for image reconstruction using basis image representations obtained from two or more basis decompositions as input to a machine learning process and / or system.

[0113] In the field of X-ray imaging, the representation of the image data may include, for example, a sinogram, a projection image, or a reconstructed CT image. Such a representation of the image data may be energy resolved if it includes multiple channels, where the different channels of data relate to X-ray data measured at different energy intervals.

[0114] Through a process of material decomposition, which takes as input a representation of the energy-resolved x-ray data, a set of basis image representations can be generated. Such a set is a collection of multiple basis image representations, each basis image representation being associated with the contribution of a particular basis function to the total x-ray attenuation. Such a set of basis image representations can be a set of basis sinograms, a set of reconstructed basis CT images, or a set of projection images. It will be appreciated that "image" in this context can mean, for example, a two-dimensional image, a three-dimensional image, or a time-resolved image series.

[0115] By way of example, the energy resolved X-ray data may be energy resolved X-ray image data and / or energy resolved X-ray measurement data. By way of example, at least one representation of the energy resolved X-ray data may originate from an X-ray detector or an intermediate storage device.

[0116] For example, a representation of energy resolved x-ray data may include a collection of energy bin sinograms, where each energy sinogram includes the number of counts measured in one energy bin. This collection of energy bin sinograms may be input to a material decomposition algorithm to generate a set of basis sinograms. Such basis sinograms may illustratively be taken as input to a reconstruction algorithm to generate reconstructed basis images.

[0117] In a two-basis decomposition, a two-basis image representation is generated based on the approximation that the attenuation of any material in the imaged object can be expressed as a linear combination of two basis functions. In a three-basis decomposition, a three-basis image representation is generated based on the approximation that the attenuation of any material in the imaged object can be expressed as a linear combination over three basis images. Similarly, four-basis decompositions, a five-basis decomposition and similar multi-basis decompositions can be defined. It is also possible to perform a one-basis decomposition by approximating all materials in the image object as having X-ray attenuation coefficients with similar energy dependence up to the density scale factor.

[0118] The two-basis decomposition can result in a set of basis sinograms including, for example, water sinograms and iodine sinograms, corresponding to basis functions given by the linear attenuation coefficients of water and iodine, respectively. Alternatively, the basis functions can represent the attenuation of water and calcium, calcium and iodine, polyvinyl chloride and polyethylene. The two-basis decomposition can result in a set of basis sinograms consisting of, for example, water sinograms, calcium sinograms and iodine sinograms. Alternatively, the basis functions can represent the attenuation of water, iodine and gadolinium, or the attenuation of polyvinyl chloride, polyethylene and iodine.

[0119] The inventors have recognized that accurate basis decompositions using more than two basis functions may be difficult to perform in practice and may introduce artifacts, bias, or excessive noise. Such basis decompositions may also require extensive calibration measurements and data pre-processing to obtain accurate results. In general, basis decompositions into a large number of basis functions may be technically more difficult than decompositions into a small number of basis functions.

[0120] For example, it may be difficult to perform a calibration accurate enough to produce a 3-basis decomposition that has a lower level of image bias and artifacts compared to a 2-basis decomposition. It may also be difficult to find a material decomposition algorithm that can perform a 3-basis decomposition on very noisy data without producing excessively noisy basis images. That is, it may be difficult to achieve a theoretical lower bound on the basis image noise given by the Cramer-Rao lower bound, which is easier to achieve when performing a 2-basis decomposition.

[0121] The inventors further appreciate that the amount of information required to generate a larger number of basis image representations may be possible to extract from a set of multiple basis image representations, each having a smaller number of basis image representations. For example, the information required to generate a three-basis decomposition into water, calcium, and iodine sinograms may be possible to extract from a set of three two-basis decompositions: a water-calcium decomposition, a water-iodine decomposition, and a calcium-iodine decomposition.

[0122] It may be easier to accurately perform multiple 2-basis decompositions than to accurately perform a single 3-basis decomposition. In a particular example of the disclosed invention, this observation may be used, for example, to solve the problem of performing an accurate 3-basis decomposition. As an example, energy-resolved image data may first be used to perform water-calcium, water-iodine, and calcium-iodine decompositions. A machine learning system, such as a convolutional neural network, may then be used to map the resulting six (3×2) basis images, or a subset thereof, to a set of three output images consisting of water, calcium, and iodine images. Such a machine learning system or network may be trained with several 2-basis image representation sets as input data and 3-basis image representation sets as output data, where the 2-basis and 3-basis image representation sets are generated from measured patient or phantom image data, or from simulated image data based on a numerical phantom.

[0123] Using the above method, bias, artifacts, or noise in the 3-basis image representation set can be significantly reduced compared to a 3-basis decomposition performed directly on the energy-resolved image data, or higher resolution images can be obtained.

[0124] The machine learning algorithm applied to the original basis images may include another machine learning method, such as a support vector machine or a decision tree based method, instead of a neural network.

[0125] More generally, the inventors have realized that it is possible to substantially improve and accelerate the efficiency, operation and / or results of machine learning processes and / or systems in performing X-ray image reconstruction. Instead of following the prevailing trend of using machine learning on raw image data, the inventors have realized that improved image results can be achieved if basis image representations resulting from two or more basis material decompositions are utilized as inputs for the machine learning process and / or system. For example, improvements can include reduced bias, artifacts and / or noise, or higher resolution. Another advantage is that the proposed technique reduces the amount of training data required to obtain image reconstructions with high reliability and quality.

[0126] The basis material decomposition used to generate the original basis image representation may use and / or involve prior information, such as, for example, volume or mass conservation constraints or non-negativity constraints. Alternatively, such prior information may take the form of a prior image representation, such as, for example, an image reconstructed from images from a previous examination or tabulated counts in all energy bins, and the algorithm may penalize deviations of the decomposed basis image representation relative to this prior image representation. Another option is to use prior information learned from a set of training images, represented for example as a learned dictionary or a pre-trained convolutional neural network, or a learned subspace in which the reconstructed image is expected to reside, i.e., a subspace of the vector space of possible images.

[0127] Material decomposition can be performed on the projection image data or sinogram data, for example, by processing each measured projection ray independently, which can take the form of maximum likelihood decomposition or maximum a posteriori decomposition, where a prior probability distribution on the material composition of the imaged object is assumed. Also, a linear or affine transform from the set of input counts to the set of output counts, an A-table estimator as exemplarily described by Alvarez (see Med Phys. 2011 May; 38(5):232-2334), a low-order polynomial approximation such as the neural network estimator exemplarily described by Lee et al. (IEEE Transactions on Medical Imaging (Volume: 36, Issue: 2, Feb. 2017: 560-573) or a look-up table. Alternatively, the material decomposition method may process multiple rays together and may consist of a one-stage or two-stage reconstruction algorithm.

[0128] WO2017 / 223560 discloses a method for reconstructing a tomographic image using a deep learning algorithm to generate a reconstructed image, for example by first applying a first algorithm to obtain intermediate data, and then applying a second algorithm to the intermediate image.

[0129] US Pat. No. 9,036,885 discloses a projection domain denoising method that processes projection data using a sparsifying transform and shrinkage, for example using a learned shrinkage function.

[0130] WO2015 / 028975 discloses a method for receiving at least two basis material sinograms and removing noise in the basis material sinograms using a decorrelation filter.

[0131] An article by Chen and Li in Optical Engineering 58(1), 013104 describes a method for performing multiple material decomposition of spectral CT data using deep neural networks, which simply performs the actual material decomposition using a neural network.

[0132] An article by Poirot et al. in Scientific Reports volume 9, Article number: 17709 (2019) discloses a method for generating non-contrast single-energy CT images from dual-energy CT images using a convolutional neural network.

[0133] According to a first aspect, a non-limiting example of a method for processing energy resolved X-ray data is provided, the method comprising the following steps. - collecting at least one representation of energy resolved X-ray data. performing at least two basis material decompositions on said at least one representation of the energy resolved X-ray data, resulting in at least two sets of original basis image representations. - obtaining or selecting at least two basis image representations from at least two of the original sets of basis image representations; - Processing the obtained or selected basis image representations with data processing (such as a data processing procedure) based on a machine learning algorithm to generate at least one representation of output image data.

[0134] In other words, it can be expressed as an (X-ray) image reconstruction method based on energy-resolved X-ray data.

[0135] According to a second aspect, there is also provided a corresponding non-limiting example of a system for processing energy resolved X-ray data, comprising: the system is configured to collect at least one representation of energy resolved X-ray data; the system is configured to perform at least two basis material decompositions on said representation of image data to generate at least two sets of original basis image representations; the system is configured to obtain or select at least two basis image representations from at least two of said original sets of basis image representations; - configured to process the acquired or selected basis image representations using machine learning based data processing (such as a data processing procedure) to generate at least one representation of output image data.

[0136] That is, it can be expressed as a system for reconstructing (X-ray) images based on energy-resolved X-ray data.

[0137] In an exemplary embodiment of the invention, the step or arrangement of acquiring at least one representation of energy resolved X-ray data is performed by a CT imaging system.

[0138] In an exemplary embodiment of the invention, said step or arrangement of collecting at least one representation of energy resolved X-ray data is performed by an energy resolving photon counting detector, also referred to as a multi-bin photon counting X-ray detector.

[0139] Alternatively, said step or arrangement of acquiring at least one representation of energy resolved x-ray data is performed by methods of multi x-ray tube acquisition, slow or fast kV switching acquisition, multi-layer detector or split filter acquisition.

[0140] In an exemplary embodiment of the invention, the at least one original or output basis representation comprises at least one of a sinogram, a projection X-ray image, or a reconstructed CT image.

[0141] In an exemplary embodiment of the present invention, the steps or configurations of processing the selected basis images include the following steps or configurations. - combining the selected basis image data with at least one representation of non-basis image data, such as bin counts sinogram data, log-normalized bin counts sinogram data, sum-over-bins sinogram or log-normalized data, reconstructed CT image data, and / or segmentation mask data. - Processing the selected basis image data together with representations of the non-basis image data using machine learning based data processing methods.

[0142] In an exemplary embodiment of the invention, said at least one representation of output image data is an approximation of a basis image representation forming part of a basis image set corresponding to a basis function different from the original basis image set.

[0143] In an exemplary embodiment of the present invention, the at least one representation of output image data consists of a set of basis image representations, and the number of basis image representations in the output set is greater than the maximum number of basis images in any of the sets of basis images in the original decomposition.

[0144] In an exemplary embodiment of the present invention, the basis functions corresponding to the output base image representation set are or are equal to a subset of the union of the basis functions corresponding to the original base image representation set, or alternatively, the basis functions corresponding to the output base image representation set may not be a subset of the union of the basis functions corresponding to the original base image representation set.

[0145] In an exemplary embodiment of the present invention, the number of the selected basis image representations is equal to or greater than the number of basis functions corresponding to the set of output basis image representations.

[0146] In an exemplary embodiment of the invention, the original basis image representation set comprises / is a 2-basis decomposition and the at least one output image representation set comprises a 3-basis decomposition.

[0147] In an exemplary embodiment of the invention, at least one of the basis material decompositions is performed by a maximum likelihood method. Alternatively, at least one of the basis material decompositions is performed via a linear or affine transformation, an A-table method, a low-order polynomial approximation, a neural network or a look-up table.

[0148] In an exemplary embodiment of the invention, at least one of said basis material decompositions is performed by a one-stage basis image reconstruction method or by a two-stage basis image reconstruction method.

[0149] In an exemplary embodiment of the invention, the machine learning comprises a machine learning architecture and / or algorithm that may be based on a convolutional neural network, or alternatively, the machine learning architecture and / or algorithm may be based on a support vector machine or a decision tree based method.

[0150] In an exemplary embodiment of the invention, the convolutional neural network is based on a residual network (ResNet), residual encoder-decoder, U-Net, AlexNet or LeNet architecture. Alternatively, the machine learning algorithm based on a convolutional neural network can be based on an unrolled optimization method based on a gradient descent algorithm, a primal-dual algorithm or an alternating direction method of multipliers (ADMM) algorithm.

[0151] In an exemplary embodiment of the invention, the convolutional neural network includes at least one forward projection or at least one back projection as part of the network architecture.

[0152] In an exemplary embodiment of the invention, the convolutional neural network is trained on at least one of phantom images, patient images, or numerical phantoms generated from patients imaged with non-energy-resolved CT.

[0153] In an exemplary embodiment of the invention, the convolutional neural network is trained using transfer learning on a first type of CT images, e.g., non-energy resolved CT images, and adapted for use using transfer learning on a second type of CT images, e.g., photon counting CT images.

[0154] In an exemplary embodiment of the present invention, said step or arrangement of processing the selected basis images includes: - combining the selected basis image data with at least one representation of prior information, such as a volume constraint on a prior image, a prior distribution represented as a learned dictionary, a regularization penalty, a learned regularization neural network, or a learned subspace, or a statistical anatomical shape model; - processing the selected basis image data with the machine learning based data processing using the representation of the prior information.

[0155] In an exemplary embodiment of the present invention, the step or step of processing the selected basis images comprises: - combining said selected basis image data with at least one computational representation of a physical process in an acquisition system, such as Compton scattering, fluorescence, charge sharing, pile-up or geometric blur; - processing the selected basis image data using a computational representation of a physical process based on a machine learning algorithm.

[0156] Alternatively, the step or arrangement of processing the selected basis images comprises combining the selected basis image data with at least one computational representation of a statistical model of the measurement data, such as a likelihood function or a derivative of a likelihood function.

[0157] In an exemplary embodiment of the invention, the machine learning architecture and / or algorithm is trained to generate one or more output basis image representations that have lower noise than the original basis image representations. Alternatively, the machine learning architecture and / or algorithm may be trained to generate one or more output basis image representations that have lower bias than the original basis image representations. In yet another alternative, the machine learning architecture and / or algorithm may be trained to generate one or more output image data representations that have higher spatial resolution than the original basis image representations.

[0158] Deep learning is generally related to machine learning methods based on artificial neural networks or similar architectures through representation learning. Learning can be supervised, semi-supervised, or unsupervised. Deep learning systems such as deep neural networks, deep belief networks, recurrent neural networks and convolutional neural networks have been applied in a variety of technology fields, including computer vision, speech recognition, natural language processing, social network filtering, machine translation, and board game programs. They produce results that rival and in some cases exceed the performance of human experts.

[0159] The adjective "deep" in deep learning comes from the use of multiple layers in the network. Early work showed that linear perceptrons could not be universal classifiers, and that networks with non-polynomial activation functions and hidden layers of unlimited width could. Deep learning is a modern variation, with an infinite number of layers, allowing optimized implementations and preserving theoretical universality under mild conditions. In deep learning, layers can be heterogeneous and can deviate significantly from biologically based connectionist models for the sake of efficiency, trainability, and understandability. For a better understanding, we now describe an illustrative and non-limiting example of the proposed technique.

[0160] The material decomposition problem is a nonlinear inverse problem that is difficult to solve, both in terms of computational cost and accuracy. The most accepted solutions involve optimization based on maximum likelihood (ML) estimation using Poisson statistics. This is a model-based approach that relies heavily on the forward model considered. On the other hand, data-driven approaches make use of available measurement data. This is the case for deep learning, which can provide prior information in a fast way.

[0161] For example, in photon-counting CT (PCCT), three materials are usually involved in clinical analysis. However, in practice, two-substance ML estimation has better convergence, stability, and less noise than three-substance estimation. However, two-substance estimation has a high bias. To correct this bias, post-processing of deep learning using measured training data can be performed to perform data-driven regularization and bias correction while maintaining the advantages of two-substance ML estimation (stability, convergence). Therefore, it is proposed to combine the advantages of two-substance basis decomposition with a data-driven approach to control bias.

[0162] In a specific, non-limiting example, the idea is to move from a two-material decomposition to a three-material decomposition. In this example, the proposed technique exploits the stability and convergence of two-material basis ML estimation and considers data-driven post-processing to correct biases. As an example, the material decomposition may be performed in the projection (sinogram) domain, i.e., the reconstruction of the CT image may be an independent second stage.

[0163] Assuming that the scanned object contains three materials, e.g., bone, soft tissue, and contrast agent (e.g., iodine), we define a set of two-material estimates to obtain all possible combinations of two basis materials, i.e., (i) bone and soft tissue, (ii) bone and iodine, and (iii) soft tissue and iodine. For example, the deep learning architecture and / or procedure of the proposed technique can consider this collection of six biased estimates as input and provide unbiased estimates of bone, soft tissue, and iodine as output.

[0164] For example, it may be feasible to use a supervised machine learning architecture and / or procedure (training data of input-output pairs) that receives a biased material decomposition sinogram (from the 2-basis ML estimate) and provides three corrected material decomposition sinograms as output.

[0165] For example, the designed architecture may be a residual network composed of five residual blocks (ResBlocks). Each block may contain three convolutional layers followed by batch normalization and rectified linear units (ReLu) activation. In a specific example, the output channels of each convolutional layer are 32, 32, and 3, respectively.

[0166] 9 is a schematic diagram illustrating an example of a scheme and / or architecture for the proposed deep learning debiasing. In a non-limiting example, a set of, for example, eight sinograms may be provided as input for a two-basis decomposition to generate two or more basis image representations. A deep learning neural network may be used to convert the resulting basis image representations into, for example, three material decompositions in projection space.

[0167] As an example, ResBlocks can mimic the iterative solution update of an optimization problem, adding information from training data with a trained convolution filter. Each block Ψ can be seen as the nth update step: the input of the (n+1)th block an+1 is the difference between the output of the nth block an and the result of the last block Ψn parameterized by its filter weight θn.

[0168]

number

[0169] It should be understood that Ψ is a function of the variables an and is parameterized by θn. Figure 10 is a schematic diagram showing an example of ResBlock.

[0170] FIG. 11 is a schematic diagram illustrating an example of a bias-removing deep learning scheme and / or architecture that includes a series of residual blocks (ResBlocks) following biased results from a two-material decomposition.

[0171] As part of the present invention and R&D, we have carried out a proof of concept simulation of this technology. As an example, a feasible setup is presented below.

[0172] A dataset of simulated PCCT data based on a Shepp-Logan head. We consider a forward model of the PCCT system and Poisson noise. The size of the phantom is 64x64 pixels in image space and 100x100 pixels in projection space. The materials considered are PVC (bone), PE (soft tissue), and iodine. To add variability to the data, we randomly changed the size, rotation, and location of the iodine region for each sample. Figure 12 is a schematic diagram showing examples of samples from the simulated dataset, where different parts of the Shepp-Logan mimic different materials.

[0173] Considering the two-substance decomposition ML function, pairs of estimations are performed: PVC / PE, PVC / I, and PE / I. These estimates are biased since they belong to a mixture of three substances.

[0174] These biased estimates are the input to a neural network, whose output consists of three unbiased and denoised estimates (PVC, PE, I), one for each material.

[0175] To train the neural network, we consider an Adam optimization scheme with a mean squared error (MSE) training loss. In this example, we use 250 samples for training and 100 samples for testing.

[0176] Evaluation of 100 test samples shows that the MSE and bias of the resulting estimates are significantly improved, see Table 1 and Figure 13 below.

[0177] [Table 1]

[0178] Figure 13(a)-(f) are schematic diagrams showing example qualitative results of applying the proposed deep learning based bias mitigation to a simulated phantom by Shep Logan.

[0179] FIG. 13(a) shows examples of true sinograms of materials: PVC (first row), PE (second row) and iodine (third row).

[0180] Figure 13(b) shows examples of the best three out of six solutions from biased two-material ML.

[0181] FIG. 13(c) shows an example of the proposed unbiased sinogram.

[0182] FIG. 13(d) shows an example of the true basis image after reconstruction by filtered back projection (FBP).

[0183] Fig. 13(e) shows an example of biased estimation after FBP.

[0184] FIG. 13(f) shows an example of the proposed unbiased result after FBP.

[0185] Below, we present alternative implementations of the proposed technique that may improve results, especially if this technology / method needs to be extended to accommodate real clinical data. (i) Neural Networks o Scaling the proposed residual network: When the input dimension is large, the size of the convolutional layers and the number of blocks can be designed accordingly, i.e., by increasing the convolutional layers in each block, increasing the convolutional output channels in each layer, or increasing ResBlock beyond 5. o Change in network architecture: The proposed residual network can be replaced by U-net, which also exploits multi-scale information of the input. For example, "U-Net" can be found in "Convolutional Networks for Biomedical Image Segmentation, Medical Image Computing and Computer-Assisted Intervention- MICCAI (2015)" by O. Ronneberger et al. Alternatively, the proposed network can be replaced by another common network architecture combining convolutional and fully-connected layers, e.g. AlexNet or LeNet. o Enhancement of neural network architecture: Consider rectangular convolution filters instead of square ones, such as width-wise (projection-dominant) and height-wise (detection-dominant) filters. For reference, please refer to “Machine Friendly Machine Learning: Interpretation of Computed Tomography Without Image Reconstruction”, by Lee, H., et al., Sci Rep 9, 15540 (2019). o Mimic the iterative update step of the optimization problem more explicitly: Therefore, consider neural network architectures whose definition includes prior information about the forward model of the system. See for example “Learned Primal Dual Reconstruction”, by Adler, J and Oktem, O., IEEE Transactions on Medical Imaging (2018). o Customize training loss: Instead of the basic mean-squared error (MSE), you can consider any quality metric of interest in the training loss, such as bias, contrast-to-noise ratio (CNR), regularized loss, etc. o Multiple forms of data can be taken as input to the network: for example, energy bin sinograms and material decomposition sinograms can be combined and used as input to the network. Also, energy bin sinograms and / or material decomposition sinograms can be combined with energy bin images and / or material decomposition images as input to the network. Also, derivatives of the likelihood function and / or Cramer-Rao lower bound representations of the basis image noise can be combined with material decomposition sinograms and input to the network. All combinations of the above input data can be used with two or more basis material sinograms, or images from at least two different sets of basis image representations. o Enhance the representation ability of the network with spectral attention modules: The energy bin data can also be the input of the network, and some information can be extracted by the spectral attention module in the network architecture. For example, see "Learning Spectral-Spatial Prior Via 3DDNCNN for Hyperspectral Image Deconvolution" by X. Wang et al., ICASSP (2020). (ii) Another decomposition: o Tissue / diagnosis / anatomical decomposition: Deep learning allows for a flexible learning process from which interesting transformations of the input data can be inferred without the need to define an underlying model. Thus, one may not necessarily obtain a 3-basis decomposition, but rather a tissue type decomposition (even larger than 3), a diagnostic domain decomposition, an anatomical cross-section decomposition. A similar setting can be used for these purposes, provided one has a set of labeled input-output training data. (iii) CT reconstruction will be considered during the process. o Steps decomposition and reconstruction: In the proposed method, it is assumed that CT reconstruction (sinogram to image space) is an independent step. A neural network can handle the two steps simultaneously. For example, a non-transfer iterative optimization network can be trained to directly transform a biased material decomposed sinogram into a biased material decomposed reconstructed image. In this way, the neural network can solve both steps (decomposition and reconstruction) in a more optimal way (computation and accuracy). See for example "Learned Primal-Dual Reconstruction" by J. Adler and O. Oktem, IEEE Transactions on Medical Imaging, (2018). Such an evolving iterative optimization network can be based on a gradient descent optimization method, a primal-dual optimization method, or an alternating direction method of multipliers method. This type of renormalization optimization method has advantageous properties compared to other neural network architectures, such as having fewer network weights and working better with a small amount of training data. Such a one-step method can take as input, for example, a combination of basis image data and energy-resolved measurement data. Alternatively, the one-step reconstruction method may be implemented as an iterative algorithm. A non-limiting example of this embodiment of the present invention is shown in FIG. 14, where a machine learning system processes material decomposition sinograms from at least two basis material decompositions and generates a set of material decomposition images as output. Alternatively, the material decomposition step used to generate the original basis image set may constitute a one-step reconstruction method resulting in reconstructed basis images as output. These are then processed by a machine learning algorithm to result in one or more output basis images. A non-limiting example of this embodiment of the invention is shown in FIG. 15, where the machine learning system processes material decomposition images from at least two one-step image reconstructions that convert input energy bin sinograms into material decomposition reconstructed images to generate a material decomposition image set as output. A further alternative is to generate the original basis image set using a two-step basis decomposition method, consisting of a first step in which a basis material decomposition is performed separately for each projection ray, and a second step in which the resulting basis sinograms are taken as input to an image reconstruction algorithm to result in reconstructed basis images that are processed by a machine learning algorithm to result in one or more output basis images. A non-limiting example of this embodiment of the invention is shown in FIG. 16, where the machine learning system processes material decomposition images from at least two two-step image reconstructions, each consisting of a basis material decomposition step followed by an image reconstruction step, to generate a set of material decomposition images as output. The one-step or two-step decomposition methods may optionally incorporate prior information such as prior images or volume preservation constraints, or may incorporate information about the noise level or correlation structure in the sinogram or image, illustratively the Cramer-Rao lower bound.

[0186] It will be appreciated that the mechanisms and configurations described herein can be implemented, combined, and reconfigured in various ways.

[0187] For example, the embodiments may be implemented in hardware, or at least partly in software for execution by suitable processing circuitry, or a combination thereof.

[0188] The steps, functions, procedures, and / or blocks described herein may be implemented in hardware using any conventional technology such as discrete or integrated circuit technology, including both general purpose electronic circuitry and application specific circuitry.

[0189] Alternatively, or complementary, at least some of the steps, functions, procedures and / or blocks described herein may be implemented in software, such as a computer program, for execution by suitable processing circuitry, such as one or more processors or processing units.

[0190] 17 is a schematic diagram illustrating an example of a computer implementation according to an embodiment. In this particular example, the system 200 comprises a processor 210 and a memory 220, the memory including instructions executable by the processor, thereby operative to perform the steps and / or actions described herein. The instructions are typically configured as computer programs 225, 235 and may be pre-configured in the memory 220 or downloaded from an external memory device 230. Optionally, the system 200 comprises an input / output interface 240 that may be interconnected to one or more processors 210 and / or memories 220 to allow input and / or output of relevant data, such as input parameter(s) and / or result output parameter(s).

[0191] In a particular example, the memory includes a set of instructions executable by the processor, such that the processor: obtaining at least one representation of energy resolved X-ray data; - performing at least two basis material decompositions based on said at least one representation of energy resolved X-ray data to generate at least two sets of original basis image representations; - extracting at least two basis image representations from at least two of said original sets of basis image representations; - Processing the basis image representations extracted by the machine learning based data processing to generate at least one representation of output image data.

[0192] The term "processor" should be interpreted in a general sense as a system or device that can execute program code or computer program instructions to perform a specific processing, decision-making, or computational task.

[0193] Thus, processing circuitry, including one or more processors, when executing computer programs is configured to perform well-defined processing tasks as described herein.

[0194] The processing circuitry need not be dedicated solely to performing the steps, functions, procedures and / or blocks described above, but may also perform other tasks.

[0195] The proposed approach also provides a computer program product comprising a computer readable medium 220, 230 having stored thereon such a computer program.

[0196] As an example, the software or computer program 225, 235 may be realized as a computer program product, which is typically carried or stored on a computer-readable medium 220, 230, particularly a non-volatile medium. The computer-readable medium may include one or more removable or non-removable memory devices, including but not limited to a read-only memory (ROM), a random access memory (RAM), a compact disc (CD), a digital versatile disc (DVD), a Blu-ray disc, a universal serial bus (USB) memory, a hard disk drive (HDD) storage device, a flash memory, a magnetic tape, or any other conventional memory device. Thus, the computer program may be loaded into the operating memory of a computer or equivalent processing device for execution by its processing circuitry.

[0197] A method flow can be considered as a computer action flow when executed by one or more processors. The corresponding device, system and / or apparatus may be defined as a group of functional modules, where each step executed by a processor corresponds to a functional module. In this case, the functional modules are implemented as computer programs executed on a processor. Thus, the device, system and / or apparatus may alternatively be defined as a group of functional modules, where the functional modules are implemented as computer programs executed on at least one processor.

[0198] Thus, the computer programs resident in the memory may be organized as appropriate functional modules configured to perform at least a portion of the steps and / or tasks described herein when executed by the processor.

[0199] It may also be implemented primarily as a hardware module, or alternately with hardware. The degree to which software and hardware are used is purely an implementation choice.

[0200] It should be understood that the above-described embodiments are given by way of example only, and the proposed technology is not limited thereto. Those skilled in the art will appreciate that various modifications, combinations and changes can be made to the embodiments without departing from the scope of the present invention defined by the appended claims. In particular, different component solutions in different embodiments can be combined in other configurations, if technically possible. [Explanation of symbols]

[0201] 10: X-ray source 11: Gantry 12: Table 13: Isocenter 20: X-ray detector 21: Detector element 25: Analog processing circuit 30: Image processing system 31: Input 32: Basis material decomposition system 33: Machine learning system 34: Output 40: Digital processing circuit 41: X-ray controller 42: Gantry controller 43: Table controller 44: Detector controller 45: Memory 50: Computer 60: Operator console 100: X-ray imaging system 200: System 210: Processor 220: Memory 225: Computer program 230: External memory device 235: Computer program 240: Input / output interface 301: Digital / analog converter 302: Comparator 303: Digital counter

Claims

1. 1. A method for image reconstruction based on energy resolved X-ray data, comprising: Collecting (S1) at least one representation of energy resolved X-ray data; performing at least two basis material decompositions based on at least one representation of the energy resolved X-ray data to generate at least two sets of original basis image representations (S2); Obtaining or selecting at least two basis image representations from at least two of the original sets of basis image representations (S3); A step (S4) of data processing the at least two acquired or selected basis image representations based on machine learning to generate at least one representation of output image data, the data processing being configured to map the at least two acquired or selected basis image representations obtained from the at least two basis material decompositions to the at least one representation of the output image data based on machine learning; A method comprising:

2. The method of claim 1 , wherein the machine learning based data processing is configured to map the acquired or selected basis image representations to at least one representation of the output image data based on a trained neural network, a support vector machine, or a decision tree.

3. The method of claim 2 , wherein the trained neural network, support vector machine, or decision tree is trained on at least one of measured patient image data or phantom image data, or simulated image data based on a numerical phantom.

4. 3. The method of claim 2, wherein a trained neural network, support vector machine, or decision tree is trained on images of a first type and adapted for use with images of a second type using transfer learning.

5. 5. The method of claim 4, wherein the trained neural network, support vector machine, or decision tree is trained on non-energy-resolved CT images and adapted for use on energy-resolved photon-counting CT images using transfer learning.

6. The method of claim 1 , wherein the at least one representation of energy resolved X-ray data originates from an X-ray detector or an intermediate storage device and comprises energy resolved X-ray image data and / or energy resolved X-ray measurement data.

7. The method of claim 6 , wherein the X-ray detector is an energy-resolving photon counting detector, also called a multi-bin photon counting X-ray detector.

8. 2. The method of claim 1, wherein at least one representation of the output image data includes a set of basis image representations, called an output set of basis image representations, and the number of basis image representations in the output set is greater than the maximum number of basis image representations in any of the at least two original sets of basis image representations.

9. 9. The method of claim 8, wherein the basis image representation of the output set has a plurality of basis functions, each of the original basis image representation sets having a corresponding number of basis functions, and the basis functions corresponding to the basis image representation of the output set are a subset of or equal to the basis functions corresponding to at least two of the original basis image representation sets.

10. 9. The method of claim 8, wherein the basis image representations of the output set have a plurality of basis functions, each of the original basis image representation sets having a corresponding number of basis functions, and the number of the obtained or selected basis image representations is equal to or greater than the number of basis functions corresponding to the basis image representations of the output set.

11. The method of claim 8 , wherein the original set of basis image representations is based on a 2-basis decomposition and the output set of basis image representations is based on a 3-basis decomposition.

12. 12. The method of claim 11, wherein the machine learning based data processing is performed by a neural network, the neural network being trained with several 2-basis image representation sets as input data and several 3-basis image representation sets as output data.

13. The method of claim 12 , wherein the two-basis image representation set and the three-basis image representation set are generated from measured patient image data or phantom image data, or from simulated image data based on a numerical phantom.

14. The step (S4) of processing the acquired or selected basis image representation based on machine learning includes: combining the obtained or selected basis image representation with at least one representation of non-base image data; processing the obtained or selected basis image representation together with at least one representation of non-basis image data by machine learning based data processing; The method of claim 1 , comprising:

15. The step (S4) of processing the acquired or selected basis image representation based on machine learning includes: combining the obtained or selected basis image representations with a representation of at least one prior information; - processing the obtained or selected basis image representations together with a representation of at least one prior information by machine learning based data processing; The method of claim 1 , comprising:

16. The step (S4) of processing the acquired or selected basis image representation based on machine learning includes: combining the acquired or selected basis image representations with at least one computational representation of a physical process in an acquisition system; - processing the acquired or selected basis image representations with at least one computational representation of a physical process by machine learning based data processing; The method of claim 1 , comprising:

17. The step (S4) of processing the acquired or selected basis image representation based on machine learning includes: combining the acquired or selected basis image representations with at least one computational representation of a statistical model of the measurement data; - processing the acquired or selected basis image representations with at least one computational representation of a statistical model by machine learning based data processing; The method of claim 1 , comprising:

18. The method of claim 1 , wherein at least one representation of the output image data comprises an output set of basis image representations and / or material decomposition reconstructed images.

19. The method of claim 1, wherein the machine learning based data processing includes deep learning post-processing configured based on the training data, the deep learning post-processing configured to receive a biased material decomposition sinogram as input and provide an unbiased material decomposition sinogram or an unbiased material decomposition reconstruction image as output.

20. 20. The method of claim 19, wherein a set of sinograms is provided as input for a two-basis decomposition to generate two or more basis image representations, and a deep learning neural network is used to convert the resulting basis image representations into a three-material decomposition.

21. 1. A system (30, 200) for image reconstruction based on energy resolved X-ray data, comprising: an input (31, 240) for receiving at least one representation of energy resolved X-ray data; a basis material decomposition system (32; 210, 225) configured to perform at least two basis material decompositions based on at least one representation of the energy resolved X-ray data to generate at least two sets of basis image representations; a trained machine learning system (33, 210, 225) having a configuration determined based on training data of input-output pairs, the machine learning system (33, 210, 225) configured to receive as input at least two basis image representations from at least two sets of basis image representations and to perform machine learning-based data processing on the received at least two basis image representations to generate at least one representation of output image data, the machine learning system (33, 210, 225) configured to map the received at least two basis image representations obtained from the at least two basis material decompositions to the at least one representation of output image data; an output (34, 240) for outputting at least one representation of the output image data; An image reconstruction system (30, 200) comprising:

22. The image reconstruction system of claim 21 , wherein the machine learning system (33, 210, 225) is based on a neural network, a support vector machine, or a decision tree-based implementation.

23. The image reconstruction system of claim 22, wherein the machine learning system (33, 210, 225) is trained on at least one of measured patient image data or phantom image data, or on simulated image data based on a numerical phantom.

24. The image reconstruction system of claim 22, wherein the machine learning system (33, 210, 225) is trained on images of a first type and adapted for use on images of a second type using transfer learning.

25. The image reconstruction system of claim 24, wherein the machine learning system (33, 210, 225) is trained on non-energy-resolved CT images and adapted for use on energy-resolved photon-counting CT images using transfer learning.

26. The image reconstruction system of claim 22, wherein at least one representation of the energy resolved X-ray data originates from the X-ray detector (20) or the intermediate storage device (220) and comprises energy resolved X-ray image data and / or energy resolved X-ray measurement data.

27. The image reconstruction system of claim 26, wherein the X-ray detector (20) is an energy-resolving photon counting detector, also called a multi-bin photon counting X-ray detector.

28. The image reconstruction system of claim 22, wherein at least one representation of the output image data includes a set of basis image representations, called an output set of basis image representations, and the number of basis image representations in the output set is greater than the maximum number of basis image representations in any of the at least two original sets of basis image representations.

29. 29. The image reconstruction system of claim 28, wherein the basis image representation of the output set has a plurality of basis functions, each of the original basis image representation sets having a corresponding number of basis functions, and the basis functions corresponding to the basis image representation of the output set are a subset of or equal to the basis functions corresponding to at least two of the original basis image representation sets.

30. The image reconstruction system of claim 22 , wherein at least one representation of the output image data comprises an output set of basis image representations and / or material decomposition reconstructed images.

31. 23. The image reconstruction system of claim 22, wherein the system comprises at least one processor and a memory, the memory comprising instructions that, when executed by the at least one processor, cause the at least one processor to perform image reconstruction.

32. The image reconstruction system of claim 22, wherein the basis material decomposition system comprises processing circuitry configured to generate at least two sets of basis image representations, and the trained machine learning system comprises processing circuitry configured to generate at least one representation of the output.

Images