Machine learning image processing that does not rely on reconstruction filters

Abstract

A method for processing an image is provided that includes retrieving measured data of a first image. The method then generates partially filtered data by applying a first filter to the measured data. The first filter is a generic filter. The method then reconstructs the partially filtered data to generate a partially filtered image. The method then generates a partially processed image by applying a first processing routine to the partially filtered image. The method then generates a filtered image by applying a second filter to the partially processed image, the second filter being a filter selected from a plurality of potential secondary filters. The method then outputs the filtered image.

Description

[Technical field]

[0001]

[0001] The present disclosure relates generally to systems and methods for processing images using trained neural networks. [Background technology]

[0002]

[0002] Traditionally, acquiring images through standard imaging modalities such as computed tomography (CT) scans results in image artifacts and noise embedded in such images. Furthermore, systems that process such images may attempt to preemptively obtain some information about the content of the scan, such as identifying the content. Thus, typically, images are first filtered and reconstructed to convert measurement data into an image, and then processed using algorithms, for example, for noise removal, segmentation, or preemptively identifying content.

[0003]

[0003] Thus, images are typically taken as measurement data and reconstructed using reconstruction filters. Such filters can be used to promote certain image characteristics, such as sharpness or soft tissue contrast. As an example, when reconstructing a head image, the system can use a filter designed to promote soft tissue contrast in brain tissue, while a different filter can be used to reconstruct a body image. In contrast, the filter used to reconstruct a body image can be sharper. Summary of the Invention [Problem to be solved by the invention]

[0004]

[0004] The reconstructed image is generally noisy and may contain various artifacts of the reconstruction process, and then various denoising algorithms, among others, may be applied to improve the image quality. Such algorithms may be learning algorithms such as convolutional neural networks (CNNs). Thus, when using learning algorithms such as CNNs to process CT scans, different filters may be used during image reconstruction to promote image features such as sharpness or to suppress different noise types. The denoising of the reconstructed image should then be performed in a way that preserves the image characteristics promoted by the filters used for reconstruction.

[0005]

[0005] Thus, learning algorithms such as CNNs can be trained for specific types of anatomical structures and specific reconstruction filters. Thus, the state of the art is to train different CNNs for different anatomical regions and reconstruction filters designed for such regions. This is particularly beneficial when filters are designed to produce very specific image characteristics in specific anatomical structures, and it is a well-understood goal to have image processing algorithms that are robust with respect to changes in reconstruction settings, such as those reflected in filters.

[0006]

[0006] Using different reconstruction filters results in different image and noise characteristics of the reconstructed image. Therefore, when a machine learning algorithm is trained on a specific reconstruction filter, the performance may be suboptimal or completely fail on other reconstruction filters, since the machine learning method generally tends to overfit the training data, and therefore often cannot generalize to parameter settings that are not sampled in training.

[0007]

[0007] Thus, a CNN may require training for all parameter settings for potential reconstruction filters to achieve the best possible results. However, training and maintaining a CNN for a large number of reconstruction filters is time consuming and typically not feasible.

[0008] What is needed are machine learning algorithms and processes for CT image processing that can be made completely independent of the reconstruction filter selected. [Means for solving the problem]

[0009]

[0009] A method and system are provided for machine learning based image processing that is independent of and independent of the reconstruction filter used during image processing.

[0010] There is provided a method for processing an image comprising deriving measurement data of a first image, the measurement data being either in the frequency domain or in a domain other than the frequency domain, which may be a spatial domain such as a projection domain or an image domain.

[0011]

[0011] The method then generates partially filtered data by applying a first filter to the measurement data. The first filter is a universal filter. The method then reconstructs the partially filtered data to generate a partially filtered image. The method then generates a partially processed image by applying a first processing routine to the partially filtered image.

[0012]

[0012] The method then generates a filtered image by applying a second filter to the partially processed image, the second filter being a filter selected from a plurality of potential secondary filters. The method then outputs the filtered image.

[0013] In some embodiments, the measured data is provided in a domain other than the frequency domain, and the method further includes first converting any measured data provided to the frequency domain. Generating the partially filtered data is then performed by applying a first filter to the measured data in the frequency domain, and reconstructing includes converting the partially filtered data to the image domain.

[0014]

[0014] In some such embodiments, the second filter is also applied in the frequency domain. Thus, generating a filtered image includes extracting partially processed data from the partially processed image and transforming the partially processed data into the frequency domain. The method then generates the partially processed data by applying the second filter in the frequency domain.

[0015]

[0015] The method then transforms the filtered partially processed data into the image domain to produce a filtered image.

[0016] In some embodiments, the measurement data includes projection data of a CT image. In some such embodiments, the reconstruction of the partially filtered data is performed by backprojecting the partially filtered data.

[0017] In some embodiments, the first processing routine is a first machine learning algorithm trained on the measurement data that has been filtered by applying the first filter but not the second filter. In some such embodiments, the first processing routine is a denoising routine, an image segmentation routine, or a diagnostic prediction routine.

[0018] In some embodiments, the first filter is a lamp filter.

[0019] In some embodiments, each of the multiple potential secondary filters, when applied to the partially processed image, produces different image and noise characteristics in the resulting filtered image, and a filtered image resulting from application of a second filter to the partially processed image differs from a hypothetical filtered image resulting from application of a different one of the multiple potential secondary filters.

[0020] In some such embodiments, for example, the first potential second filter is a soft reconstruction filter and the second potential second filter is a sharp reconstruction filter.

[0021] In some such embodiments, the second filter is selected from a plurality of potential secondary filters based on the type of body part or tissue represented in the first image.

[0022]

[0022] In some embodiments, the method further includes evaluating the partially processed image and outputting a result of the evaluation of the partially processed image before or together with the filtered image.

[0023]

[0023] In some embodiments, the method further includes evaluating the partially processed image before generating the filtered image, and selecting a second filter for application based at least in part on the evaluation of the partially processed image.

[0024] In some such embodiments, the first processing routine is an image segmentation routine, and the partially processed image is segmented into a plurality of segments, and different second filters selected from a plurality of potential secondary filters are applied to different segments of the plurality of segments.

[0025]

[0025] An imaging system is also provided, which may include a memory storing a plurality of instructions, an imaging unit, a database storing a plurality of potential secondary filters, and a processing circuit coupled to the memory and configured to execute the instructions to perform the method as described above.

[0026]

[0026] Also provided is a method for training a neural network model for use in an image processing system. The method includes retrieving sample measurement data for an image of a target, and retrieving a first target image related to the sample image data for use as ground truth. The target image data may also be retrieved after the process.

[0027]

[0027] Next, the method proceeds by generating partially filtered sample measured data by applying a first filter to the sample measured data, the first filter being a generic filter.

[0028]

[0028] The method then reconstructs the partially filtered data to generate a partially filtered image.

[0029]

[0029] The method then applies a first processing routine based on a neural network model that has been trained on the partially filtered image to generate a partially filtered image, and generates a first filtered image by applying a second filter to the partially processed image, the second filter being a filter selected from a plurality of potential secondary filters.

[0030]

[0030] The method then evaluates the output of the processing routine by comparing the first filtered image to a first target image, the target image being associated with a second filter.

[0031] In some embodiments, the first target image is one of a plurality of target images associated with the sample image data, each of the plurality of target images being associated with a different second filter of a plurality of potential secondary filters. In such embodiments, the method further includes generating a second filtered image by applying an alternative second filter to the partially processed image, the alternative second filter being selected from the plurality of potential secondary filters.

[0032]

[0032] The method then further evaluates the output of the processing routine by comparing the second filtered image to an alternative target image associated with an alternative second filter.

[0033]

[0033] Typically, the training method is repeated on sample measurement data for multiple images, and after each iteration of the method the neural network model is modified based on an evaluation of the output of the processing routines. [Brief description of the drawings]

[0034] [Figure 1] FIG. 1 is a schematic diagram of a system according to one embodiment of the present disclosure. [Diagram 2] 1 illustrates an exemplary imaging device according to one embodiment of the present disclosure. [Diagram 3] 4 shows a sample image reconstructed using a soft reconstruction filter. [Figure 4] Shows the sample image of Figure 3 reconstructed with a sharp reconstruction filter. [Diagram 5] We demonstrate the use of the Fourier slice theorem to transform projection data between the projection domain and the frequency domain coupled to the image domain. [Figure 6] Transformation of the raw data into the frequency domain is shown. [Figure 7] 1 illustrates a ramp filter for use in processing image data in the frequency domain. [Figure 8] Transformation of the filtered data into the projection domain is shown. [Figure 9] 1 illustrates a method for processing an image according to the present disclosure. [Figure 10] 1 illustrates a method for training a neural network model according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035]

[0044] The description of exemplary embodiments according to the principles of the invention is intended to be read in conjunction with the accompanying drawings, which are to be considered as part of the entire written description. In describing the embodiments of the invention disclosed herein, any reference to direction or orientation is intended for convenience of description only and is in no way intended to limit the scope of the invention. Relative meanings such as "bottom", "top", "horizontal", "vertical", "up", "bottom", "up", "top", "top", "bottom", and derivatives thereof (e.g., "horizontal", "bottom", "up", etc.) should be interpreted to refer to the orientation as then described or as shown in the drawings under discussion. These relative meanings are for convenience of description only and do not require that the device be constructed or operated in a particular orientation unless expressly so indicated. "Mounted", "mounted", "connected", "coupled", "interconnected", and similar terms refer to a relationship in which structures are fixed or attached to one another, both directly or indirectly through intervening structures, and through movable or rigid attachments or relationships, unless expressly stated otherwise. Furthermore, the features and advantages of the present invention are illustrated by reference to the illustrated embodiments, and therefore the present invention should not be limited to such illustrative embodiments, which show some possible non-limiting combinations of features that may exist alone or in combination of features, and the scope of the present invention is defined by the claims appended hereto.

[0036]

[0045] This disclosure describes the best mode of carrying out the present invention currently contemplated. This description is not intended to be understood in a limiting sense, but provides examples of the present invention presented for illustrative purposes only, with reference to the accompanying drawings, to advise those skilled in the art of the advantages and structure of the present invention. In the various views of the drawings, like reference numerals indicate like or similar parts.

[0037]

[0046] It is important to note that the disclosed embodiments are merely examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of this application do not necessarily limit any of the various claimed disclosures. Moreover, some statements may apply to some inventive features but not to others. In general, unless otherwise stated, singular elements may be in the plural and vice versa without loss of generality.

[0038]

[0047] Typically, images are reconstructed using filters, the filters being selected appropriately for the subject of the image being reconstructed. Thus, when reconstructing a head image, the system may use a filter designed to promote soft tissue contrast in brain tissue. However, when reconstructing a body image, a different, potentially sharper, filter may be used. Similarly, a different filter may suppress noise instead of, or in combination with, promoting image sharpness.

[0039]

[0048] Furthermore, in the context of computed tomography (CT)-based medical imaging, different image processors may be used to process the images, such as machine learning algorithms, which may take the form of convolutional neural networks (CNNs), which are then trained, in the case of machine learning algorithms, on the corresponding different anatomical regions and structures in the context of appropriate reconstruction filters.

[0040]

[0049] Since CNNs are trained in the context of a particular reconstruction filter, such training is specific to the filter used during training. Therefore, in order for CNNs to be universally usable to process filtered images, CNNs are trained independently on any filter that may be applied prior to such processing. In the embodiments described herein, the reconstruction filter for a particular body part or set of desired characteristics is split into two separate reconstruction filters, a general reconstruction filter for promoting the desired characteristics and a target reconstruction filter.

[0041]

[0050] Thus, the systems and methods described herein first apply a generic filter, or a generic portion of a filter, to measurement data associated with an image. The image is then reconstructed using the generic filter to provide a partially filtered image as a generic reconstruction, and a processing routine is applied to the partially filtered image. Only after the desired processing routine is applied to the partially filtered image is a second target reconstruction filter applied to the partially processed image. The target reconstruction filter may, for example, be one of multiple potential secondary filters used to promote desired characteristics.

[0042]

[0051] The processing routine may then be a machine learning algorithm such as a CNN trained on data filtered with the generic filter, and the relevant ground truth data used for training may be data filtered with the generic filter only. Alternatively, as discussed below with respect to FIG. 10, the CNN may be trained on a fully filtered image used as ground truth, and the training method may then apply the target reconstruction filter after applying the processing routine and before evaluating the results of the filtering. In either case, the machine learning algorithm may be generically usable, since it is trained and applied prior to the use of a particular target reconstruction filter.

[0043]

[0052] Medical imaging other than CT, such as MRI (magnetic resonance imaging) and PET (positron emission tomography), can use different methods for reconstruction to recreate and process images. Thus, different reconstruction algorithms can be used depending on the type of scan or data acquisition, as well as the desired image characteristics for a particular scan. The reconstruction algorithms can then be adjusted using filters to obtain specific image characteristics. In this disclosure, embodiments are described for CT imaging. However, it will be understood that the methods and systems described herein can be used in the context of other images that similarly rely on filtering for reconstruction.

[0044]

[0053] In some embodiments, other techniques may be used by those skilled in the art to modify the frequency content of data in the image or raw data domain in addition to Fourier analysis or convolution. Such techniques may be, but are not limited to, wavelet analysis, multi-scale analysis such as Laplacian pyramid decomposition, and other transformations such as discrete cosine transform. It will be understood that the discussion of filtering herein includes such processing techniques insofar as it can typically be divided into first, general processing techniques, and second, targeted processing techniques.

[0045]

[0054] 1 is a schematic diagram of a system 100 according to one embodiment of the present disclosure. As shown, the system 100 typically includes a processing device 110 and an imaging device 120.

[0046]

[0055] The processing device 110 can apply processing routines to images or measurement data, such as projection data received from the imaging device 120. The processing device 110 can include a memory 113 and a processor circuit 111. The memory 113 can store a number of instructions. The processor circuit 111 can be coupled to the memory 113 and configured to execute the instructions. The instructions stored in the memory 113 can comprise the processing routines, as well as data associated with the processing routines, such as machine learning algorithms, and various filters for processing the images. It will be appreciated that while all data is described as being stored in the memory 113, in some embodiments some data, such as potential filters for use in processing measurement data associated with the images, can itself be stored in the memory or can be stored in a database, which can be stored in a separate system.

[0047]

[0056] The processing device 110 may further include an input 115 and an output 117. The input 115 may receive information, such as images or measurement data, from the imaging device 120. The output 117 may output information, such as a filtered image, to a user or a user interface device. The output may include a monitor or display.

[0048]

[0057] In some embodiments, the processing device 110 may be directly associated with the imaging device 120. In alternative embodiments, the processing device 110 may be separate from the imaging device 120, receiving image or measurement data for processing at input 115 via a network or other interface.

[0049]

[0058] In some embodiments, imaging device 120 may include an image data processing device and a spectral or conventional CT scanning unit for generating CT projection data when scanning an object (eg, a patient).

[0050]

[0059] 2 illustrates an exemplary imaging device 200 according to one embodiment of the present disclosure. Although a CT imaging device is illustrated and the following description is in the context of CT images, it will be understood that similar methods may be applied in the context of other imaging devices, and that the images to which these methods may be applied may be obtained in a wide variety of ways.

[0051]

[0060] In the imaging device according to the embodiment of the present disclosure, the CT scanning unit may be adapted to perform multiple axial and / or helical scans of the object to generate CT projection data. In the imaging device according to the embodiment of the present disclosure, the CT scanning unit may comprise an energy-resolving photon counting image detector. The CT scanning unit may include a radiation source that emits radiation for traversing the object when acquiring the projection data.

[0052]

[0061] 2, a CT scanning unit 200, e.g., a computed tomography (CT) scanner, may include a fixed gantry 202 and a rotating gantry 204 that may be rotatably supported by the fixed gantry 202. The rotating gantry 204 may rotate about a longitudinal axis about an examination region 206 of the object when acquiring projection data. The CT scanning unit 200 may include a support 207 for supporting a patient in the examination region 206 and is configured to move the patient through the examination region during the imaging process.

[0053]

[0062] The CT scanning unit 200 may include a radiation source 208, such as an X-ray tube, which may be configured to be supported by and rotate with a rotating gantry 204. The radiation source 208 may include an anode and a cathode. A power supply voltage applied between the anode and the cathode may accelerate electrons from the cathode to the anode. A current flow may be provided from the cathode to the anode such that the electron flow generates radiation to traverse the examination region 206.

[0054]

[0063] The CT scanning unit 200 may include a detector 210. The detector 210 may subtend an angled arc on the opposite side of the examination region 206 relative to the radiation source 208. The detector 210 may include a one-dimensional or two-dimensional array of pixels, such as direct conversion detector pixels. The detector 210 may be adapted to detect radiation traversing the examination region 206 and generate a signal indicative of its energy.

[0055]

[0064] The CT scanning unit 200 may further include generators 211, 213. The generator 211 may generate tomographic projection data 209 based on signals from the detector 210. The generator 213 receives the tomographic projection data 209 and, in some embodiments, may generate a raw image 311 of the object based on the tomographic projection data 209. In some embodiments, the tomographic projection data 209 may be provided to an input 115 of the processing device 110, while in other embodiments the raw image 311 is provided to an input of the processing device.

[0056]

[0065] 3 and 4 show the same image processed with two separate filters resulting in different processed images. Measurement data corresponding to the image, such as projection data 209 or raw image 311, is typically received at the input 115 of the processing device 110. The projection data 209 or raw image 311 is then processed to reconstruct the image and improve the image quality. The processing of the image in the embodiments discussed herein includes filtering the measurement data in the form of the projection data 209 or raw image 311 with a generic filter, performing a processing routine, and filtering the image again by applying a second targeted filter. The processing routine includes implementing a machine learning algorithm, such as a convolutional neural network (CNN).

[0057]

[0066] The second filter is typically a targeted reconstruction filter designed for a particular anatomical region, such as a head or body filter, or for a particular type of tissue, such as bone or soft tissue. A head filter may be specifically designed to promote soft tissue contrast in brain tissue, while a body filter may be designed to provide sharper results. As shown, FIG. 3 shows a cross-section through the head, where an image was reconstructed using a soft reconstruction filter. FIG. 4 shows the same cross-section with the image reconstructed using a sharp reconstruction filter.

[0058]

[0067] Figure 5 illustrates the use of the Fourier slice theorem to transform projection data between the projection domain and the frequency domain coupled to the image domain. As shown, the two-dimensional Fourier spectrum of the image can be obtained by inserting the one-dimensional Fourier transform of the filtered signal profile as a line passing through the spectral origin with the respective angle θ at which the corresponding profile was obtained.

[0059]

[0068] It should be noted that in the following description, reference is made to the spatial domain as well as the image domain, the projection domain, and the frequency domain. The spatial domain is referred to in contrast to the frequency domain, but both the projection domain and the image domain are spatial domains. Thus, when a spatial domain is generally referred to, it is understood that such a domain can be either the projection domain, the image domain, or a different number of spatial domains, depending on the type of image and the stage of processing considered.

[0060]

[0069] Thus, a CT image is first recorded as a set of projections 500 in the sinogram or projection domain, each taken at an angle θ and corresponding to a function p(θ, r). As shown, if the image of the object 510 being processed is two-dimensional, each projection 500 is transformed into frequency space by a one-dimensional Fourier transform, resulting in a single line passing through the origin at angle θ 520. Once the two-dimensional Fourier transform of the object 530 has been constructed in frequency space by multiple projections taken at different angles, the object can then be recreated 540 in a real space, such as the image domain, by applying a two-dimensional inverse Fourier transform.

[0061]

[0070] The examples shown are two-dimensional images processed as one-dimensional lines, but it will be understood that the same approach works for higher dimensions as well. Thus, if a three-dimensional image is being processed, the filtering is on a two-dimensional plane that passes through the origin of the three-dimensional frequency space at the acquisition angle of the projection view.

[0062]

[0071] Although the object 510 is represented in frequency space, the filter can be applied by multiplying the two-dimensional image spectrum 530 with a rotationally symmetric two-dimensional filter spectrum. Thus, the reconstruction filter can be applied in the frequency domain before reconstructing the object representation in the spatial domain.

[0063]

[0072] Because application of a filter in the frequency domain involves multiplication, the filter may be divided into components, such that some components of the filter may be general across a type of imaging, while other components of the filter are specific to the characteristics that the filter seeks to enhance or suppress in a particular image or class of images.

[0064]

[0073] Thus, in the illustrated embodiment involving CT imaging, the measurement data may be projection data 500. Due to the nature of the Fourier slice theorem, transformation of this data into frequency space results in lines passing through the origin at different angles θ. An infinite number of projections can be taken, but any representation in the frequency domain will be over-represented near the origin and under-represented further from the origin. Thus, higher frequencies are oversampled and lower frequencies are sampled. To adjust for this, a generic filter may be applied to all of the CT data, such as forming a ramp filter.

[0065]

[0074] In practice, after converting the image spectrum 530 back to the spatial domain in the frequency domain, one can apply, for example, an inverse fast Fourier transform (iFFT) to the lines of the spectrum, which can then be "backprojected" into image space. This is equivalent to implementing a two-dimensional inverse Fourier transform, as shown in FIG. 5. However, such a backprojection will be blurry, since the lower frequencies are not sampled. To sharpen the image, the frequency data may be multiplied by a ramp filter before converting the data back to the spatial domain and backprojecting it. This process results in a sharper image and is called "filtered backprojection" or FBP.

[0066]

[0075] The components of the filter that are specific to the characteristic that the filter is trying to suppress can then be split out from the reconstruction filter as a different second filter that can be applied separately. This approach can result in a separate generic filter, such as the ramp filter shown in FIG. 7, and a targeted second filter. Such a second filter can be applied in the frequency domain by converting data corresponding to the image to the frequency domain, multiplying the frequency data with the second filter, and then converting the data back to the image domain. Alternatively, the second filter can be applied in the spatial domain by applying a convolution of the filter kernel independently of the generic filter.

[0067]

[0076] This second filter is therefore an uncommon target filter and may be a Modulation Transfer Function (MTF). It will be appreciated that although Modulation Transfer Functions are typical in the context of CT imaging, different types of filters or secondary processing routines may be applied in other imaging modalities.

[0068]

[0077] Because convolution in the image or spatial domain is equivalent to multiplication in the frequency domain, filter application may be less resource intensive in the frequency domain than in the image domain, and therefore there may be advantages in processing speed or power requirements to transform data into the frequency domain for applying the filter.

[0069]

[0078] FIG. 6 shows the conversion of raw data to the frequency domain. FIG. 7 shows a ramp filter for use in processing image data in the frequency domain. In the spatial domain, such as the projection domain, each projection p φ (x) can be thought of as a function of X. By transforming the measured data into the frequency domain, such as by applying a Fast Fourier Transform (FFT), each projection TIFF2025500162000002.tif1418 can be thought of as a function of ω, which allows the application of filters in the frequency domain by multiplication.

[0070]

[0079] When transforming the measured data into the frequency domain, the higher frequencies ω are suppressed, as shown in Figure 6. To correct for this, or any other artifacts that are consistent across the set of measured data, a general purpose filter, such as the ramp filter shown in Figure 7, can be applied to the data in the frequency domain. Thus, for CT data, where sampling of higher frequencies is a known issue, for example, due to the method of data collection, an initial general purpose filter can be applied.

[0071]

[0080] Figure 8 shows the filtered data in the frequency domain. Transformation of TIFF2025500162000003.tif1214 into the spatial domain. As shown, after filtering, the filtered data TIFF2025500162000004.tif1214 is no longer artificially suppressed at high frequencies. We can therefore apply an inverse FFT to bring the data back into the spatial domain. In the case of a CT scan, the resulting data q φ (x) remains within the projection domain and can then be backprojected into the image domain to construct an image 540 associated with the processed data, as shown in FIG.

[0072]

[0081] The reconstruction filter can be separated into a general first filter and a targeted second filter, and the filters can be combined by simple multiplication in the frequency domain, so that the image can be first filtered with the first filter, reconstructed, and then separately filtered with the second filter and reconstructed. The image can then be partially filtered and reconstructed based on the first filter, and then processed using a machine learning algorithm. The image can then be further filtered using the targeted portion of the filter, or the MTF.

[0073]

[0082] Since the machine learning algorithm is applied before the second filter that includes the target portion of the filter or MTF, the machine learning algorithm is independent of the second reconstruction filter. Thus, a single machine learning algorithm can be trained to support a large set of different second filters.

[0074]

[0083] FIG. 9 illustrates a method for processing an image according to the present disclosure.

[0075]

[0084] As shown, as described above, the processing device 110 may first receive (900) at the input 115 measured data, such as raw CT data 209, for a first image from the imaging device 120. The measured data may be in the frequency domain or in some domain other than the frequency domain. For example, in the case of CT data, the measured data may be provided in a spatial raw data domain, such as a sinogram or projection domain. Raw data for other types of images, such as MRI, may be provided in the frequency domain. Alternatively, the received measured data may be pre-processed into the image domain as a raw image 311.

[0076]

[0085] In any case, once the measurement data is received by the processing device 110 (at 900), the processing device can proceed by generating partially filtered data (910) by applying a first filter to the measurement data. The first filter is a generic filter that is applicable to data corresponding to any image of the group of images being processed by the method. For example, the first filter may be a ramp filter that is applicable to any CT image being processed, regardless of what body type or tissue type appears in the first image. In this context, a "generic" filter is a filter that can be used for a relatively broad set of images and a broad set of context data. For example, a generic filter may be applicable to all images drawn from a particular source or acquired via a particular imaging modality. In contrast, a targeted filter, such as the second filter described below, may be specific to a narrower group of images, such as images related to a particular body part or tissue type.

[0077]

[0086] In some embodiments, the generation of the partially filtered data is optionally performed by first transforming any measured data into the frequency domain (920) if it is in a domain other than the frequency domain. This may be, for example, by applying an FFT if the measured data is first provided in a spatial domain, such as the projection or image domain, as shown in FIG. 6. A first filter is then applied (930) to the measured data in the frequency domain to generate the partially filtered data, as shown in FIG. 8. If the first filter is applied in the frequency domain, it may be applied as a simple multiplication. Alternatively, if the first filter is applied in the spatial domain, the application may take the formation of a convolution of the image with a corresponding filter kernel.

[0078]

[0087] After filtering the measurement data to generate partially filtered data, the partially filtered data is reconstructed (940) to generate a partially filtered image. If the first filter was applied in the frequency domain, such reconstruction includes transforming the partially filtered data to the image domain. This may be implemented using techniques such as filtered backprojection using the first filter, as described above with respect to FIG. 5.

[0079]

[0088] A processing routine is then applied to the reconstructed partially filtered image (950) to generate a partially processed image (960). Such a processing routine is typically a machine learning algorithm, such as a convolutional neural network, that is trained on the measurement data that has been filtered by applying the first filter but not the second filter. Such a processing routine may be, for example, a denoising routine, an image segmentation routine, or a diagnostic prediction routine.

[0080]

[0089] A second filter is then selected (970) from the plurality of potential secondary filters for application to the image. The partially processed image is then filtered with a second filter (980) to generate a filtered image (990). The second filter is typically a targeted reconstruction filter designed for a particular anatomical region or for a particular type of tissue. In some embodiments, the second filter is a modulation transfer function (MTF). The plurality of second filters may be stored in a database that may be provided in the system memory 113. Alternatively, the database of potential second filters may be provided independently of the system. In some embodiments, the second filter may be designed, for example, by a third party vendor and thus may be a standalone database.

[0081]

[0090] Thus, unlike the first filter, the second filter is not a universal filter, and different filters from the plurality of potential secondary filters, when applied to the partially processed image, will produce different image and noise characteristics in the resulting filtered image. Thus, a filtered image (990) resulting from application of the second filter (980) to the partially processed image will differ from a hypothetical filtered image (1010) resulting from application of a different filter (1000) from the plurality of potential secondary filters.

[0082]

[0091] In some embodiments, for example, one potential second filter (980) of the plurality of potential secondary filters may be a soft reconstruction filter resulting in a filtered image (990) similar to that shown in Figure 3, and an alternative potential second filter (1000) may be a sharp reconstruction filter resulting in a filtered image (1010) similar to that shown in Figure 4. The second filter used in the reconstruction may be selected based on the type of body part or tissue represented in the first image.

[0083]

[0092] In some embodiments, generating the filtered image (990) is by first extracting the partially processed data from the partially processed image and transforming the partially processed data into the frequency domain. In such embodiments, the filtered partially processed data may be generated by applying a second filter in the frequency domain. After applying the second filter, the filtered partially processed data (960) is transformed into the image domain to generate the filtered image.

[0084]

[0093] Alternatively, in some embodiments, generating a filtered image (990) is by processing in the image domain. Such a filter may be applied as a convolution of the partially processed image with a filter kernel.

[0085]

[0094] Once the filtered image (990) has been generated, it can be output (1020) to a user of the system, which may be via the system output 117, for example via a display.

[0086]

[0095] Although the methods described herein are described in the context of CT scan images or raw data, it will be understood that a variety of imaging techniques, including a variety of medical imaging techniques, are contemplated and images generated using a wide variety of imaging techniques may be effectively processed using the methods described herein.

[0087]

[0096] In some embodiments, a learning algorithm applied as a processing routine (950) may be utilized to provide insight to a user, either independently of or in combination with the resulting image. Thus, the partially processed image (at 960) may be evaluated (at 1030) prior to generation of a filtered image (at 990). For example, if the processing routine is a diagnostic prediction routine, the partially processed image (960) may be evaluated (1030) and the result may be output (1040) as a diagnosis and provided to the user before or together with the filtered image (at 1020).

[0088]

[0097] In some such embodiments, instead of outputting the results of the evaluation (1030) as a diagnosis (1040), the results of the evaluation can be used in the context of a method for processing the image. Thus, the evaluation (1030) can be used to inform the selection of a second filter (970), such that the second filter to be applied is based at least in part on the evaluation of the partially processed image. For example, if the processing routine (950) is an image segmentation routine, the routine can identify a segment of the image that includes a particular body part, and the identified segment can then be filtered with a second filter associated with that body part.

[0089]

[0098] In some such embodiments in which the processing routine (950) is an image segmentation routine, the partially processed image is segmented into a plurality of segments as part of evaluating the image (1030). In such embodiments, different second filters (980, 1000) selected from a plurality of potential secondary filters may be applied to different segments of the plurality of segments.

[0090]

[0099] 10 illustrates a method for training a learning algorithm, such as a neural network model. The method first involves providing or retrieving sample measurement data for an image of an object (1100). During the training process, the method also retrieves at least a first target image (1110) associated with the sample image data for use as ground truth. Thus, when training the method, the learning algorithm is a processing method that is applied to the sample measurement data during the course of training with the goal of approximating the first target image (1110).

[0091]

[0100] It is noted that in the present description, the method retrieves both sample measurement data (1100) and target images (1110) at the beginning of the process. Such data and images may be provided as a match set for training purposes. Alternatively, the target images may be provided (1160) after a second filter has been selected such that the target images are tuned to the second filter that will actually be used.

[0092]

[0101] The method then generates a partially filtered sample image (1130) from the measured data by applying a first filter (1120) to the sample measured data, where the first filter is a general purpose filter. As explained above with respect to Figure 9, this filtering can be in the spatial raw data domain, the frequency domain, or alternatively in the spatial domain, among others.

[0093]

[0102] After initially filtering the data by applying a first filter (1120), the method reconstructs the partially filtered data to generate a partially filtered image (1130). The method then applies a first processing routine to the partially filtered image (1140), where the first processing routine is based on the trained neural network model. Application of the first processing routine to the partially filtered image generates a partially processed image (1150).

[0094]

[0103] In some embodiments, the first target image (1110) corresponds to ground truth data filtered only by the first filter (1120). In such embodiments, the partially processed image (1150) is then compared to the first target image (1110) to evaluate the partially processed image and adjust the learning algorithm.

[0095]

[0104] Alternatively, in some embodiments, instead of evaluating the partially processed image (1150), the partially processed image is then filtered with a second filter (1160), which is a filter selected from a plurality of potential secondary filters. As discussed above with respect to Figure 9, the second filter may be drawn from a database stored in memory 113 that includes a plurality of potential secondary filters. The output of the second filter (1160) is then the filtered image (1170).

[0096]

[0105] In such an embodiment, a first target image (1110) corresponds to ground truth data that has been filtered by a first filter (1120) and further filtered by a second filter 1160 that is utilized to filter the training data. Thus, the output of the processing routine (1140) is evaluated (1180) by comparing the filtered image (1170) to the first target image (1110).

[0097]

[0106] The results of such evaluation can then be used to tune the learning algorithm, and such training can then be repeated on a large set of training data.

[0098]

[0107] In some embodiments, the first target image (1110) is one of a plurality of target images associated with the sample image data, and each of the plurality of target images is associated with a different second filter (1160, 1190) of a plurality of potential secondary filters.

[0099]

[0108] In such an embodiment, the method can further train the learning algorithm by applying an alternative potential filter (1190) to the partially processed image, where an alternative second filter is selected from the plurality of potential secondary filters, which then generates a second filtered image (1200).

[0100]

[0109] The output of the processing routine may then be further evaluated (1210) by retrieving an alternate target image associated with an alternate second filter and comparing the second filtered image (1200) to the alternate target image.

[0101]

[0110] In any such training routine, after evaluating the output of the routine, the process is repeated on sample measurement data for multiple images. Thus, the method retrieves (1100) additional sample measurement data and performs the method iteratively. With each iteration of the method, the neural network model is modified based on an evaluation of the output of the processing routine.

[0102]

[0111] The method according to the present disclosure may be implemented on a computer as a computer-implemented method, or on dedicated hardware, or a combination of both. The executable code for the method according to the present disclosure may be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Preferably, the computer program product may include non-transitory program code stored on a computer-readable medium for performing the method according to the present disclosure when said program product is executed on a computer. In an embodiment, the computer program may include computer program code adapted to perform all steps of the method according to the present disclosure when the computer program is executed on a computer. The computer program may be embodied on a computer-readable medium.

[0103]

[0112] While the present disclosure has been described at some length and in particular with respect to certain illustrated embodiments, it should not be limited to any such details or embodiments or to any particular embodiment, but should be construed with reference to the appended claims so as to provide the broadest possible interpretation of such claims in view of the prior art, and therefore so as to effectively encompass the intended scope of the present disclosure.

[0104]

[0113] All examples and conditional language recited herein are for educational purposes to aid the reader in understanding the principles of the present disclosure and concepts contributed by the inventors to further the art, and should not be construed as being limited to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the present disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Moreover, such equivalents are intended to include both currently known equivalents and equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

Claims

1. 1. A method for processing an image, comprising: retrieving measured data for a first image, the measured data being in a frequency domain or a domain other than the frequency domain; generating partially filtered data by applying a first filter to the measured data, the first filter being a generic filter; reconstructing the partially filtered data to generate a partially filtered image; generating a partially processed image by applying a first processing routine to the partially filtered image; generating a filtered image by applying a second filter to the partially processed image, the second filter being a filter selected from a plurality of potential secondary filters; outputting the filtered image; The method comprising:

2. 2. The method of claim 1, further comprising the step of first converting any measured data provided to the frequency domain if the measured data is in a domain other than the frequency domain, and generating the partially filtered data is performed by applying the first filter to the measured data in the frequency domain, and reconstruction comprises converting the partially filtered data to an image domain.

3. The step of generating a filtered image comprises: extracting partially processed data from the partially processed image and transforming the partially processed data into the frequency domain; generating filtered partially processed data by applying the second filter in the frequency domain; transforming the filtered partially processed data into the image domain to generate the filtered image; 3. The method of claim 2, comprising:

4. The method of claim 1 , wherein the measured data comprises projection data for a CT image.

5. The method of claim 4 , wherein the reconstruction of the partially filtered data is done by backprojecting the partially filtered data.

6. 2. The method of claim 1, wherein the first processing routine is a first machine learning algorithm trained on measured data that has been filtered by applying the first filter but not the second filter.

7. The method of claim 6 , wherein the first processing routine is a noise removal routine, an image segmentation routine, or a diagnostic prediction routine.

8. The method of claim 1 , wherein the first filter is a ramp filter.

9. 2. The method of claim 1, wherein each of the plurality of potential secondary filters, when applied to the partially processed image, produces different image and noise characteristics in a resulting filtered image, and the filtered image resulting from application of the second filter to the partially processed image differs from a hypothetical filtered image resulting from application of a different one of the plurality of potential secondary filters.

10. 10. The method of claim 9, wherein a first filter of the potential second filters is a soft reconstruction filter and a second filter of the potential second filters is a sharp reconstruction filter.

11. The method of claim 9 , wherein the second filter is selected from the plurality of potential secondary filters based on the type of body part or tissue represented in the first image.

12. 2. The method of claim 1, further comprising the steps of evaluating the partially processed image and outputting a result of the evaluation of the partially processed image before or together with the filtered image.

13. 10. The method of claim 1 , further comprising: evaluating the partially processed image prior to generating the filtered image; and selecting the second filter for application based at least in part on an evaluation of the partially processed image.

14. 14. The method of claim 13, wherein the first processing routine is an image segmentation routine, and the partially processed image is segmented into a plurality of segments, and different second filters selected from the plurality of potential secondary filters are applied to different ones of the plurality of segments.

15. 1. An imaging system, comprising: A memory for storing a plurality of instructions; An imaging unit; a database storing a plurality of potential secondary filters; a processing circuit coupled to the memory, obtaining measured data from the imaging unit, the measured data being in a frequency domain or a domain other than the frequency domain; generating partially filtered data by applying a first filter to the measured data, the first filter being a generic filter; reconstructing the partially filtered data to generate a partially filtered image; generating a partially processed image by applying a first processing routine to the partially filtered image; retrieving a second filter from the database, the second filter being selected from the plurality of potential secondary filters; generating a filtered image by applying the second filter to the partially processed image; outputting the filtered image; a processing circuit configured to execute instructions to perform the An imaging system comprising:

16. 16. The imaging system of claim 15, wherein the processing circuitry first transforms any measured data acquired in a domain other than the frequency domain into the frequency domain and transforms the partially filtered data into an image domain during reconstruction, the generation of the partially filtered image data being performed by applying the first filter to image data in the frequency domain, and the reconstruction of the partially filtered image data being performed by backprojecting the partially filtered data.

17. 16. The imaging system of claim 15, wherein the first processing routine is a first machine learning algorithm trained on measured data that has been filtered by applying the first filter but not the second filter.

18. 1. A method for training a neural network model, comprising: retrieving sample measured data for an image of the object; retrieving a first target image related to the sample image data for use as ground truth; generating partially filtered sampled data by applying a first filter to the sampled data, the first filter being a general purpose filter; reconstructing the partially filtered data to generate a partially filtered image; applying a first processing routine based on the trained neural network model to the partially filtered image to generate a partially processed image; generating a first filtered image by applying a second filter to the partially processed image, the second filter being a filter selected from a plurality of potential secondary filters; evaluating an output of the processing routine by comparing the first filtered image to a first target image, the target image being associated with the second filter; The method comprising:

19. the first target image is one of a plurality of target images associated with the sample image data, each of the plurality of target images being associated with a different second filter of the plurality of potential secondary filters, the method further comprising: generating a second filtered image by applying an alternative second filter to the partially processed image, the alternative second filter being selected from the plurality of potential secondary filters; further evaluating the output of the processing routine by comparing the second filtered image to an alternative target image associated with the alternative second filter; 20. The method of claim 18, further comprising:

20. 20. The method of claim 18, wherein the method is repeated on sample measured data for a number of images, and for each iteration of the method, the neural network model is modified based on an evaluation of the output of the processing routine.

Images