Noise suppression using deep convolutional networks

Abstract

A machine learning network is trained to generate noise field images from radiographic (x-ray) images. The training includes accessing a number of previously acquired radiographic images, duplicating the previously acquired radiographic images, and conditioning each of the duplicated images with simulated noise content to form a plurality of simulated low-exposure images. Each of the simulated low-exposure images is paired with its corresponding previously acquired image to form a learning pair. The machine learning network is trained to generate noise field images using the learning pairs of images. A noise-suppressed image of a subject can be generated by applying a scaling factor to at least a portion of a corresponding noise field image and combining the scaled noise field image with a currently captured image of the subject.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the field of digital image processing, and more particularly to a method for noise suppression in digital X-ray images. BACKGROUND

[0002] In order to provide radiographic images suitable for diagnostic purposes, some image enhancement measures are generally desirable. Among the basic properties governing the image quality of radiographic images are brightness, dynamic range, contrast of details (or detail contrast), sharpness of edges, image latitude, and occurrence of noise in the image.

[0003] Noise suppression is of considerable interest to practitioners using digital X-ray imaging. By minimizing or eliminating noise content, an image processing system can render images that more accurately represent anatomical or structural features useful in promoting diagnosis and treatment.

[0004] Noise suppression using deep convolutional neural networks (a particular artificial intelligence (AI) tool) is a relatively recent approach for achieving significantly lower noise in digital medical X-ray images. One advantage of this approach is that high frequency image content, such as high and low contrast edges, is better preserved compared to using traditional noise reduction methods based on low pass filtering. In addition, flat regions in the image maintain the original noise texture (albeit at a much lower level), preventing blocky texture of noise in these areas after image enhancement. This allows subtle image details to be more aggressively enhanced in post-capture image processing and rendering.

[0005] Noise suppression using deep convolutional neural networks can result in a meaningful dose reduction following the principle of As Low As Reasonably Achievable (ALARA). The International Commission on Radiological Protection (ICRP) has proposed the adoption of ALARA policies for radiation workers since 1977, and recently for patients undergoing X-ray imaging.

[0006] To address this problem, manufacturers and users of X-ray equipment have been working on both procedural techniques and threshold settings that help reduce exposure levels. For example, a technical chart providing recommended exposure settings for various conditions can be developed to meet the ALARA goal. These reduced settings can then be used in system tools that help control dose levels, such as automatic exposure control (AEC) and anatomical programmed radiography (APR).

[0007] While exposure reduction is a worthy goal, the implementation of exposure reduction should not compromise the ability of a radiological imaging system to provide diagnostic physicians. Incorrectly lowering the X-ray exposure level can result in poor quality images that have reduced diagnostic value. Images produced with too small an exposure can be characterized by problems such as excessive graininess and low contrast. Such images can be difficult to use and can potentially compromise diagnosis. In some cases, problems such as these can even require that the image be re-taken, severely compromising the effort to reduce overall patient exposure.

[0008] In industrial radiography, low dose is not a primary consideration, but noise suppression can have significant value. In fact, low noise images are often produced by using long exposure times (computed radiography and film), or by averaging a large number of frames at shorter exposure times per frame. For industrial and non-medical uses, exposure reduction can help reduce the amount of time required to obtain a suitable image, resulting in energy savings and productivity improvements.

[0009] Even with AI, the actual dose required for an X-ray capture is still determined by the size and contrast of the most critical features that need to be detected in the image. However, for many exams (e.g. verifying placement of hardware such as tubes and wires, pacemakers, or orthopedic implants, or measuring distances and angles of anatomical features to hardware), the dose can be reduced substantially. In addition, cleaner, nearly noise-free images can help reduce radiologist fatigue during long viewing sessions.

[0010] Conventional noise suppression techniques can be satisfactory in some cases, but often lack what is needed to provide high quality radiographic image results, particularly in cases where exposure reduction is most desirable. Conventional noise suppression approaches have followed a pattern developed for traditional X-ray processing, and these conventional methods have failed to accurately characterize and correct the noise behavior that is prevalent in images obtained from digital radiography systems.

[0011] Thus, it can be seen that there is a need for noise suppression techniques that can utilize artificial intelligence and machine learning capabilities, but allow for user adjustment and adaptation for tuning digital radiographic image content. SUMMARY

[0012] It is an object of the present disclosure to address the need for improved noise suppression for digital radiographic images, whether for medical use, or for industrial / commercial use.

[0013] These objects are given by way of illustrative example only, and such objects can exemplify one or more embodiments of the present disclosure. Other desirable objectives and advantages inherently can be gained from a reading and understanding of the following detailed description of embodiments of the present disclosure. The present invention is defined by the appended claims.

[0014] A novel aspect of the method of the present disclosure is that the amount of noise reduction produced by the neural network can be customized to the preferences of the practitioner, such as a physician and / or radiologist. In addition, additional features such as automatic angle and distance measurements or disease feature detection can work more reliably with additional enhancement by image processing as well as reduced image noise.

[0015] According to one aspect of the present disclosure, there is provided a computer- implemented method of generating a noise-suppressed radiographic image. The method includes training a machine learning network to generate a noise field image from a current radiographic image. The method further includes accessing a plurality of previously-acquired radiographic images; duplicating the previously-acquired radiographic images; conditioning each of the duplicated images with simulated noise content, thereby forming a plurality of simulated low-exposure images. Each of the simulated low-exposure images is paired with its corresponding previously-acquired image to form a learning pair. The machine learning network is trained to generate a noise field image using the learning pairs of images. Thereafter, the trained learning network can use a current captured image of a subject to form a corresponding noise field image therefrom. A noise-suppressed image of the subject can then be generated by applying a scaling factor to at least a portion of the corresponding noise field image and combining the scaled noise field image with the current captured image of the subject. The noise-suppressed image can be processed as a complete image in the usual course, such as by displaying, storing or transmitting the image. BRIEF DESCRIPTION OF DRAWINGS

[0016] The foregoing objects, features and advantages of the present invention as well as others will be more readily understood upon reading the following detailed description of embodiments of the present disclosure, taken in conjunction with the drawings in which:

[0017] Figure 1 is a schematic diagram showing the role of the convolutional network logic in generating a noise field and showing how the noise field is scaled and used to form a noise-suppressed image.

[0018] Figure 2 is a simplified schematic diagram showing how a convolutional network structure can be structured for noise field generation.

[0019] Figure 3 is a schematic diagram showing the sequence for forming a noise-suppressed image.

[0020] Figure 4 is a schematic diagram showing a training sequence for a convolutional neural U-Net network.

[0021] Figure 5 is a diagram showing an example of scaling a noise field according to image code values, according to an embodiment of the disclosure.

[0022] Figure 6 is a diagram showing an alternative example in which scaling of the noise field is determined by the standard deviation of a fully overlapping region of interest (ROI).

[0023] Figure 7 is a diagram showing an example of a step wedge test image processed by a network for noise suppression.

[0024] Figure 8 shows an example with varying levels of distortion that affect contrast.

[0025] Figure 9 is a plan view showing controls for adjusting noise reduction at a user interface.

[0026] Figure 10 is a diagram showing some of the factors that affect exposure reduction and can have an impact on noise suppression as described herein.

[0027] Figure 11A is a schematic diagram showing a system for general radiography or fluoroscopy.

[0028] Figure 11B is a schematic diagram showing a system for non-destructive testing and industrial radiography.

[0029] Figure 11C is a schematic diagram showing a cone beam computed tomography system.

[0030] Figure 11D is a logical flow diagram showing how noise suppression using a U-Net fits into the workflow of Figure 11C .

[0031] Figure 12 shows a comparison of an image with enhanced contrast due to scatter suppression without noise suppression and with noise suppression.

[0032] Figures 13A-13B shows a captured image and a captured image with a noise coverage mask that identifies regions of the captured image that correspond to pixels that are sub-threshold or quantum deficient. DETAILED DESCRIPTION

[0033] This application claims priority to U.S. Provisional Application No. 63 / 074,129, filed September 3, 2020, entitled “NOISE SUPPRESSION USING DEEP CONVOLUTIONAL NEURAL NETWORKS” in the names of William J. Schnert, Karin Toepfer, and Levon Vogelsang, which is hereby incorporated by reference in its entirety.

[0034] The following is a detailed description of the preferred embodiment with reference to the accompanying drawings, in which like reference characters designate like elements or components in each of the several figures.

[0035] In this document, the terms “a” or “an” are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of “at least one” or “one or more.” In this document, the term “or” is used to refer to a nonexclusive or, such that “A or B” includes “A but not B,” “B but not A,” and “A and B,” unless otherwise indicated. In this document, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.” Also, in the following claims, the terms “including” and “comprising” are open-ended, that is, a system, device, article, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim.

[0036] In the claims, the term “first,” “second,” and “third,” etc. can be used merely as labels, for clarity, and are not intended to impose numerical or prioritization requirements on their objects. The term “plurality” means at least two.

[0037] In the context of the present disclosure, the terms “viewer,” “operator,” and “user” are considered equivalent and refer to a viewing practitioner or other person who views and manipulates x-ray or other diagnostic images on a display monitor.

[0038] As used herein, the term “set” refers to a non-empty set, as the concept of a set or collection of members or elements is widely understood in elementary mathematics. Unless otherwise explicitly stated, the term “subset” is used herein to refer to a non-empty proper subset, that is, to a subset having one or more members of a larger set. For a set S, a subset can include the entire set S. However, a “proper subset” of a set S is strictly contained in the set S and excludes at least one member of the set S.

[0039] The phrase "radiographic image" refers to an image generated from non-visible radiant energy, typically ionizing radiation, and recorded on a radiation sensitive surface. Radiographic imaging is performed by systems of various modalities, including: general x-ray imaging equipment; volumetric imaging systems such as cone-beam computed tomography (CBCT) and computed tomography (CT) systems; fluoroscopy systems; and can include equipment used in medical / veterinary applications as well as industrial systems for non-destructive testing (NDT).

[0040] As noted in the Background section, machine learning techniques have been applied to the problem of noise suppression, but not specifically to the noise characteristics that are prevalent in digital radiography. In machine learning practice, neural networks are used to predict a noise field to be removed from a captured image.

[0041] According to embodiments of the present disclosure described in greater detail herein, as illustrated in Figure 1 As illustrated schematically in the process of FIG. 1, the predicted noise field N can be generated and scaled according to user preference to produce a controlled amount of noise in the final displayed image. As described in greater detail subsequently, the low dose image 10 is input to a convolutional network 12, such as a U-Net arrangement, to generate the noise field N. In addition, the subtracted noise field N can be scaled according to an estimated amount of spatial detail in any region of interest in the original image and / or a code value to optimize the appearance and diagnostic value of the noise suppressed image 16. Noise suppression can be part of the imaging process that includes: capturing a digital x-ray image for medical or industrial purposes; image enhancement and rendering for display; and optional additional algorithms such as tube and wire enhancement, automatic measurement of distance, angle, and detection of disease characteristics or material defects. In addition, the radiographic imaging equipment can recommend a reduction in x-ray dose relative to current practice based on the purpose of the examination specified by additional metadata from a medical or industrial information system and the scaling factor S of the subtracted noise field. Thus, the type of examination can be a factor in determining the amount of noise suppression to apply.

[0042] The present disclosure is applicable to radiographic imaging equipment for medical and industrial digital x-ray imaging. Included modalities are conventional two-dimensional radiography with both computed radiography (CR) cassettes and digital flat panel detectors, long length imaging, dual energy projection radiography, CR mammography. Also included are any form of frame captured digital radiography, such as computed tomography, including cone-beam computed tomography (CBCT), tomosynthesis, serial radiography, and fluoroscopy / real-time imaging.

[0043] As described herein, where multiple frames are sequentially acquired at a radiographic imaging device (e.g., such as in CBCT imaging), it can be advantageous to provide the series of images acquired as a set of input images for processing. The use of multiple frames can be beneficial to reveal the random nature of the inter-frame variation, thereby serving as a more accurate noise predictor than that provided by a single image.

[0044] Generally, deep neural networks have a large number of parameters (typically in the order of millions) and employ a large number of training samples. To develop an AI application for noise suppression, these aspects of the machine learning environment must be considered.

[0045] The training samples for noise suppression consist of one or more sets of pairs of low dose and high dose images. Given the potentially deleterious effects of ionizing radiation and the need for accurate registration of low dose image data and high dose image data, the development of the sample sets uses paired images. The approach begins with existing clinical images taken at dose levels representative of current best practice. Each image is then copied using image simulation techniques, and the copy is then adjusted to add noise corresponding to a lower dose. With the added noise simulated from, the use of the same image content helps to eliminate the possibility of small amounts of misregistration between the higher dose original image and its low dose counterpart.

[0046] By way of example, details of a simulation method for generating a low dose image from a high dose image were previously disclosed (U.S. Patent 7,480,365, entitled "Dose reduced digital medical image simulations", commonly assigned to Topfer and Ellinwood, Figures 1-3). The noise simulations are each associated with a noise model for a specific detector type, and are based on a noise model for a specific detector type, e.g., a flat panel detector with cesium iodide scintillator or a flat panel detector with gadolinium oxysulfide scintillator. The noise model is obtained by capturing flat field images at different exposure levels under specific beam conditions (tube voltage and filtration), e.g., RQA-5 beam. The code values of the images from the detector vary with exposure. The noise power spectrum of the flat field captures is computed, and a three-dimensional surface is fitted as a function of exposure and spatial frequency. Using this approach, the amplitude and shape of the noise power spectrum corresponding to each detector code value can be determined. If the detector code value varies with exposure in a linear fashion, a low dose image can be generated by dividing a high dose image by an exposure scaling factor. The corresponding noise power spectrum is the product of the original noise power spectrum and the inverse square of the exposure scaling factor. This is used as a reference for adding noise to the low dose image. The difference between the original noise power spectrum as a function of spatial frequency and exposure and the scaled power spectrum is added to the scaled high dose image via an image simulation using a noise table (i.e., standard deviation of noise as a function of code value) and a convolution filter.

[0047] For some applications, it is advantageous to use binned images, i.e., to combine multiple pixels in each direction to form a larger pixel. In many applications, 2x2 binning is practical, i.e., the combined pixel covers 4 times the area of the original pixel, and the pixel pitch for these applications is 2x the native pixel pitch of the sensor. This is often done in order to increase the frame rate for real-time imaging and other applications that take multiple images at a fast rate. These applications include, but are not limited to, fluoroscopy, cone beam CT (CBCT), and tomosynthesis. In these cases, the same training set can be used for training as for regular 2D radiography. Binning is carried out via image simulation, and the analysis of flat field images as described above is used to obtain the noise model for the binned system.

[0048] This description of the method of the present disclosure is intended to provide an overview of the subject matter disclosed herein according to one or more illustrative embodiments. The description is provided to introduce some of the illustrative aspects of the embodiments. The description is intended to identify certain features of the subject matter. The subject matter is not limited to implementing the described functionality in conjunction with the identified aspects. One or more of the features can be implemented individually or in any combination. The method can be described with reference to certain embodiments, some of which are illustrated in the drawings. It is noted that the drawings illustrate certain embodiments of the disclosure and therefore should not be considered to limit the scope of the disclosure, as the disclosure is encompassed by other equally effective embodiments. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of certain embodiments of the disclosure.

[0049] A favourable implementation of noise suppression using a deep convolutional neural network is the U-Net "fully" convolutional network by Ronneberger et al. (Olaf Ronneberger, Philipp Fischer, Thomas Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation", International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015, pp. 234-241) which is incorporated herein by reference. In the context of the present disclosure, the U-Net is used as a convenient example of a convolutional neural network acting as a processor trained to perform machine learning software. The operations and functions described herein for the U-Net are applicable to any similar convolutional neural network or processor employed. It is emphasized that there can be other types of processors employed for performing trained logical functions in addition to convolutional neural networks such as the U-Net as described herein.

[0050] As schematically represented in Figure 2 , the U-Net can be configured parametrically in terms of convolutional blocks B, features and levels of down resolution. The U-Net process flow follows a convolutional network model with a contracting path on the left side, with repeated convolutions arranged in multiple down-sampling levels L. Each step in down-sampling can double the number of feature channels. Multiple convolutional blocks B can be used, with a corresponding filter F at each level. An expanding path is shown on the right side of the network schematic. Each step in the expanding path can up-sample the feature maps and provide an up-convolution that halves the number of feature channels.

[0051] Max-pooling, convolution and concatenation of convolutional blocks are used to train the U-Net process to generate a noise field N. Max-pooling is understood as a well-known discretization process that operates to reduce the image dimensionality. Max-pooling can be used to speed up the downscaling of input data from one successive processing stage to the next. Max-pooling can be implemented using, for example, a max filter applied to non-overlapping sub-regions of the input image or corresponding feature map content. Training of the U-Net is carried out using a training set of pairs of low- and high-dose images, each pair having the same image content.

[0052] The high-dose image is scaled to the same average digital code value as the low-dose image. The network preferably predicts a residual between the high-dose image, the target and the generated low-dose image.

[0053] As described above, image simulation is used to generate low-dose images from high-dose images. The simulation is based on the physical properties of noise in x-ray imaging. For flat-panel digital x-ray detectors, the noise is primarily a combination of electronic noise and quantum noise. The noise varies with spatial frequency and x-ray exposure. Using image simulation to generate low-dose images from corresponding higher dose images allows the advantage that actual patient images can be used for training. Furthermore, the images of the training pairs can be precisely registered in space. The parameters of the U-Net or other convolutional neural network are optimized (i.e., trained) by minimizing a cost function that quantifies the difference between the low-dose image and the high-dose reference image. In its simplest form, the cost function can represent the mean absolute error or the mean squared error between the low-dose image processed by the U-Net and the high-dose reference image. However, a combination of two or more cost functions can be used, where one of the cost functions is perceptual in nature, i.e., it predicts whether a human observer can perceive the difference between the reference image and the predicted image.

[0054] While most digital flat-panel detectors produce raw images with a linear relationship between code value and exposure, the images can be transformed in other ways. For example, an exposure map (e.g., such as using a lookup table or a mathematical function) can be used to transform the images to different code values prior to training. Some alternative examples include using a logarithmic relationship between image code value and exposure or employing an Anscombe transformation that represents a proportionality between the square root of the image code value and the exposure with an offset to compensate for electronic noise (see, e.g., M., Foi, A. (2011)“Optimal inversion of the Anscombe transformation in low- count Poisson image denoising”, IEEE Transactions on Image Processing, 20(1), pp. 99-109). Furthermore, deep neural networks, such as those following the U-Net architecture, work best if the overall of the raw images is normalized to zero mean and unit standard deviation, or at least the input is in the range between 0 and 1, where 1 represents the maximum possible code value that the system can produce.

[0055] The cost function can be applied to the difference between the image predicted by the U-Net and the reference image. Alternatively, the cost function can be used to evaluate the reference noise field, i.e., to provide a measure of the difference between the exposure-scaled high-dose image and the low-dose image in the case of a noise field predicted from a low-dose image generated using the U-Net or other convolutional neural network.

[0056] Once the performance of the U-Net or other convolutional neural network is suitable, the training parameters can be saved. In image processing, the saved parameters are applied to medical or industrial image captures from digital x-ray detectors in order to predict and remove a noise field. This processing can be implemented on a computer's CPU (central processing unit) or a dedicated graphics processing unit (GPU), which can be part of the computer or can be an additional device connected to the computer. The execution time of the algorithm on the GPU is typically significantly faster than on the CPU.

[0057] According to embodiments of the present disclosure, the network predicts a noise field N to subtract from the original low dose noisy image to form a noise suppressed image. In Figure 3 The process is outlined in the schematic of FIG. 1. The noise field N most likely has a mean close to zero. An advantage of this implementation is that the noise field N can be scaled or otherwise adjusted based on additional parameters to achieve a pre-determined image processing goal. For example, the noise field N can be scaled according to image code values or according to an amount of detail on some local portion of the image, for example characterized by a variation within a pre-determined local region of interest.

[0058] Figure 4 is a schematic showing a training sequence for a convolutional neural U-Net network. A first set having a plurality of original standard dose images provides a starting point for generating a training set. Each original element or member of the first set is duplicated. Each duplicated image is then adjusted to provide a simulated low dose image having higher noise content. The original image and its adjusted duplicate then form an image pair for use within the training set. In this way, a training set is generated formed of pairs of original images and corresponding simulated low dose images, and this training set can be processed by the neural network to generate a predicted noise field N. In the training sequence, a comparison with the true noise field T is used to compute a cost factor, shown in FIG. 1 as L1 loss, and used to update the U-Net parameters until a sufficiently low difference is achieved. Once the difference is acceptably small or negligible, the convolutional neural network is considered trained and can be put into use. Figure 4 The cost factor, shown in FIG. 1 as L1 loss, and used to update the U-Net parameters until a sufficiently low difference is achieved. Once the difference is acceptably small or negligible, the convolutional neural network is considered trained and can be put into use.

[0059] Figure 5 The graph of FIG. 1 illustrates an example according to embodiments of the present disclosure for scaling the noise field according to image code values, where the scaling is by a scaling factor s cThis indicates that, in this case, noise suppression is phased out linearly without any original image code value higher than 16384. This example is for a substantially linear 16-bit system with 65536 possible code values ​​trained for a 4x exposure scaling factor. Due to the exposure scaling factor of 4, the maximum code value in the system's training set is 16384; in this embodiment, the U-Net is not trained for code values ​​higher than 16384. To avoid potential loss of detail at higher code values, noise reduction is phased out. However, in many cases, these higher code values ​​represent background open fields and are therefore irrelevant for medical diagnosis or for finding defects in industrial parts.

[0060] Figure 6 The figure illustrates an alternative example where the scaling of the noise field varies with the spatial frequency in the image content (as determined by the standard deviation of fully overlapping 8x8 pixel regions of interest (ROIs)). The ROI size (8) is for illustrative purposes only and can be chosen in different ways. The standard deviation of the pixels in the ROI in the input image is compared with the standard deviation of the ROI in the predicted noise field. In this case, the scaling factor s d The scaling of the characterized noise field depends on the ratio of the two standard deviations, such as Figure 6 As shown in the diagram. This type of scaling can be useful because the network achieves more noise reduction in flat parts of the image compared to regions containing high-frequency details.

[0061] Noise fields can also be based on Figure 5 and Figure 6 The examples shown combine scaling techniques. Similarly, the function shown for the scaled predicted noise field N can be nonlinear. Furthermore, other statistical measures or methods for filtering the noise field and the input image can be used to determine the scaling factor for the noise field.

[0062] Suitable for each image, the predicted noise field N can also be scaled based on the standard IEC (International Electrotechnical Commission) exposure index for the current radiographic image, the IEC target index for the current exam type, or the standard IEC deviation index for the current radiographic image. The IEC exposure index is defined in IEC 62494-1: Exposure Index for Digital X-ray Imaging Systems - Part 1 : Definition and Requirements for General Radiography and AAPM Report 116. For example, applying a noise field scaling factor that monotonically decreases with the IEC deviation index provides the ability to render images with a more uniform noise appearance regardless of the exposure used in the radiographic capture. Alternatively, the noise field scaling factor can be set to monotonically decrease with the IEC exposure index of the radiograph, or as a function of the difference between the IEC target index for the current exam type and the IEC exposure index.

[0063] A third factor for scaling the noise field is the user's preference for more or less noise reduction in the image. This subjective factor can be used to adjust the noise field generation parameters, and can be variable from site to site, or even from one user to another.

[0064] In training, the maximum achievable noise reduction is controlled by the exposure ratio between the high dose image and the low dose image. A preferred exposure ratio is in the range between 2 and 10. The network can typically achieve the largest fraction of noise reduction in flat ROIs of the image. Figure 7 is a graph showing an example for a step wedge test image processed by a network for noise suppression. The test image was captured at 4 different exposure levels to cover a wide range of available digital code values. The graph shows the robust mean and standard deviation for several 64 x 64 pixel ROIs for each step. The top curve (X symbols) represents the noise input image. The middle curve (+ symbols) uses the parameters for a U-Net trained with a 2x exposure ratio between the high dose image and the low dose image. The bottom curve (square symbols) uses the parameters for a U-Net trained with a 4x exposure ratio between the high dose image and the low dose image.

[0065] As Figure 8As shown in the middle, even with large neural networks, high levels of noise suppression can distort low-contrast details. Furthermore, high levels of noise reduction can make small systematic flaws in the image capture process more visible, thus distracting from the overall purpose of image capture. Thus, it can be advantageous to train the network for a maximum amount of enhanced noise reduction that does not result in significant distortion of details or enhancement of capture flaws. This training level can result in a small enhancement in the detectability of features at the noise threshold level compared to the original captured noisy image. In most cases, the network does not enhance features that are truly hidden by noise due to low dose. Above the noise threshold, the network can most likely be able to enhance the contrast-to-noise ratio of small and low-contrast objects. The amount of resulting enhancement depends on the amount of noise reduction. However, with or without enhancement, objects and features will be detectable by a trained radiologist. One of the advantages of the method described herein is that the noise level of the presented image can be customized, allowing for greater or lesser noise levels according to radiologist preference. This scaling factor s p is applied directly to the noise field predicted by the network.

[0066] The amount of noise in an image varies with the X-ray exposure amount. Low exposure amount images tend to look noisier than exposure amount pictures acquired at higher levels. Typically, imaging centers (e.g., hospitals) establish a target exposure level that is recommended as a balance between image quality and radiation burden on patients. The radiation exposure amount can be quantified by a metric such as the previously noted IEC exposure index. Deviation from the exposure target can be measured by the IEC deviation index expressed in decibels, where a negative value indicates an exposure level lower than the target exposure level. When the exposure deviates from the target, the amount of noise also changes, such that it is advantageous to modify the amount of noise reduction based on the IEC deviation index. This can be achieved, for example, by modulating the amount of noise reduction with a factor that is a non-increasing function of the IEC deviation index.

[0067] Controlling the amount of noise reduction based on the IEC deviation index is relative to the target IEC exposure index. However, this can not be the desired implementation. Control can also be exercised on an absolute basis by modulating the amount of noise reduction with a factor that is a non-increasing function of the IEC exposure index.

[0068] The concept of non-incremental noise involves the fundamental physics of X-ray quanta (the signal-to-noise ratio of X-ray signals increases with increasing exposure), leading to the general notion that noise reduction decreases with increasing exposure. However, the interaction between the X-ray and the detector is only one factor to consider, as the capture system includes other components, such as mechanical materials and electronic components including the system housing. Since multiple factors can influence noise generation, in some cases, it may be advantageous to modulate the noise using a function that is not purely non-incremental, so as to optimally implement this when all aspects of the capture process are incorporated.

[0069] Generally speaking, according to the noise-suppressed image I of this disclosure s Use scaling factor s c s d and s p The IEC deviation index d and IEC exposure index e, along with the scaling function F (which is preferably monotonic in each of the variables), are calculated based on the noise field N predicted by U-Net or other convolutional neural networks and the input image I0, as follows:

[0070] I s =I0-F(s) c ,s d ,s p ,d,e,I0,N)

[0071] You can, for example, use slider 90 to define preference factors on the settings screen. p This allows adjustment between zero and 100% or a numerical input. Figure 9 An example is shown below. Optionally, a preferred scaling factor can be defined at the facility for each radiologist. p A set of.

[0072] The preference factor can also be associated with the exposure control factor (ECF), which controls the threshold voltage of automatic exposure control (AEC) that shuts off in response to accumulated X-ray exposure. The software interface control for the ECF is, for example, a digital setting that varies from -5 to +5 in increments of one unit. A change of one unit corresponds to a 12.5% ​​increment change in the threshold voltage of the AEC from the default baseline, followed by a 12.5% ​​change in exposure. An ECF setting of 0 represents the default baseline exposure configured by the facility. The association between the preference factor and the ECF can be specified by a monotonically decreasing relationship, whereby the ECF decreases as the preference factor increases. Using this relationship, a comprehensive (across-the-board) dose reduction can be achieved by increasing the preference factor.

[0073] Figure 10Some of the factors that influence the exposure reduction and define the recommended dose reduction 130 that can affect noise suppression as described herein are shown. The medical or industrial imaging system can recommend additional dose reduction beyond current practice, which operates without noise suppression employing machine learning based methods, based on the scaling factor S and other metadata related to the particular exam. For example, as shown in Figure 10 The exam metadata 110, which includes information about the body part and the test required, can have an impact on noise suppression that exceeds other factors, as shown in

[0074] Band decomposition is one proven approach for controlling image quality by allowing improvement in specific aspects of the rendered image without compromising the image content. For example, U.S. Patent No. 7,848,560 to Wang et al., commonly assigned and incorporated herein by reference, entitled "Control of Multiple Frequency Bands for Digital Image," employs frequency decomposition to divide the image content into specific spectral bands to allow adjustment of gain and / or contrast within each band and reconstruction of the adjusted image for rendering.

[0075] Figures 11A-11D Various medical and industrial imaging systems for patients 1180 or industrial parts 1190 are shown that can integrate U-Net implementation based noise suppression, respectively. Figure 11AA system for general radiography or fluoroscopy is shown. The system has an X-ray tube and generator 1112 in an image acquisition system 1110; a manual switch 1120 connected to the generator and optionally to a workstation 1130 with a keyboard 1132 and / or a touch screen for user input; a digital display 1140; and a digital X-ray detector 1150 or computed radiography cassette. The workstation 1130 has software for managing work lists, technique selection, image acquisition; for performing image processing; and for displaying images. It can receive user input for selecting anatomic configuration and projection; examination purpose; setting preferences; and modifying default techniques. Preferably, the X-ray generator 1110 is connected to the workstation 1130 for direct communication of the selected technique. Alternatively, communication with the generator 1110 can be wireless, and in addition, the system can be configured for automatic exposure detection without requiring wired synchronization between the detector / workstation 1130 and the generator 1110. The digital flat panel detector 1150 can have a wired or wireless connection to the workstation. The workstation 1130 and display 1140 can be combined into a single unit, and it is also possible to run the software on a tablet that includes a high resolution display. Image processing at the workstation 1130 can include the noise suppression described herein.

[0076] For X-ray examinations, the software can automatically select anatomic configuration, projection, and technique based on the purpose of the requested examination. Alternatively, the user can manually select the body part and projection and override the default technique. Typically, AEC is used based on AEC settings for a predetermined system speed. The patient is positioned between the X-ray tube 1112 and the detector 1150, and the technician uses the manual switch 1120 to initiate the X-ray exposure. The detector 1150 sends the gain-corrected, offset-corrected, and defect-corrected image to the workstation 1130 via a wireless or Ethernet connection. As a first step, noise suppression is performed on the system’s GPU 1170 based on pre-trained weights 1160 for a U-Net, which can vary depending on the detector 1150 type. This can include any shifting / scaling operations or applying a lookup table or applying a transform using an equation before applying noise suppression. Then, image processing is performed, which can include but is not limited to histogram equalization, lookup tables (LUTs), and convolution filters. This renders the image in the best way for display. In addition, the system can contain special software that performs the following: performs measurement tasks (e.g., measuring distances and angles), provides statistical data about the distribution or size of disease features, and automatic detection of disease features, which are marked and shown on the display.

[0077] Noise suppression applied using the machine learning methods described herein can be used prior to multi-frequency decomposition or after multi-frequency decomposition in order to improve the estimation of high frequency image content while not amplifying noise content, and to provide improved sharpening of high frequency image content. Noise suppression, contrast enhancement, and edge sharpening can be applied in varying amounts to different frequency band content. For example, higher levels of noise suppression can be more appropriate for lower frequency bands; it can be appropriate to scale noise suppression progressively over some higher frequency portions of the image in order to render areas with significant detail more clearly.

[0078] Noise suppression can have the perceptual effect of softening the image so that it appears less sharp to a human observer. Thus, the ability to control the amount of noise removed from the image or to add back a certain amount of noise to the image can provide more satisfactory results. If noise is added back to the image, the shape of the spectrum used for noise adjustment can be optimized for human perception. Knowledge gained about the modulation transfer function (MTF) of the radiographic imaging device after noise suppression can be used for image edge restoration. Optionally, the amount of noise suppression can also be adjusted to simulate a given exposure speed. For example, the image can be adjusted to appear as if it was acquired at a speed of 200.

[0079] Noise suppression can be applied to images with other processing features including tube and line enhancement, bone suppression, or other image adjustments. Noise suppression based on machine learning techniques can be applied in combination with other image restoration tasks based on machine learning techniques such as super-resolution over the entire image or portions of the image.

[0080] Region of Interest

[0081] Embodiments of the present disclosure enable a user to define a region of interest of a captured image and to define an amount of noise suppression specific to pixels within the defined region. Other portions of the image outside the region of interest can not be processed, or can have scaled noise suppression applied. The operator can adjust the amount of noise suppression applied within the region of interest.

[0082] Scatter Correction

[0083] According to embodiments of the present disclosure, an optional scatter image can be generated and combined, such as using subtraction, with the acquired image data in order to reduce the scatter impact. Scatter correction can be provided at workstation 1130 or other processor as a correction measure against noise and in order to help improve image contrast. Image processing strategies designed to remove scatter from the exposure field generally predict a smooth scatter field to be subtracted from the image, thereby increasing the overall contrast. However, since the estimated scatter field is smooth, the noise associated with the scattered radiation can remain in the image. Since the process increases the overall contrast, the presence of noise can be distracting. Applying deep learning based noise suppression prior to scatter compensation can significantly reduce the noise presence after scatter compensation.

[0084] Figure 12 An image 1200 with scatter suppression only, without machine learning based noise suppression is shown. Image 1210 shows the improved result achievable by the machine learning based noise suppression following the prior scatter suppression.

[0085] Scatter correction can also be extended to CBCT projections, where scatter estimation is useful for improving contrast in the reconstructed volume as well as for improved estimation of Hounsfield units (see Figure 11D Step S430 in

[0086] A similar workflow applies to fluoroscopy or real-time imaging as well as serial radiography, where the user also selects a desired frame rate (typically between 1 frame / second and 30 frames / second, depending on the purpose of the acquisition). In serial radiography, a smaller set of images is captured compared to fluoroscopy, and typically, the frame rate is lower. In both scenarios, multiple frames are captured. In serial radiography, the frames do not need to be delivered to the workstation in real-time, and if delivered to the workstation later, the processing can be done offline. The video frames are reviewed after the acquisition rather than during the acquisition, thereby reducing the computational power required for the processing. However, unlike serial radiography, in fluoroscopy, each frame is transmitted to the workstation in real-time, and the noise reduction is received on GPU 1170. Noise suppression in fluoroscopy is valuable because the individual images are acquired at very low dose, and the acquisition is typically quantum limited. Typically, multiple pixels are binned to create a single larger pixel. This helps reduce noise and enables a higher frame rate. For optimal noise reduction, fluoroscopy and general radiography acquisition on the same detector model can require different model weights 1160 for the U-Net.

[0087] Figure 11BThis diagram illustrates a system for non-destructive testing of industrial parts 1190 and other objects of interest (e.g., fossils, artworks, and forensic objects). The system operates in a manner similar to medical systems, except that it can work with both X-ray and gamma sources. X-ray sources can have properties different from medical X-ray sources, for example, to achieve higher kVp to penetrate thick metal sections, or to achieve very small focal lengths to allow for visualization and magnification of very fine details. The diversity of sources used in NDT makes it difficult to find a common noise model for all cases. It may be necessary to train separately for gamma and X-ray sources and store models for these two different cases. Objects in NDT are inanimate, and dose is not a critical factor. Commonly, multiple registered frames of the same object are obtained and averaged to increase the signal-to-noise ratio. Reduced exposure in NDT results in time savings and a more efficient workflow. In NDT, a preferred implementation of noise suppression is to run the algorithm on individual images before multi-frame averaging.

[0088] Figure 11C The diagram illustrates a CBCT (cone-beam computed tomography) system 100, in which a flat panel detector 24 rotates synchronously with an X-ray source 22 around a subject or object 20, thereby covering a total rotation angle between 180 and 360 degrees and capturing multiple 2D projected image frames, typically exceeding 100 projected images. Furthermore, compartmentalization is used at high speeds to acquire images, creating larger pixels, thus minimizing the total acquisition time and reducing the impact of patient movement. Low doses for each projected image frame are necessary to keep the total dose as low as possible for these multi-frame captures.

[0089] Figure 11D This demonstrates how noise suppression using U-Net or other convolutional neural networks can fit into... Figure 11C The following is a logic flowchart of the workflow for CBCT imaging. In the acquisition step S400, the raw image is obtained from the detector. As described in detail herein, the noise suppression step S410 operates on the individual raw images in real time on the GPU before performing any other image processing steps, which typically also requires GPU processing for fast display times. The normalization step S420 is typically performed in conjunction with an optional logarithmic transformation. The scattering correction step S430 can provide scattering compensation. The iterative reconstruction step S440 performs volumetric reconstruction based on the set of acquired images. The optional enhancement step S450 provides variable levels of image enhancement for volumetric content. Special analysis and measurement processing can then be applied in the optional analysis and measurement step S460.

[0090] According to embodiments, a display application can be provided for indicating predicted noise on different parts of an image. For example, a semi-transparent color overlay can be displayed over the image content to convey the amount of predicted noise.

[0091] In addition to reducing dose and suppressing noise, users of the aforementioned imaging systems desire to quickly assess image quality for maintaining an efficient workflow. Users of the system should be confident that the acquired image has the necessary anatomical information using the lowest useful X-ray dose. When acquiring an image, a preview image is typically utilized to enable the user to assess the technique and positioning of the acquisition before the full resolution image is processed and ready for display. The preview image typically has reduced resolution and is not suitable for assessing subtle details and noise in the image. To facilitate acceptance of studies based on preview images that have not been noise suppressed, a display application is provided for indicating which part(s) of the image data (preview or full resolution) are at sub-threshold signal levels for noise suppression. Such regions can be highlighted in a color overlay that can be displayed over the acquired image to convey to the user the quantum deficient pixel data. While noise can be suppressed in these regions, it can be possible that important anatomical information was not adequately captured in the image acquisition. When critical anatomical structures related to the purpose of the examination are emphasized in the overlay, the user can quickly assess and / or reject the image, increase the exposure, and reacquire the image. Figure 13A An image 1300 of a forearm acquired before the overlay is superimposed is shown, and Figure 13B The acquired image with the superimposed overlay 1310 is shown, where the pixels 1311 shown in red indicate sub-threshold or quantum deficient pixel data. The sub-threshold pixels are in areas of the image that are not relevant to the diagnosis of the forearm, so the user can be confident that the exposure was sufficient.

[0092] CBCT and general radiographic acquisitions on the same detector model can require different weights for the U-Net or other convolutional neural network for optimal noise reduction. As noted in Figure 11D As noted in the sequence, other image processing steps for CBCT in the case of iterative reconstruction and enhancement for display generally include log conversion and normalization, scatter reduction. This image processing can be followed by additional software for measuring distances and angles and / or for detecting disease characteristics; these processes can also be aided by optional hardware. Similar to the Figure 11D A similar workflow can apply to digital tomosynthesis, where typically less than 100 frames are captured, with the X-ray source moving in a limited angular arc relative to the patient and detector.

[0093] As those skilled in the art will appreciate, aspects of the present disclosure can be embodied as a system, method, or computer program product. Accordingly, aspects of the present disclosure can take the form of an implementation in hardware and software (including firmware, resident software, micro-code, etc.) or combinations thereof. The disclosure can also be implemented in software and / or firmware, which can be stored on or otherwise provided to a computer-readable medium, which can be a non-transitory computer-readable medium, to cause a computer or other processing system to operate as described herein.

[0094] This written description uses examples to disclose the application (including the best mode), and also to enable any person skilled in the art to practice the application, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the application is defined by the claims, and can include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent

[0095] A computer program product that executes the processes described herein can include one or more storage media, for example, magnetic storage media such as magnetic disks (such as floppy disks or hard drives) or magnetic tape; optical storage media such as optical disks, optical tape, or machine-readable bar code; solid-state electronic storage devices such as random access memory (RAM), or read-only memory (ROM); or any other physical device or media employed to store computer program instructions comprising a computer program for controlling one or more computers to practice the methods according to the present disclosure.

[0096] The present application has been described in detail and with particular reference to preferred or presently preferred embodiments, but it will be understood that variations and modifications can be effected within the spirit and scope of the application. The presently disclosed embodiments are, therefore, to be considered in all respects as illustrative and not restrictive. The scope of the application is indicated by any appended claims, and all changes that come within the meaning and range of equivalents are intended to be embraced therein.

Claims

1. A computer-implemented method of generating a noise-suppressed radiographic image, the method comprising the steps of: training a machine learning network to generate a noise field image from a current radiographic image by: accessing a plurality of previously-acquired standard-exposure radiographic images; copying each of the previously-acquired standard-exposure radiographic images; associating each simulated low-exposure radiographic image with its corresponding previously acquired standard-exposure radiographic image to form a plurality of learning pairs of radiographic images; adjusting each of the copied images with simulated noise content to form a plurality of simulated low-exposure images, and training the machine learning network to generate a noise field image using the plurality of learning pairs of radiographic images; capturing a current radiographic image of a subject and using the trained machine learning network to generate a corresponding noise field image from the current radiographic image of the subject; suppressing noise in the current radiographic image of the subject, including applying a scaling factor to at least a portion of the corresponding noise field image and combining the scaled noise field image with the current radiographic image of the subject; and displaying, storing or transmitting the noise-suppressed radiographic image of the subject.

3. The computer-implemented method of claim 2, wherein, 2. The computer-implemented method of claim 1, further comprising determining a magnitude of the scaling factor based on an exposure level in a same portion of the current radiographic image of the subject. the scaling factor comprises a substantially linear relationship with respect to the exposure level.

4. The computer-implemented method of claim 1, further comprising determining a magnitude of the scaling factor as a function of a spatial frequency of the current radiographic image of the subject.

5. The computer-implemented method of claim 1, further comprising determining a magnitude of the scaling factor in response to a variable user input.

7. The computer-implemented method of claim 1, further comprising:

6. The computer-implemented method of claim 5, further comprising simultaneously displaying a current exemplary noise-suppressed radiographic image of the subject while a user is changing the variable user input. capturing a plurality of current radiographic images of the subject; and applying an iterative reconstruction method to the plurality of current radiographic images of the subject to generate a volumetric image of the subject.

8. The computer-implemented method of claim 1, further comprising determining a magnitude of the scaling factor based on an IEC index value of the current radiographic image of the subject.

9. The computer-implemented method of claim 8, further comprising determining the magnitude of the scaling factor based on an IEC target index for a type of examination used to capture the current radiographic image of the subject.

11. The computer-implemented method of claim 1, wherein, 10. The computer-implemented method of claim 1, further comprising deriving the scaling factor from an IEC bias index of the current radiographic image of the subject. the step of applying the scaling factor comprises taking an inverse of at least a portion of the generated noise field image.

12. The computer-implemented method of claim 1, further comprising determining a magnitude of the scaling factor based on a signal-to-noise ratio of a same portion of the current radiographic image of the subject.

13. The computer-implemented method of claim 1, wherein, The step of combining includes subtracting the scaled noise field image from the current radiographic image of the subject.

14. The computer-implemented method of claim 1, wherein, The step of capturing includes capturing a fluoroscopic image of the subject.

15. The computer-implemented method of claim 1, further comprising: Identifying image data in the current radiographic image of the subject that is sub-threshold; Forming an overlay image corresponding to the identified sub-threshold image data; And displaying the overlay image in combination with the current radiographic image of the subject.

12. The computer-implemented method of claim 1, further comprising determining a magnitude of the scaling factor based on a signal-to-noise ratio of a same portion of the current radiographic image of the subject. The step of combining includes subtracting the scaled noise field image from the current radiographic image of the subject. The step of capturing includes capturing a fluoroscopic image of the subject. Identifying image data in the current radiographic image of the subject that is sub-threshold; Forming an overlay image corresponding to the identified sub-threshold image data; And displaying the overlay image in combination with the current radiographic image of the subject.

Images