Noise label based machine learning for anomaly assessment in medical imaging

Abstract

The present invention relates to noise label based machine learning for anomaly assessment in medical imaging. For machine learning and application of machine learning models for anomaly assessment in medical imaging, the machine learning uses regularization of loss, such as regularization used to train anomaly classification in chest radiographs. The regularization can be noise and / or correlation regularization, which targets noisy ground truth labels of the training data. As a result of using noise and / or correlation regularization in the training, the resulting machine learning model can better classify anomalies in medical images.

Description

[0001] Related applications

[0002] This patent document claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 054,823, filed July 22, 2020, pursuant to Section 119(e) of Chapter 35 of the United States Code, which is incorporated herein by reference. Background Technology

[0003] This embodiment relates to machine learning. Machine learning algorithms have shown great promise in computer-aided classification of medical images. For example, machine learning has been used to develop automated chest X-ray systems. Chest X-ray evaluation is used for the detection of chest diseases and abnormalities. However, developing these systems is challenging because there is significant inter-evaluator variability in the interpretation of chest X-rays. High error rates in annotation methods, such as those based on natural language processing (NLP), and the inherent ambiguity of pathological appearances lead to incorrect dataset labels. In the context of machine learning-based systems, these factors result in overconfident systems that generalize poorly to unknown data. Deep learning methods, which are known to perform well in other domains, may still be overconfident.

[0004] To correct this, a higher-quality test set re-annotated by radiologists can be used for training. Predictive uncertainty can be estimated as an orthogonal measure of the probability of predicted anomalies using subjective logic. However, label noise can still cause machine learning models to perform poorly. Summary of the Invention

[0005] For introduction, the preferred embodiments described below include methods, systems, instructions, and computer-readable media for machine learning in anomaly assessment in medical imaging and the application of machine learning models. Machine learning uses regularization of the loss, such as the regularization used to train anomaly classification on chest X-rays. The regularization can be noise and / or correlation regularization, applied to the noisy ground truth labels of the training data. Due to the use of noise and / or correlation regularization during training, the resulting machine learning model can better classify anomalies in medical images.

[0006] In a first aspect, a method for machine learning of anomaly assessment in medical imaging is provided. Training data is obtained, comprising medical images and ground truth labels for the medical images. The ground truth labels indicate any anomalies represented by the medical images. The machine trains a model based on the training data. The machine training uses a loss function incorporating regularization. The regularization is noise regularization and / or correlation regularization. The model resulting from the machine training is stored in memory.

[0007] In one embodiment, machine training includes machine training using a loss function that compares the model's classification of anomaly outputs with ground truth labels. Other loss functions may be used. In another embodiment, machine training includes machine training using ground truth labels and a loss function, where ground truth labels are binary labels indicating the presence or absence of an anomaly, and the loss function is weighted as a function of the number of positive and negative instances of anomalies in the medical images of the training data. Other labels, such as ranks or scores, may be used.

[0008] In one embodiment, the regularization of the loss function is noise regularization. For example, the noise level of the ground truth labels is measured. Machine training includes machine training utilizing noise regularization as a function of the noise level. In one approach, the noise level is represented by specificity and sensitivity to anomalous ground truth labels. Noise regularization includes a first weight as a function of specificity and a second weight as a function of sensitivity, or any other metric describing the label-to-noise ratio. Noise regularization can be any function, such as the inverse binary cross-entropy function.

[0009] In another embodiment, ground truth labels indicate at least first and second types of anomalies. The regularization of the loss function is correlation regularization. Correlation regularization correlates the ground truth labels for the first type of anomalies with the ground truth labels for the second type of anomalies. In one approach, correlation regularization is covariance. For example, at least four types of anomalies are provided. Correlation regularization is the sum of the covariances among all at least four types of anomalies.

[0010] In other embodiments, both noise regularization and correlation regularization are used for training.

[0011] Various types of medical images and / or abnormalities can be used. For example, the medical images used in the training data are chest X-rays, and abnormalities include effusion, cardiomegaly, consolidation, atelectasis, and masses.

[0012] In the application, the machine-trained model is applied to patient images. The application outputs a classification of the patient images with or without anomalies.

[0013] In a second aspect, a system for anomaly detection in medical imaging is provided. The medical imaging system is configured to generate images of a patient. A processor is configured to apply a machine learning model to the patient images. The machine learning model is trained using noise and / or correlation regularization to detect anomalies in the images. A display is configured to classify the patient as having or not having an anomaly based on the detections from the application.

[0014] In one embodiment, the machine learning model is trained using noise regularization. Noise regularization accounts for noise in the ground truth labels used during machine training. In another embodiment, the machine learning model is trained using correlation regularization, which accounts for erroneous representations between different types of anomalies.

[0015] Although applicable to different medical imaging environments, noise or correlation regularization can be used to identify ground truth labels for anomalies in X-ray images from X-ray imaging systems.

[0016] In a third aspect, a system for machine training for anomaly classification is provided. A memory is configured to store training data and a machine learning classifier, the training data including anatomical images and ground truth classifications for the images. A processor is configured to perform machine training based on the training data. Machine training includes calculating a loss using noise and / or correlation regularization. The processor is configured to perform machine training using the loss, resulting in a machine learning classifier.

[0017] In one embodiment, the processor is configured to perform machine training using noise regularization. In another embodiment, the processor is configured to perform machine training using correlation regularization.

[0018] This invention is defined by the following claims, and nothing in this section should be construed as limiting those claims. Other aspects and advantages of the invention are discussed below in conjunction with preferred embodiments and may be claimed later, independently or in combination. Attached Figure Description

[0019] The components and figures are not necessarily drawn to scale; rather, the focus is on illustrating the principles of the invention. Furthermore, in the figures, the same reference numerals indicate corresponding parts in different views.

[0020] Figure 1 This is an example of a method for applying machine training and the resulting machine learning model that utilizes noise and / or correlation regularization;

[0021] Figure 2 The figure illustrates an example architecture for machine training that leverages regularization;

[0022] Figure 3 The correlation of the anomalies is shown; and

[0023] Figure 4 This is a block diagram of an embodiment of a system for training and applying a trained model using noise and / or correlation regularization. Detailed Implementation

[0024] Machine learning has improved the generalization of anomaly classification based on label error rate assessments from chest X-rays or other medical imaging. Different regularization techniques can handle label noise, such as dropout regularization or dimensionality-driven learning strategies. Regularization can be applied to many areas of medical imaging, such as image reconstruction or image segmentation.

[0025] To provide robust classification from noisy labels used for anomaly assessment in medical imaging, regularization is applied to the classification loss. To improve the generalization accuracy of the machine learning system, regularization is also applied to the classification loss. Two example regularization components are noise regularization based on prior label noise probabilities and correlation regularization based on the correlation between anomalies. Both noise and correlation regularization lead to improvements in the generalization performance of anomaly detection and classification.

[0026] Figure 1 An embodiment of a method for machine learning anomaly evaluation in medical imaging is illustrated. In chest X-rays, regularizations such as noise, correlation, or dropout are used in training. For general medical imaging (e.g., X-ray, computed tomography (CT), magnetic resonance (MR), ultrasound, single-photon emission computed tomography (SPECT), and / or positron emission tomography (PET)), regularization is at least noise and / or correlation regularization. Noise and / or correlation regularization used for medical image classification reduces the influence of noisy labels in the training data.

[0027] This method is by Figure 4 This method can be implemented by a system or another system. For example, the method can be implemented by a system 40 for training, which includes a memory 41 for storing training data and learning a model, and a processor 43 for performing machine training using a regularizer. Different devices can be used.

[0028] Additional, different, or fewer actions may be provided. For example, application action 18 and / or storage action 16 are not provided. As another example, neither action 13 nor 14 is provided, where regularization is applied to a chest X-ray. In yet another example, only one of action 13 or action 14 is provided. As yet another example, actions for designing the architecture of a model (e.g., the layer structure of a neural network) are included.

[0029] Actions are performed in the order shown (e.g., top to bottom or by number) or another order. For example, action 16 can be applied after action 18. As another example, actions 13 and 14 can be performed in any order or simultaneously (e.g., where noise and correlation regularization terms are both included in the loss function).

[0030] In Action 10, training data is obtained. Data is obtained through searching, data mining, loading from memory, identification, transmission over computer networks, and / or collection. Designers (e.g., computer scientists) obtain training data, such as data on specific types of medical imaging, organs of interest, diseases of interest, and / or one or more anomalies of interest. Computers, research facilities, and / or databases can be used to obtain the data.

[0031] Training data includes medical images. Dozens, hundreds, or thousands of sample medical images are obtained. For example, X-ray images from many different patients can be obtained. Actual medical images from patients can be used. Alternatively, simulations of medical imaging are used to generate medical images. In yet another embodiment, images of phantoms are used. The medical images in the training data can come from multiple sources, such as actual images of patients, simulations, and phantom imaging. Any subset of data from any domain (e.g., ultrasound, MR, CT, PET, or SPECT) can be used. Chest radiographs are used as an example here because chest radiographs tend to have noisy ground truth and / or a large number of abnormality types represented in the image.

[0032] For the sample images, other information may be included. For example, the patient's clinical and / or laboratory results associated with the image may be included. Age, weight, smoking history, blood tests, and / or other information may be provided as samples along with the medical images to train a classifier to detect anomalies from the input images and other types of information. In other embodiments, only medical images are used in the samples of the training data.

[0033] Training data includes ground truth labels for each sample. Ground truth labels are mined from patient records, indicated by metrics (e.g., the application of another classifier), and / or provided by expert review of the samples. Ground truth labels indicate the presence or absence of an abnormality, its location, and / or its degree or level (e.g., size or score). Ground truth labels are provided for each type of abnormality for each sample. For example, a chest X-ray includes a positive label for one type of abnormality and a negative label for another. Ground truth labels indicate any abnormality represented by each sample (e.g., a medical image).

[0034] Ground truth labels can be noisy. Some of the labels may be incorrect. Since machine learning relies on the accuracy of ground truth labels to learn whether an image contains an anomaly, noisy labels introduce errors into the trained classifier. Errors may be due to incorrect labeling in a binary sense (e.g., whether anomaly X is represented) and / or in an incorrect recognition sense (e.g., a mass is labeled as fluid accumulation).

[0035] Samples and labels can be used for any number or type of abnormality. For example, labels can be used for a single type of abnormality (e.g., cancerous lesions). Each sample is labeled with a ground truth value for whether an abnormality is present in the sample. As another example, labels can be used for two or more, three or more, or four or more types of abnormalities. In the chest X-ray example, the types of abnormalities include effusion, cardiomegaly, consolidation, atelectasis, and mass. Additional, different, or fewer types of abnormalities can be classified or labeled. Each sample (e.g., medical images in the training set) is labeled with one or more labels for the representation and / or binary or non-binary indication of the location of the corresponding one or more types of abnormality.

[0036] In Action 12, the machine performs machine training. The processor or computer uses the training data to perform machine learning. The model is defined and trained by establishing values ​​for learnable parameters based on the training data. Samples are the input, and the resulting output is compared to ground truth labels. Through optimization (e.g., Adam), the training data is used to establish values ​​for learnable parameters of a defined model that lead to accurate output.

[0037] Any training method can be used, such as deep learning for neural networks. Support vector machines, regression, or other machine learning and corresponding models can be used. In one embodiment, deep learning is used. Using piecewise differentiable functions or other deep learning functions, the machine trains the network to output a classification (e.g., whether an abnormality was detected) in response to input samples (e.g., medical images). The machine trains the network using regression.

[0038] The neural network is either a fully connected network (FCN) or a convolutional neural network. Other models can also be used.

[0039] The defined model is trained to estimate using a loss function. Any loss function can be used, such as cross-entropy, L2 (e.g., least squares error), L1 distance, or other losses, to obtain optimal values ​​for the network parameters. The difference between the ground truth labels for the training images and the model's predictions is minimized based on the loss or measure of the difference in the loss function. Through optimization, the values ​​of the learnable parameters are tuned to minimize the loss.

[0040] The loss function includes regularization. One or more terms specifying the loss are added. The regularizer can be a term added to the loss. Alternatively, the regularizer is a weight or adaptive change in the loss calculation that takes into account the noise label. In one embodiment, the regularization is noise regularization. In another embodiment, the regularization is correlation regularization. In yet another embodiment, both noise regularization and correlation regularization are used. Additionally or alternatively, other regularizations, such as dropout regularization and / or dimension-driven learning, can be used.

[0041] Figure 2 An example layout or architecture for machine training as a pipeline is shown. Image 20 is used as input to deep learning architecture 22 (i.e., a model with relevant learnable parameters). The model's output and ground truth are used to determine the classification loss 24 during training. The classification loss 24 includes noise regularization 25 and / or correlation regularization 26. Once trained, the model outputs anomaly classifications 28 without loss and / or regularization. The previously used loss and regularization provide values ​​for the model in architecture 22 to provide accurate anomaly classifications 28.

[0042] The ground truth labels in the training data are binary labels used to indicate whether anomalies exist in the samples; therefore, the loss is based on binary predictions for many or all samples. In other embodiments, the labels are continuous values ​​or have more than two discrete values; therefore, the loss is based on accuracy along a continuous region or across a discrete set.

[0043] To determine the loss, the labels are a set [c (1) c (2) ... c (d) ]ϵ{0, 1} (representing non-existence or presence of an anomaly, respectively) and compared with the network output [p (1) p (2) ... p (d) The loss is measured based on comparisons within the range [0, 1]. The loss function provides the comparisons.

[0044] In one embodiment that addresses imbalance in the training data, the loss function is weighted as a function of the number of positive and negative instances of anomalies in the medical images of the training data. Due to the imbalance problem (e.g., more negative samples than positive samples), an additional weight constant is provided for each anomaly indexed by n. and . and , where P (n) and N (n) These indicate the number of positive and negative instances for the entire training dataset, respectively. Integrating the weights into the example cross-entropy loss function provides:

[0045] (1)

[0046] The loss is calculated based on the sum across all images or samples. The function is indexed by i, where F represents the total number of images in the set, and D represents the total number of anomaly types.

[0047] For action 13, the loss function includes noise regularization. Machine training is performed using regularization of the loss function, including noise regularization. The noise level of the ground truth labels is measured and used for regularization. For example, the specificity and / or sensitivity of the ground truth labels in the training set are used for regularization.

[0048] In one embodiment, an expert reading procedure is defined. A radiology expert reads the samples and blindly relabels them. One or more experts classify each sample (i.e., identify the ground truth) without access to ground truth labels and / or classifications by other experts. The original dataset labels are not provided during the expert reading process to avoid biased decisions regarding the original labels. Multiple experts may perform the reading, providing multiple instances of the ground truth labels for each sample. For all cases where consensus on all labels cannot be reached through independent readings, public discussion or majority voting can be conducted to establish consensus labels. Assuming the redefined labels are correct, prior probabilities are calculated using both the original and redefined labels. Table 1 shows the sensitivity of the original ground truth labels to expert relabeling for five selected abnormality types on chest radiographs. sens and specificity spec .

[0049] Table 1:

[0050]

[0051] A low score indicates stronger label noise.

[0052] To incorporate noise regularization into the loss function, a term needs to be added to the loss function. Any regularization term can be used, such as the inverse binary cross-entropy function. The added term is the noise regularization, which is a function of the noise level. Any function can be used. In one embodiment, two weights are added, one of which is a function of specificity and the other is a function of sensitivity. In other embodiments, only sensitivity, only specificity, or another measure of noise level is used.

[0053] In one embodiment, noise regularization is added as an inverse binary cross-entropy function to the loss function in Equation 1. The resulting regularized loss function is given as:

[0054]

[0055] Where f P and f N These are the individual regularization weights for positive and negative examples. Any function using the noise level can be used. In one embodiment, and Additional parameter λ noiseThis is another weight used to define the overall effect of the regularization term. Any value can be used for this additional parameter, such as 0.1.

[0056] Noise can be incorporated into regularization and / or loss functions in various ways, such as weights, ratios, subtraction, or using different regularization functions (e.g., L2). Instead of weights, noise levels can be incorporated through addition, subtraction, or other functions.

[0057] For action 14, correlation regularization is used in the loss function during machine training. Correlation regularization uses the correlation between the ground truth labels for type 1 anomalies and the ground truth labels for type 2 anomalies. Anomalies with stronger correlations are more likely to be misclassified in the ground truth labels, thus introducing label noise.

[0058] Figure 3 Examples from chest X-rays are shown. The strength of the correlation between each abnormality is visualized. The correlation levels between the five different types of abnormalities (effusion, cardiomegaly, consolidation, atelectasis, and mass) are graphically illustrated. Darker colors correspond to the correlation levels, with black indicating perfect correlation (e.g., a correlation coefficient of 1.0) and white indicating no correlation (e.g., a correlation coefficient of 0.0). In these examples, the correlation is 1.0 for the abnormality itself, and the correlation between different abnormalities ranges from 0.0 to 0.5.

[0059] Correlation regularization is based on the set of classification labels c for anomaly n. (n) With the set of classification labels c for anomaly r (r) The correlation strength is denoted by r, where r ∈ {1…D}\{n}. D indicates the number of anomalies. Any correlation coefficient can be used as a measure of correlation. Figure 3 The Pearson correlation coefficient is used. In other embodiments, the correlation coefficient is the covariance.

[0060] To incorporate relevance regularization into the loss function, a term needs to be added to the loss function. Any regularization term can be used, such as the cross-entropy function. The added term is the relevance regularization, which is a function of the level of correlation between different types of anomalies to be classified. Any function can be used. In one embodiment, it is added to the sum of cross-entropy weighted by the relevance coefficients across the different types of anomalies. For example, sums are used across two, three, four, or more types of anomalies, such as... Figure 3 The sum of five exceptions in the example.

[0061] In one embodiment, the original loss function of Equation 1, which takes into account correlation information across all anomalies, is expressed as:

[0062]

[0063] in λc orr It is a convolution with weights (e.g., set to 1.0) and elements (n, r). (n,r) Measure the covariance between the label at index n and the label at index r. Depending on the covariance matrix, all anomalous labels may influence another given anomalous label.

[0064] In other embodiments, the loss function includes two or more additional terms. For example, both noise and correlation regularization terms may be included. Relative weights may be used to control the relative contribution of the regularizers to the loss. The model is trained using any number of regularizers, such as noise and correlation regularization. Additional regularization components may be added.

[0065] Refer again Figure 1 In action 16, the machine (e.g., a processor or computer) stores the model trained by the machine in memory. Models and / or copies for use by different machines can be transferred via computer networks. Machine learning classifiers are stored. For example, after creation, a machine learning network includes one or more layers with values ​​for various learnable parameters, such as convolutional kernels, downsampling weights, and / or connections. The trained parameters and / or network values ​​are stored. The machine learning network is stored in memory, such as the machine's memory or a database with examples. The machine learning network can be transferred to different memories. The machine learning network can be copied for application in other devices or machines, such as the processor of an X-ray scanner. The memory of an X-ray or other scanner can store copies of the machine learning network for application to a specific patient.

[0066] In Action 18, a machine learning model is applied. The processor or computer applies the model to patient images, with or without clinical data about the patient. The same or different machines used to train the model are then applied.

[0067] For application, patient images, such as X-ray scans from patients, are used as input to a machine learning model, with or without other data (e.g., clinical data). In response, the machine learning model outputs a classification of the patient image. The classification can be the detection of one or more abnormality types. A usable classification may include no abnormalities. The patient image is classified as including or excluding one or more different types of abnormalities. The patient may or may not have one or more different types of abnormalities. A machine learning classifier, trained to classify based on noise and / or correlation regularization, classifies the input image and / or data. The patient is imaged, and the resulting image is classified using the machine learning model.

[0068] Images demonstrating the application results can be generated. Images can be color-coded, annotated, or labeled to indicate classification. The image can be either the classification itself or a classification with anatomical representation (e.g., a chest X-ray with annotations showing the classification for the image or classification by image region). Classifications can be added to the patient record.

[0069] Noise and correlation regularization improve the performance of machine learning models in classification. For models with the noise levels shown in Table 1... Figure 3 Five abnormal chest X-rays with correlation were used to train a neural network using Equation 1 as the baseline loss, Equation 2 for noise regularization, and Equation 3 for correlation regularization. Table 2 shows the area under the curve score of the machine learning model obtained for the results.

[0070] Table 2:

[0071] fluid accumulation Cardiac hypertrophy Real change Atelectasis lump <![CDATA[Baseline L Abn > 0.923 0.926 0.812 0.821 0.804 <![CDATA[Prior label noise L Noise > 0.940 0.927 0.836 0.845 0.829 <![CDATA[Label Relevance L corr > 0.915 0.940 0.831 0.831 0.838

[0072] The performance and versatility of the detection and classification system are improved. Robustness to label noise is enhanced based on loss regularization. These improvements are achieved through a regularization component that avoids generating an overconfident system through regularization applied to the loss function. Knowledge about the label noise for each anomaly is increased. The expert reading procedure results in a noise ratio between the original label and the expert label. The received label noise ratio helps analyze the interpretability difficulties of different anomalies in chest radiograph evaluation. Training time may be reduced due to regularization. The baseline classification loss can be extended to handle label noise using one or two regularization components. Prior label noise probabilities and anomaly correlation information are integrated, which improves the accuracy of the classification system.

[0073] Figure 4 A block diagram of one embodiment of the arrangement including a system 40 for machine training for anomaly classification and a system 45 for anomaly detection in medical imaging is shown. The system 40 for training uses noisy training data 42 to train a machine learning model 44 using noise, correlation, and / or other regularization. The machine learning model 48, previously trained using regularization, is used by the system 45 to apply it to a patient.

[0074] Systems 40 and 45 are two independent systems. The only shared component is the final machine learning model 44 used for training system 40, which is copied and / or used as the machine learning model 48 for detection in system 45. In alternative embodiments, one or more components are shared, such as memories 41 and 50 being the same memory and / or processors 43 and 47 being the same processor. It is possible to provide one system 40 and 45 without the other system 45 and 40.

[0075] The system 40 for training includes a memory 41 and a processor 43. The memory 41 stores training data 42 and / or the machine learning model 44, such as the defined architecture of the model and values ​​for learnable parameters. The processor 43 is used for machine learning. Additional, different, or fewer components may be provided. For example, a network or network connection is provided, such as for networking the memory 41 with the processor 43. As another example, a user interface or user input device is provided along with the processor 43 for defining the machine learning model 44, controlling training, and / or obtaining training data 42.

[0076] Memory 41 and processor 43 are part of a server, workstation, or computer. Memory 41 is part of a computer associated with processor 43, or is used for accessing a separate or remote database via a computer network, such as in a cloud-hosted electronic health record or electronic medical record system.

[0077] The system 45 for detection includes one or more medical imagers 46, a processor 47, a memory 50 (e.g., a medical record database), and a display 49. Additional, different, or fewer components may be provided. For example, a user interface or input device may be provided on the medical imager 46 and / or for the processor 47. In another example, a network or network connection may be provided, such as for networking different components (e.g., networking the medical imager 46 with the processor 47 and / or networking the processor 47 with the memory 50).

[0078] Memory 50, processor 47, and / or display 49 are part of a server, workstation, or computer. In one embodiment, memory 50, processor 47, and / or display 49 are a server or workstation. Memory 50 may be part of the same computer as processor 47 or a different computer, such as in a cloud-hosted electronic health record or electronic medical record system. Medical imager 46 and processor 47 are located in different facilities, such as remotely to each other, or in the same facility. Alternatively, processor 47 may be part of medical imager 46 or located in the same facility as medical imager 46 (i.e., locally).

[0079] Memory 41 and 50 are random access memory, system memory, cache memory, hard disk drive, optical media, magnetic media, flash drive, buffer, database, combinations thereof, or other memory devices now known or hereafter developed for data storage. Memory 41 stores training data 42, loss data, regularization data, and / or machine learning model 44. For example, anatomical images and ground truth classifications for the images are stored as training data 42. In one embodiment, training data 42 is an X-ray image, such as a chest X-ray. Memory 50 stores patient information (e.g., one or more images and clinical data), machine learning model 48, and / or output detections.

[0080] Memory 41, 50, or other memory may alternatively or additionally be a non-transitory computer-readable storage medium storing data representing instructions executable by program processor 43, program processor 47, and / or medical imager 46. Instructions for implementing the processes, methods, and / or techniques discussed herein are provided on a non-transitory computer-readable storage medium or memory, such as cache, buffer, RAM, removable media, hard disk drive, or other computer-readable storage medium. Non-transitory computer-readable storage media include various types of volatile and non-volatile storage media. Functions, actions, or tasks illustrated in the figures or described herein are performed in response to one or more sets of instructions stored in or on a computer-readable storage medium. Functions, actions, or tasks are independent of a particular type of instruction set, storage medium, processor, or processing strategy and can be performed by software, hardware, integrated circuits, firmware, microcode, etc., either individually or in combination. Similarly, processing strategies may include multiprocessing, multitasking, parallel processing, etc.

[0081] In one embodiment, the instructions are stored on a removable media device for reading by a local or remote system. In other embodiments, the instructions are stored at a remote location for transmission over a computer network or telephone line. In still other embodiments, the instructions are stored within a given computer, CPU, GPU, Tensor Processing Unit (TPU), Neural Processing Unit, AI accelerator, or system.

[0082] Processors 43 and 47 are general-purpose processors, control processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), GPUs, AI accelerators, neural processing units, TPUs, or other hardware processors used for training model 44 and / or applying machine learning model 48. In one embodiment, processor 43 is part of a computer, workstation, server, or other device configured for machine training. Processor 47 is part of a computer, workstation, server, or other device configured to apply image processing and / or apply machine learning model 48 to a given patient. Processors 43 and 47 may be a network of computing devices, such as multiple computers or servers. Processors 43 and 47 are configured by software, hardware, and / or firmware.

[0083] Processor 43 is configured to perform machine training based on training data 42. Machine training includes calculating a loss using noise and / or correlation regularization. Noise in the labels between anomalies and / or anomalies is canceled out using regularization. Expert reading studies and / or the correlation between anomalies and comorbidities are used to determine the weights, functions, or other aspects of the regularization. Machine training using the regularized loss results in a machine learning classifier or model 44. The trained model 44 or a copy thereof is provided to system 45 as a machine learning model 48 for detection.

[0084] For application to a given patient, medical imager 46 scans the patient and / or loads one or more stored images from previous scans from memory 50. Medical imager 46 is an MR, CT, X-ray, ultrasound, nuclear medicine (e.g., PET or SPECT) or other scanner. In yet another embodiment, medical imager 46 is a multimodal device, such as a combination of nuclear medicine and X-ray or CT. In yet another embodiment, invasive, other non-invasive, or minimally invasive imaging systems are used.

[0085] Medical imager 46 is configured to scan or image a patient. The same imager 46 can be used to scan different patients at different times. Other imagers 46 can be used to scan other patients. Medical imager 46 is configured to output scan data to processor 47, memory 50, and / or display 49. Scan data is data resulting from a scan at any stage of processing. For example, an image generated from a scan is provided. For an X-ray system, the image may be a chest radiograph. Medical imager 46 provides image data as scan data resulting from a scan, which has any amount of processing for generating the image. Image data may be formatted for display (such as RGB values) or may be in a scan format (e.g., scalar values).

[0086] Processor 47 is configured to apply machine learning model 48 to images of patients. Machine learning model 48 is trained using noise and / or correlation regularization to detect anomalies in the images. For any given anomaly used in machine training, noise regularization takes into account noise in the ground truth labels of the training data 42. Correlation regularization takes into account erroneous representations between different types of anomalies in the training data 42 used in machine training. Processor 47 is configured to apply machine learning model 48 to scan data with or without other data (e.g., the patient's clinical data).

[0087] Display 49 is a monitor, LCD, projector, plasma display, CRT, printer, or other device now known or later developed for displaying images of a patient classified as having or not having one or more abnormalities based on detections from an application. Display 49 is located at medical imager 46, processor 47, a physician's computer, or other location. Display 49 receives output from processor 47, medical imager 46, or memory 50. Processor 47 formats data for display (e.g., maps to RGB values) and stores images in a buffer, configuring display 49. Display 49 uses the images in the buffer to generate images for viewing.

[0088] The output from machine learning model 48 is displayed. Classification can be indicated along with anatomical images. Images include graphics, alphanumeric text, anatomical scans, coded spatial representations of anatomy, and / or combinations showing the classification along with, or not showing, anatomical or medical images.

[0089] While the invention has been described above with reference to various embodiments, it should be understood that many changes and modifications can be made without departing from the scope of the invention. Therefore, the foregoing detailed description is intended to be illustrative rather than restrictive, and it should be understood that the following claims, encompassing all equivalents, are intended to define the spirit and scope of the invention.

Claims

1. A method for machine learning assessment of anomalies in medical imaging, the method comprising: Obtain training data, which includes medical images and ground truth labels for the medical images, the ground truth labels indicating any anomalies represented by the medical images; The model is trained by a machine using the training data. The machine training uses a loss function, which includes regularization, including noise regularization and correlation regularization. as well as The model resulting from the machine training is stored in memory; The ground truth labels indicate at least a first type of anomaly and a second type of anomaly, and the machine training includes machine training with a regularization that includes the correlation regularization of the loss function, which correlates the ground truth labels for the first type of anomaly with the ground truth labels for the second type of anomaly.

2. The method of claim 1, wherein machine training comprises machine training using a loss function, said loss function comprising a cross-entropy function that compares the classification of the model’s outlier outputs with the ground truth labels.

3. The method of claim 1, wherein machine training comprises machine training using the ground truth labels and the loss function, the ground truth labels comprising binary labels for the presence or absence of the anomaly, and the loss function being weighted as a function of the number of positive and negative instances of the anomaly in the medical images of the training data.

4. The method of claim 1, wherein machine training comprises machine training with regularization of the loss function including the noise regularization.

5. The method of claim 4, further comprising measuring the noise level of the ground truth label, wherein machine training includes machine training using noise regularization as a function of the noise level.

6. The method of claim 5, wherein the noise level includes specificity and sensitivity to the ground truth label of the anomaly, and wherein the noise regularization includes a first weight as a function of the specificity and a second weight as a function of the sensitivity.

7. The method of claim 4, wherein the noise regularization comprises an inverse binary cross-entropy function.

8. The method of claim 1, wherein the correlation regularization includes covariance.

9. The method of claim 8, wherein the at least first and second types of anomalies comprise at least four types of anomalies, and wherein machine training comprises machine training utilizing the correlation regularization as the sum of the covariances among all at least four types of anomalies.

10. The method of claim 1, wherein machine training comprises machine training using the regularization that includes both the noise regularization and the correlation regularization.

11. The method of claim 1, wherein obtaining comprises: The medical images are obtained as training data for chest X-rays, and the abnormalities include effusion, cardiac hypertrophy, consolidation, atelectasis, and masses.

12. The method of claim 1, further comprising applying the model resulting from the machine training to patient images of patients, wherein the classification of the patient images output by the application has or does not have any anomalies.

13. A system for anomaly detection in medical imaging, the system comprising: A medical imaging system configured to generate images of a patient; A processor configured to apply a machine learning model to images of the patient, the machine learning model having been trained using noise regularization and correlation regularization to detect anomalies in the images, wherein the machine learning model is trained using the method of any one of claims 1-12. and A display configured to classify the patient as having or not having the abnormality based on detections from the application.

14. The system of claim 13, wherein the processor is configured to apply the machine learning model trained using the noise regularization, the noise regularization taking into account noise in the ground truth labels used in machine training.

15. The system of claim 13, wherein the processor is configured to apply the machine learning model trained with correlation regularization, the correlation regularization taking into account erroneous representations between different types of anomalies.

16. The system of claim 13, wherein the medical imaging system comprises an X-ray system, and wherein the noise regularization or correlation regularization is used for ground truth labeling of anomalies in the X-ray image.

17. A system for machine training for anomaly classification, the system comprising: A memory configured to store training data and a machine learning classifier, the training data including anatomical images and ground truth classifications for the images; and A processor configured to perform machine training using training data according to any one of claims 1-12.

18. The system of claim 17, wherein the processor is configured to perform machine training using the noise regularization.

19. The system of claim 17, wherein the processor is configured to perform machine training using the correlation regularization.

Images