Conformal training of machine learning models

Abstract

A method, computer system, and apparatus, including a computer program encoded on a computer storage medium, for training a classification machine learning model, the system obtains calibration training examples and prediction training examples, determines a threshold based on the calibration training examples, generates data characterizing a predicted confidence set based on the threshold and the prediction training examples, and updates model parameters based on at least the predicted confidence set.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 252,521, filed October 5, 2021, the entire disclosure of which is incorporated herein by reference.

[0002] This specification relates generally to training machine learning models to generate predictions regarding inputs, and in particular to training machine learning models to generate conformal classifications that define a confidence set containing true predictions at a user-specified confidence level. [Background technology]

[0003] A machine learning model is configured to process input data according to the parameters of the machine learning model to generate an output that defines a prediction or decision. For example, a neural network is a machine learning model that uses one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as an input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from the received input according to the current values of a respective set of parameters.

[0004] The parameters of a machine learning model may be determined through a training process based on training data including one or more training examples. For example, a neural network may be trained by updating network parameters, including, for example, weights and bias coefficients of network layers of the neural network. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Sadinle et al., “Least ambiguous set-valued classifiers with bounded error levels,” Journal of the American Statistical Association (JASA), 114(525):223–234. [Non-patent document 2] Blondel et al., "Fast differentiable sorting and ranking", Proc. of the International Conference on Machine Learning (ICML), 2020 [Non-patent document 3] Cuturi et al., “Differentiable ranking and sorting using optimal transport,” Advances in Neural Information Processing Systems (NeurIPS), 2019 [Non-patent document 4] Romano et al., “Classification with valid and adaptive coverage,” Advances in Neural Information Processing Systems (NeurIPS), 2020 Summary of the Invention [Means for solving the problem]

[0006] Conformal prediction provides uncertainty estimates and explicit guarantees regarding output predictions from an underlying machine learning model, such as a classification model. For example, a conformal predictor can predict a confidence set containing the true classification at a user-specified confidence level for the output generated by a classification model. The predictive efficiency of a conformal predictor can be measured by the size of the confidence set, with larger confidence set sizes indicating reduced predictive efficiency.

[0007] This specification describes methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for training a classification machine learning model for use with a conformal predictor to predict confidence sets.

[0008] Generally, a classification model is configured to process model inputs to generate, for each particular classification in a set of classifications, a classification output that indicates the predicted probability of the model input with respect to the particular classification.

[0009] A classification model can be a classifier neural network configured to perform any of a variety of classification tasks. As used herein, a classification task is any task that requires the model to generate an output that includes a respective score (e.g., predicted probability) for each of a set of multiple categories. The respective scores can be used to select one or more of the categories as the "classification" of the model input using the respective scores.

[0010] An example of a classification task is image classification, where the input to the classification model is an image, e.g., intensity values of pixels in the image, the categories are object categories, and the task is to classify the image as depicting an object from one or more of the object categories. That is, the classification output for a given input image indicates a prediction of one or more object categories depicted in the input image. As used herein, image may refer to a still image or a moving image (e.g., video). The input to the classification model (i.e., image data) may include pixels of the image or another representation of the image, such as might be generated by an encoder, e.g., an encoder neural network. The image may include color or monochrome pixel value data. Such an image may be captured from an image sensor, such as a camera or LIDAR sensor.

[0011] Another example of a classification task is text classification, where the input to a classification model is text and the task is to classify the text as belonging to one of multiple categories. One example of such a task is a sentiment analysis task, where the categories each correspond to a different possible sentiment of the task. Another example of such a task is a reading comprehension task, where the input text includes a context passage and a question, and the categories each correspond to a different segment from the context passage that may be an answer to the question. Other examples of text processing tasks that can be categorized as classification tasks include entailment tasks, paraphrasing tasks, text similarity tasks, sentiment tasks, sentence completion tasks, grammaticality tasks, etc.

[0012] Other examples of classification tasks include audio classification tasks, such as voice processing tasks, where the input to the classification model is audio data representing speech. Examples of voice processing tasks include language identification (where categories are different possible languages of speech), hot word identification (where categories indicate whether one or more particular “hot words” are said in the audio data), etc. The audio data may be obtained from a sensor, e.g., an audio transducer or microphone, and may include a representation of a digitized audio waveform, e.g., a speech waveform. Such a representation may include samples representing digitized amplitude values of the waveform, or a time-frequency domain representation of the waveform, e.g., an STFT (Short-Time Fourier Transform) or MFCC (Mel-Frequency Cepstral Coefficients) representation. The output of the (audio) classification model may include predictions that audio signals indicative of different categories of audio signals (e.g., signals corresponding to an alarm sounding, a person speaking, a machine or vehicle operating, etc.) are present in the input audio data, in which case the output may be respective category scores indicating the likelihood of the different audio signals being present in the audio data or a segment of audio data. Classification tasks may include speech or sound recognition tasks (e.g., category scores may indicate the likelihood that each different word is present in the audio data), telephone or speaker classification tasks (e.g., category scores may indicate the likelihood that each different speaker was speaking in the audio data).

[0013] In general, input data to a classification model may include any signal, e.g., waveforms (e.g., time series) of signals (e.g., image data of an object) from a sensor, e.g., a sensor sensing a physical feature of a real-world object. In some implementations, the sensor data may include data characterizing the state of a robot or vehicle, e.g., pose data and / or position / velocity / acceleration data, or data characterizing the state of an industrial plant, factory, or data center, e.g., sensed electronic signals such as sensed current and / or temperature signals. The classification task may include an event or state tagging task, in which case the output of the classification model may include respective category scores indicating whether different events or different states (e.g., of an environment or equipment within the environment) are represented in the input data or segments of the input data. In some examples, such events or states may include the operational state of a device or machine and / or transitions between different states, e.g., a transition between an expected or functioning state and an unexpected or non-functioning state of a device or machine. The classification model output may include category scores for each of the different categories that indicate the physical characteristics (eg, size, position, speed) or state of the object or environment.

[0014] A conformal predictor can be used to assess the uncertainty of a classification prediction output by a classification machine learning model. For any suitable classification model applied to any classification task, the predictor is configured to process an input including a classification output from the classification model to generate data specifying a confidence set including one or more output classifications selected from a set of classifications such that, for a model input to the classification model, the probability that the true classification of the model input is included in the confidence set is equal to or greater than a confidence level, e.g., a user-defined confidence level. That is, the classification accuracy of the confidence set output by the conformal predictor (in the sense that the confidence set includes the true classification) is guaranteed to comply with a confidence level that can be defined by a user depending on the particular application.

[0015] The above properties of conformal predictors are important for achieving the reliability of conformal classification machine learning models, especially in applications where the risk of failure is high, such as medical diagnosis, autonomous vehicles, and robotics. For example, in computer-aided cancer detection, it may be more important and useful to capture a set of potential diagnoses with high confidence (e.g., a 99% confidence level) than to provide a single most likely diagnosis without providing any guarantee of reliability.

[0016] In one example, the classification model and conformal predictor are used for medical screening and diagnostic tasks, where the input to the model is medical data characterizing a user's health, and the model output is a prediction across multiple categories, each representing a different diagnosis. The model input may include medical images, such as one or more medical scans, X-rays, CT scans, MRI scans, or ultrasound scans of the subject. Alternatively or additionally, the model input may include microscopic histology images from biopsied tissue. Alternatively or additionally, the model input may include physiological parameters of the subject, such as the subject's BMI, blood pressure, heart rate, diabetic status, serum levels of various hormones, and genetic markers, as well as symptom information of the subject, such as the location and severity of bodily discomfort. Alternatively or additionally, the model input may optionally include other information characterizing the subject. The classification model may be configured to process the model input to predict probability scores for multiple diagnostic categories. For example, when a medical diagnostic task is cancer, e.g., breast cancer, screening and detection, the diagnostic categories may include different diagnoses for breast cancer screening and detection, including, for example, “normal,” “adenosis,” “fibroadenoma,” “ductal carcinoma,” “tubular carcinoma,” “lobular carcinoma,” etc. Because missing an early diagnosis of a breast malignancy such as invasive ductal carcinoma can cause a significantly worse prognosis for the patient, it may be important to include a set of possible diagnoses in the prediction with a guaranteed confidence level to guide follow-up examinations and interventions. For example, on the one hand, the classification output may indicate that the most likely diagnosis is the benign breast tumor “fibroadenoma” (e.g., with a predicted probability of 0.6). On the other hand, the confidence set predicted by a conformal predictor at a 95% confidence level may include {“fibroadenoma,” “ductal carcinoma,” “tubular carcinoma”}. Thus, the predicted confidence set provides more information than the classification output alone, as simply looking at the highest scoring category does not indicate any cause for concern, but malignancies included in the predicted confidence set should cause concern and follow-up.

[0017] In another example, the classification model and conformal predictor are used onboard an autonomous vehicle. In this example, the input to the model can be sensor data captured by the vehicle's sensors, such as image data, LIDAR data, radar data, or some combination, and the classification model output is a respective score for each of multiple object categories representing different types of objects, such as vehicles, pedestrians, traffic signs, cyclists, etc. Because missing a pedestrian or cyclist in a scene can cause a fatal error in vehicle operation, it can be important to include a set of possible classifications of the object in the prediction with a guaranteed confidence level to guide vehicle operation. For example, on the one hand, the classification output may indicate that the most likely classification is a traffic sign. On the other hand, the confidence set predicted by the conformal predictor at a 95% confidence level may include a pedestrian and / or a cyclist. Thus, the predicted confidence set provides more information than the classification output alone, as simply looking at the highest scoring category does not indicate any cause for concern, but pedestrians and / or cyclists included in the predicted confidence set should signal vigilance in maneuvering the vehicle.

[0018] In one innovative aspect, this specification describes a method for training a conformal classification machine learning model, the method being implemented by a system including one or more computers.

[0019] The system obtains a set of calibration training examples and uses the calibration training examples to determine a threshold. Each calibration training example includes a respective training model input and a respective classification label for the respective training model input. For each respective calibration training example in the set of calibration training examples, the system processes the respective training model input of the respective calibration training example using a classification model according to current values of the model parameters to generate a respective classification output for the respective training model input. The system determines the threshold based on at least the classification output generated for the calibration training example, the classification label for the calibration training example, and a confidence level.

[0020] The system further obtains a set of prediction training examples. Each prediction training example includes a respective training model input and a respective classification label for the respective training model input. For each respective prediction training example in the set of prediction training examples, the system processes the respective model input of the respective prediction training example using a classification model according to current values of the model parameters to generate a respective classification output, and processes the input including the respective classification output and a threshold with a smooth prediction function to generate a respective prediction output. The smooth prediction function is differentiable with respect to the input to the smooth prediction function (e.g., one or more derivatives of the smooth prediction function with respect to the input may be continuous functions).

[0021] The system determines gradients of the training loss with respect to the model parameters. The training loss includes at least an inefficiency loss that measures, for each respective predicted training example, a value indicative of the size of each predicted confidence set characterized by each predicted output. The system uses the gradients to update the current values of the model parameters.

[0022] In some implementations, each prediction output includes a confidence score for each classification in the set of classifications, each confidence score characterizing the respective probability that the respective classification is included in the predicted confidence set.

[0023] In some implementations, the classification model includes a neural network with any suitable architecture, such as a multi-layer perceptron or a ResNets.

[0024] In some implementations, the inefficiency loss measures a weighted size that is calculated based on the size of each predicted confidence set scaled by a weighting factor for a set of classifications.

[0025] In some implementations, the training loss further includes a classification loss that measures the classification error of the output classifications included in the predicted confidence set characterized by each predicted output.

[0026] The classification loss may include a first term that measures the error of not including the true classification in the predicted confidence set characterized by the predicted output for each respective predicted training example.

[0027] In implementations, the model inputs may include sensor data (e.g., image, video, and / or audio data), with the classifications corresponding to object categories, and / or the model inputs may include medical data such as patient physiological measurements, with the classifications each corresponding to a different patient diagnosis, and / or the model inputs may include environmental observations, with the classifications each corresponding to a different state of the environment.

[0028] The classification loss may also include a second term that measures the error of including a specified classification in the respective output confidence set rather than the true classification for each respective predicted training example. The specified classification may be determined as a classification opposite to the true classification based on a priori information (i.e., prior information, or information known "a priori," e.g., information separate from or in addition to information included in the model inputs for determining the classification output). For example, in the case of medical diagnosis classification, it may be desirable to not include high-risk diseases such as "ductal carcinoma" in the confidence set when the ground truth label is "normal." This is motivated by avoiding unnecessary anxiety or testing for the patient. Therefore, the second term in the classification loss may be used to penalize certain high-risk classifications when the true label indicates a low-risk classification.

[0029] In some implementations, the prediction function includes a smoothed implementation of a threshold function applied to the difference between a first value calculated based on the classification output and a second value calculated based on a threshold. In one example, for each respective prediction training example in the set of prediction training examples, the first value may be calculated as the predicted probability indicated by the respective classification output for a particular classification. In another example, for each respective prediction training example in the set of prediction training examples, the first value may be calculated as the sum of multiple predicted probabilities indicated by the respective classification outputs for multiple classifications from the set of classifications. The smoothed threshold function may be a sigmoid function.

[0030] In some implementations, in determining the threshold, the system processes the classification output from the classification model, the classification labels of the calibration training examples, and a confidence value with a smooth calibration function that outputs the threshold. The smooth calibration function is differentiable with respect to the classification output. The calibration function may be configured to, for each respective calibration training example in the set of calibration training examples, calculate a respective conformity score based on the respective classification output and the respective classification label, and perform a smoothed implementation of a quantile operation on the conformance score and the confidence level.

[0031] In some implementations, the system further obtains a batch of training examples and randomly samples the batch of training examples to generate a set of calibration training examples and a set of prediction training examples. The system can also obtain multiple batches of training examples and repeatedly train the classification machine learning model on each of the batches of training examples.

[0032] This specification also describes a method for performing conformal classification. The method is implemented by a system including one or more computers. The system obtains model inputs, obtains a classification model trained using one of the methods described above, and processes the model inputs with the classification model to generate, for each specific classification in a set of classifications, a classification output indicating the predicted probability of the model input for the specific classification. The system further obtains a threshold predicted by a non-smooth calibration function according to a confidence value based on a calibration dataset, and processes the model output and the threshold to generate a confidence set that may include one or more output classifications selected from the set of classifications. (If none of the classifications meet the confidence value condition, the output confidence set may be empty.)

[0033] In implementations, the method may further include (automatically) performing one or more of the following steps based at least in part on the confidence set: controlling (i.e., giving commands to) a robot or vehicle (e.g., an autonomous or semi-autonomous land, air, or sea vehicle), where the model input includes sensor data (e.g., image, video, and / or audio data) acquired by one or more sensors (e.g., sensors of the robot or vehicle) and the classifications correspond to object categories (e.g., categories of objects with which the robot or vehicle may interact or manipulate, or that are present in an environment surrounding the robot or vehicle); providing one or more medical diagnoses, where the model input includes medical data (e.g., medical images) such as physiological measurements of a patient, and the classifications each correspond to a different diagnoses of the patient; and controlling an agent to perform a task in an environment, where the model input includes observations of the environment (e.g., sensor data acquired from sensors of the environment), and the classifications each correspond to a different state of the environment.

[0034] In some implementations, a robot (e.g., a stationary or moving machine) interacts with a real-world environment to accomplish a specific task, for example, to locate an object of interest within the environment, or to move an object of interest to a specified location within the environment, or to navigate to a specified destination within the environment.

[0035] In some implementations, the agent may perform actions within a real-world (or simulated) environment based at least in part on the confidence set. The environment may include, for example, equipment within a facility such as a data center, a server farm, a power transmission or water distribution system, or a manufacturing plant or service facility. The observations may then be related to the operation of the plant or facility. For example, additionally or alternatively, the observations may include observations of power or water use by the equipment, observations of power generation or distribution control, or observations of resource use or waste. The agent may perform actions within the environment to increase efficiency, for example, by reducing resource use and / or reducing the environmental impact of operations in the environment, for example, by reducing waste, and / or for safety reasons (e.g., to prevent harm to equipment and / or users). The actions may include actions that control or impose operating conditions on equipment in the plant / facility and / or result in changes to settings of the plant / facility's operation, for example, to adjust or turn on / off components of the plant / facility.

[0036] This specification also describes a system that includes one or more computers and one or more storage devices that store instructions that, when executed by the one or more computers, cause the one or more computers to perform the methods described above.

[0037] This specification also describes one or more computer storage media that store instructions that, when executed by one or more computers, cause the one or more computers to perform the methods described above.

[0038] In situations where the systems discussed herein may collect information about users or utilize such information, users may be given the opportunity to control whether a program or feature collects user information. Additionally, certain information may be processed in one or more ways before it is stored or used to remove personally identifiable information from the information. Thus, users may be able to control how information is collected about them and used by the systems described herein.

[0039] The subject matter described herein can be implemented in particular embodiments to realize one or more of the following advantages.

[0040] Recent deep learning-based classifiers demonstrate very high accuracy on test data, but do not guarantee safe deployment, especially in high-risk machine learning applications such as medical diagnosis and autonomous driving. For typical machine learning classification models, predictions are obtained without reliable uncertainty estimates and explicit guarantees. Conformal prediction addresses these issues by using the classifier's predictions, e.g., its probability estimates, to predict a confidence set containing the true class with a user-specified probability.

[0041] Conformal prediction is typically used as a separate processing step after a machine learning model has been trained. That is, the machine learning model is not trained with the objective of predicting an optimal confidence set. For example, a machine learning classification model is typically trained to minimize cross-entropy loss without considering factors such as reducing the expected size of the confidence set output by the conformal prediction function. After the classification model is trained, the parameters of the classification model are fixed when the conformal predictor is used to calculate the confidence set, leaving the conformal predictor little or no control over the predicted confidence set, such as its size (inefficiency) or composition (i.e., the classes it contains).

[0042] To overcome the limitations of conventional conformal prediction methods, this specification provides techniques for end-to-end training of classification models using conformal predictors. By developing a differentiable model for conformal prediction and training the classification model with an objective calculated based on a predicted confidence set, the described techniques enable optimization of specific objectives defined for the predicted confidence set without negating the guarantee. For example, the provided techniques can be used to generate conformal predictions that reduce inefficiencies, i.e., the size of the predicted confidence set, and therefore provide better guidance in decision-making based on the predicted classification. Reducing inefficiencies in the output confidence set can benefit many practical applications. For example, in medical diagnosis, smaller confidence sets are important to avoid confusion and anxiety for doctors and patients, ultimately leading to better diagnoses and improved decision-making regarding medical interventions.

[0043] Furthermore, the described techniques enable shaping the confidence set according to the application scenario by designing the inefficiency loss and / or classification loss used for training. This allows the system to better utilize a priori information, such as the importance of certain classifications and / or the relationships between different classifications. For example, in the context of medical diagnosis, the provided techniques can be used to reduce inefficiency, i.e., uncertainty, particularly regarding low-risk diseases. This may guide doctors to focus more on high-risk diseases. Alternatively or additionally, the described techniques can be used to enforce constraints on the composition of the confidence set, such that the confidence set is less likely to include two diseases that are frequently confused by doctors, or to include both high-risk and low-risk diseases. This flexibility in shaping the confidence set optimization objectives can be advantageous in certain scenarios to further improve diagnosis and decision-making. [Brief explanation of the drawings]

[0044] [Figure 1] FIG. 1 illustrates an exemplary conformal training system for training a classification model. [Figure 2] FIG. 1 illustrates an example of performing conformal training of a classification machine learning model. [Figure 3] FIG. 1 illustrates an exemplary conformal classification system. [Figure 4] 1 is a flow diagram illustrating an example process for training a classification model. [Figure 5] 1 is a flow diagram illustrating an example process for performing conformal classification. [Figure 6] FIG. 10 shows a performance comparison between a classification model trained using conformal training and a base classification model without conformal training in predicting the confidence set. DETAILED DESCRIPTION OF THE INVENTION

[0045] Like reference numbers and designations in the various drawings indicate like elements.

[0046] 1 shows an example of a conformal training system 100 for training a machine learning model, e.g., a classification machine learning model 120. System 100 is an example of a system implemented as a computer program on one or more computers at one or more locations in which the systems, components, and techniques described below may be implemented.

[0047] Generally, the classification machine learning model 120 is configured to perform a classification task, i.e., to process input data and generate output data that reveals predicted classification information for the input data. The input data can reveal any type of data to be analyzed, e.g., images, text, audio signals, sensor measurements, event logs, etc. The output data can reveal classification information for the input data, e.g., one or more object categories for objects depicted by input images, one or more text categories for input text, one or more status categories for a system, one or more health state categories for a person, etc.

[0048] The classification model 120 can be any suitable machine learning model for performing classification tasks. In some implementations, the classification model 120 can be a classifier neural network having any suitable architecture configured to perform any of a variety of classification tasks. For example, a classifier neural network can be used to process inputs and generate outputs including respective scores (e.g., predicted probabilities) for each of a set of multiple classifications. The respective predicted probabilities can be used to select one or more of the categories as the "classification" of the model input.

[0049] The goal of the conformal training system 100 is to optimize the parameters (e.g., network weights and bias coefficients) of the classification model 120 with respect to one or more optimization measures of a predicted confidence set based on a confidence level 116. This process may be referred to as conformal training of the classification model 120. The predicted confidence set includes one or more output classifications selected from a set of classifications such that, for a model input of the classification model 120, the probability that the true classification of the model input is included in the confidence set is equal to or greater than the confidence level 116. That is, the predictive accuracy of the confidence set (in the sense that the confidence set includes the true classification) is guaranteed to meet a confidence level that may be defined by a user depending on the particular application. As discussed below, the optimization measure of the confidence set may be configured to aim to minimize inefficiency, i.e., the size of the confidence set, while ensuring inclusion of the true classification at the confidence level 116.

[0050] The training process performed by system 100 differs from training a classification model in a typical conformal classification process. In typical conformal classification, the classification model is pre-trained on training data without considering the characteristics of a predicted confidence set. For example, the classification model may be trained using only cross-entropy loss. After the classification model is trained, a conformal calibration process is applied to the classification model (with its model parameters fixed at their trained values) on a calibration dataset to determine a threshold τ for the suitability score, which is calculated based on the predicted classification probabilities. The suitability score may be understood as a measure that quantifies the suitability or insuitability (or "strangeness") of a data sample in a given dataset. The threshold τ may be determined by analyzing the data distribution of the classification model output on the calibration dataset with respect to a user-specified confidence level. The threshold τ may then be compared with the suitability score calculated for the classification model output to determine the confidence set.

[0051] In contrast, in the conformal training process performed by system 100, classification model 120 is trained end-to-end with conformal calibration (to determine threshold τ 135) and conformal prediction (to predict the confidence set). As a result, the training process directly optimizes the properties of the predicted confidence set. This process offers several advantages over regular conformal classification, as discussed below.

[0052] The system 100 acquires a number of training examples 110. Each training example includes a training model input, e.g., x i and x i The ground truth classification labels of each, e.g., y i In some implementations, the system 100 may perform conformal training of the classification model 120 using stochastic gradient descent on mini-batches of the set of training examples 110. For each batch of training examples, the system 100 may partition the batch (e.g., by randomly sampling the batch) into a set of calibration training examples 112 and a set of prediction training examples 114. Generally, the calibration training examples 112 and the set of prediction training examples 114 are disjoint sets. As described below, the system 100 may use the calibration training examples 112 in a calibration process to determine a fitness score threshold τ. The system 100 may use the prediction training examples 114, along with the threshold τ, in a prediction process to predict data characterizing a predicted confidence set for each prediction training example 114. The predicted confidence set may be used to calculate a training loss to update the model parameters of the classification model 120.

[0053] During the calibration process, each calibration training example 112, e.g., (x i ,y i ), the system 100 calculates the training model input x i Each classification output π θ (x i) according to the current values of the model parameters θ. θ Using each training model input x i For example, if the set of multiple classifications contains K classifications [K], the classification output π θ (x i ) is the K predicted probabilities {π θ,k (x)}.

[0054] Each calibration training example (x i ,y i ), the system 100 calculates the ground truth label y i Based on the model output π θ (x i ) suitability score E θ (x i ,y i ) is calculated.

[0055] In some implementations, according to a threshold conformal predictor (THR) formulation, the fitness score of a classification model output is: E θ (x,k):=π θ,k (x) (1) It is defined as follows:

[0056] That is, the fitness score for a particular classification k is the classification model output π θ,k The predicted probability π for class k in (x) θ,k In typical conformal classification using the THR formulation, the confidence set is given by C θ (x;τ):={k:E θ (x,k):=π θ,k (x) ≥ τ} (2) is determined as follows.

[0057] That is, if the predicted probability for a particular classification is greater than or equal to a threshold, the particular classification is included in the confidence set C θ(x;τ). Details of the THR formulation can be found in Sadinle et al., "Least ambiguous set-valued classifiers with bounded error levels," Journal of the American Statistical Association (JASA), 114(525):223-234.

[0058] Also according to the THR formulation, the system 100 calculates the i ,y i ) with respect to the ground truth label y i Model output for π θ (x i ) suitability score E θ (x i ,y i ) is the predicted probability of the classification output for the ground truth classification of the model input, i.e.,

[0059]

number

[0060] It may be calculated as:

[0061] In practice, the fitness score may be defined as a logit (THRL) or log probability (THRLP) instead of a probability.

[0062] In conventional conformal prediction using the THR formulation, for a confidence level 1-α (where α is the significance level that limits the error rate of the predicted confidence set), the threshold τ is

[0063]

number

[0064] It is calculated as follows:

[0065] In the formula, |I cal | is the size of the calibration training example set 112. The quantile operation in conventional THR is to set the threshold τ to the classification output π θ (x i ) and make it non-differentiable with respect to

[0066] To enable end-to-end training of the classification model 120 and the conformal predictor, the system 100 uses a smooth calibration function 130 to perform a smoothed, i.e., differentiable, implementation of the quantile operation on the fitness scores, so that the calculated threshold τ is a function of the calibration example.

[0067]

number

[0068] is differentiable with respect to the classification output of θ and is therefore differentiable with respect to the model parameter θ. This can be achieved using an appropriate smooth sorting technique. One example of a sorting technique is described in Blondel et al., "Fast differentiable sorting and ranking," Proc. of the International Conference on Machine Learning (ICML), 2020. Another example of a sorting technique is described in Cuturi et al., "Differentiable ranking and sorting using optimal transport," Advances in Neural Information Processing Systems (NeurIPS), 2019. These smooth sorting implementations often include a "dispersion" hyperparameter ε such that the smooth sort approximates a "hard" sort for ε → 0.

[0069] In some implementations, following the adaptive prediction set (APS) formulation, the fitness score of a classification model output is

[0070]

number

[0071] It is defined as follows:

[0072] During the ceremony,

[0073]

number

[0074] where U is a uniform random variable in [0,1] for breaking ties. That is, the fitness score for a particular classification k is the sum of the predicted probabilities π ranked (from highest to lowest) such that the smallest predicted probability is scaled by the uniform random variable. θ,k It is defined as the sum of (x).

[0075] In typical conformal classification using the APS formulation, the confidence set is given by C θ (x;τ):={k:E θ (x, k) ≤ τ} (6) is determined as follows.

[0076] That is, after ranking the predicted probabilities for a set of classifications from highest to lowest based on Equations 5 and 6, if a particular classification k belongs to the subset of top-ranked probabilities whose sum is less than or equal to a threshold τ, it is included in the confidence set C θ (x;τ). Details of the APS formulation can be found in Romano et al., "Classification with valid and adaptive coverage," Advances in Neural Information Processing Systems (NeurIPS), 2020.

[0077] In typical conformal prediction using the APS formulation, for a confidence level of 1-α, the threshold τ is Suitability score E θ (x i ,y i ) of (1-α)(1+1 / |I cal |)Quantile (7) It is calculated as follows:

[0078] Similar to the smoothed implementation of the THR formulation for calculating the threshold τ, the system 100 uses a smooth calibration function 130 to perform a smoothed implementation of the quantile operation in the APS formulation for calculating the threshold τ, so that the calculated threshold τ is

[0079]

number

[0080] is differentiable with respect to the classification output of θ and hence the model parameters θ.

[0081] Although the THR and APS formulations are described above as examples, any other suitable formulation of conformal calibration may be used as long as the formulation can be adopted or converted into a differentiable form implemented by a smooth calibration function 130.

[0082] After the threshold (τ) 135 is determined by the smooth calibration function 130 based on the calibration training examples 112, the system 100 uses the smooth prediction function 140 to generate data characterizing the predicted confidence set. In particular, the system can use the smooth prediction function 140 to generate a confidence score 145 based on the predicted training examples 114 and the threshold 135. The confidence score 145 can be understood to characterize the probability that each classification will be included in the confidence set.

[0083] For each predicted training example 114, e.g., (x i ,y i ), the system 100 calculates the training model input xi Each classification output π θ (x i ) according to the current values of the model parameters θ. θ Using each training model input x i The system 100 then calculates the respective fitness scores E for each classification using the selected formulation in a smooth calibration process. θ (x i , k) and generate a confidence score 145 using a smooth prediction function 140.

[0084] In typical conformal prediction, the confidence set is determined by performing a threshold operation on fitness scores above a threshold 135. The threshold operation, and the typically discrete numbers representing the classifications selected for the confidence set, are not differentiable.

[0085] To enable end-to-end training of the classification model 120 and conformal prediction, the system 100 performs a smooth, i.e., differentiable, implementation of the thresholding operation. That is, the system 100 calculates the fitness score E θ A smooth threshold function is applied to the difference between a first value calculated based on (x, k) and a second value calculated based on a threshold τ. Any suitable smooth threshold function may be used. In one particular example, C θ,k (x;τ):=σ((E θ (x,k)-τ) / T) (8) A sigmoid function σ may be applied to the difference to calculate a confidence score 145, such that:

[0086] where T is the temperature hyper-parameter.

[0087] Trust score C θ,k(x;τ)∈[0,1] represents the soft assignment of classification k to the confidence set, i.e., it can be interpreted as the probability that k is included in the confidence set. For T→0, a "hard" confidence set is recovered, i.e., k∈C θ For (x;τ), C θ,k (x;τ)=1 otherwise, C θ,k (x;τ)=0.

[0088] The confidence score C above is calculated using a smooth threshold function θ,k (x;τ) is the fitness score E θ It is differentiable with respect to (x, k) and the threshold τ. Therefore, the confidence score C θ,k (x;τ) is such that the threshold τ 135 output by the smooth calibration function 130 is differentiable with respect to the classification output, so that the fitness score E(x,k) is proportional to the classification output π θ (x i ) is differentiable with respect to the model parameter θ.

[0089] After the confidence score 145 is calculated based on the predicted training examples 114 and the threshold, the system uses a parameter update engine 150 to update the model parameters of the classification model 120. For example, the parameter update engine 150 updates the confidence score C θ,k The gradient of the training loss computed with respect to (x;τ) with respect to the model parameters can be determined and any suitable backpropagation technique that uses the gradient can be used to update the current values of the model parameters.

[0090] As described with reference to FIG. 3 , after the classification model 120 is conformally trained, i.e., trained with a training loss calculated based on the confidence scores, the trained classification model 120 can be used with regular conformal calibration and prediction to predict a confidence set based on a user-defined confidence level. That is, a regular non-smooth conformal calibration process (of any suitable formulation) can be performed on a calibration dataset to determine a threshold τ based on a user-defined confidence level so that inclusion at the user-defined confidence level can be guaranteed. In some implementations, the threshold 135 can be calculated based on the same calibration dataset used to train the classification model 120. However, instead of using a “soft” sort of a smooth calibration function to train the classification model 120, the threshold 135 used to predict the confidence set is obtained using a “hard” sort to guarantee inclusion.

[0091] Generally, the training loss is at least a function of the inefficiency loss L, which measures a value indicative of the size of each predicted confidence set in each predicted output generated for each predicted training example. ineff That is, the inefficiency loss L ineff teeth, L ineff =Ω(C θ (x;τ)) (9) It can be defined as:

[0092] where Ω is a “smooth” size function of the expected inefficiency of the confidence set, e.g.,

[0093]

number

[0094] The trust score C π,k (x;τ) is the confidence set C θThis can be understood as a soft allocation of class k to (x;τ), so that in some implementations, the inefficiency loss L ineff is based on the sum of the confidence scores of a set of classifications, e.g.

[0095]

number

[0096] It can be calculated as follows:

[0097] where by default, κ=1 to not penalize singletons. In general, κ∈{0,1} can be treated as a hyperparameter.

[0098] In some implementations, the training loss is a classification loss L that measures the classification error of the output classifications contained in the predicted confidence set characterized by each predicted output. class The total training loss is L=L class +λL ineff (11) It can be calculated as follows:

[0099] where λ is a hyperparameter.

[0100] In some implementations, the classification loss L class explicitly enforces inclusion, i.e., the ground truth label Y is included in C θ (X;τ), and optionally, C θ To penalize the other classes k from being included in the confidence set C θ To this end, in one implementation, the classification loss L class teeth,

[0101]

number

[0102] It can be defined as:

[0103] In the formula, L y,k [.] is the loss matrix.

[0104] As mentioned above, 1-C θ (x;τ) is k is C θ The confidence score C can be understood as the likelihood that the θ,k (x;τ)∈[0,1]. L class The first term of can be used to encourage inclusion, while the second term can be used to prevent predicting other classes. This is achieved by using the loss matrix L y,k Governed by [.]. L=I K , i.e., an identity matrix with K rows and columns, the loss matrix simply enforces inclusion. On the other hand, for any L y,k Setting L > 0 penalizes the model from including class k in the confidence set with ground truth y. Defining L in this way allows us to define complex objectives tailored to specific applications.

[0105] For example, in skin disease classification, predicting a set of classes, e.g., the top-k diseases, is already a common strategy for dealing with uncertainty. In these cases, in addition to inclusion guarantees, features of the confidence set can also be important. Constraints in terms of the predicted confidence set cannot be easily handled by regular conformal classification. On the other hand, conformal training performed by system 100 makes it possible to define complex objectives for enforcing constraints on the predicted confidence set.

[0106] For example, the system can reduce uncertainty about "low-risk" diseases at the expense of higher inefficiency, i.e., uncertainty, about "high-risk" diseases. This can be thought of as reallocating time spent by the physician to high-risk cases. Using conformal training, the system can reduce uncertainty about "low-risk" diseases by a weighted size loss L ineffWe can manipulate group- or class-conditional inefficiency using = ω Ω(C(X;τ)), where ω := ω(Y) depends on the ground truth Y.

[0107] The system 100 can further take into account which classes are included in the confidence set. For example, the classification loss can be designed to penalize "mix-ups" between pairs of classes. For example, if two diseases are frequently confused by doctors, it makes sense to train a model that avoids confidence sets that include both diseases. Reducing the probability of including both the y classification and the k classification in the confidence set reduces the positive component L in Equation (12). y,k This can be achieved by using >0.

[0108] Additionally, the conformal training process of system 100 allows for explicit penalization of "overlap" between groups of classes in the confidence set. For example, a medical professional may want to avoid including very high-risk diseases in a confidence set alongside low-risk diseases to avoid unnecessary anxiety or testing for the patient. To reduce the probability of including a class from a particular group of classes, K1, in a confidence set that also includes any of class K0, the system may use the L y,k >0 can be set.

[0109] 2 illustrates an example of performing a conformal training process 200 for a classification machine learning model. For convenience, conformal training process 200 is described as being performed by one or more computer systems located at one or more locations. For example, a conformal training system suitably programmed in accordance with this specification, such as conformal training system 100 of FIG. 1, can perform process 200.

[0110] 2, the classification model 220 is configured to process training example model inputs 210a to generate classification outputs 225. The system may split the training example dataset into calibration batches 212 and prediction batches 214, for example, with a 50%-50% split.

[0111] The system applies a smooth calibration function 230 to the classification outputs generated from the calibration batch 212, the ground truth labels of the calibration batch 212, and a user-specified significance level α to generate a threshold τ, which is differentiable with respect to the model parameters of the classification model 220 through the classification outputs.

[0112] The system then applies a smooth prediction function 240 to the classification outputs generated from the prediction batch 214 and a threshold τ to obtain a confidence score C θ (x;τ) to generate a confidence score C θ (x;τ) is differentiable with respect to the classification output and the model parameters of the classification model 220 through the threshold τ.

[0113] The system then calculates a predicted confidence score C based on the prediction batch 214. θ Calculate a training loss based on (x; τ). The training loss may include a size loss (or inefficiency loss) Ω and a classification loss L. As shown in the example at 245, the classification loss L may be shaped to implement constraints on the confidence set, for example, by penalizing groups of classes (e.g., vehicles) in the confidence set. Thus, the classification loss L may assign a positive value to the class "truck" when that class is not a ground truth label.

[0114] After the training loss is computed for a batch of predictions, the system backpropagates the gradient of the training loss with respect to the model parameters to update the values of the model parameters.

[0115] 3 shows an example conformal classification system 300 for predicting a confidence set 345 for input data 310. System 300 is an example of a system implemented as a computer program on one or more computers at one or more locations in which the systems, components, and techniques described below may be implemented.

[0116] In general, the system 300 generates a confidence set 345 that includes one or more output classifications selected from a set of classifications such that the probability that the true classification of the model input specified in the input data 310 is included in the confidence set is equal to or greater than a user-specified confidence level 316, e.g., 1-α. That is, the classification accuracy of the confidence set (in the sense that the confidence set contains the true classification) is guaranteed to meet a confidence level that may be defined by the user depending on the particular application.

[0117] System 300 includes a classification model 320 that was conformally trained by a conformal training system, such as system 100 of FIG. 1. That is, classification model 320 was trained end-to-end with a smooth calibration function (which determines the threshold) and a smooth conformal prediction function. Parameters of classification model 320 were optimized to minimize a training loss (e.g., including an inefficiency loss) calculated for a predicted confidence set based on a particular confidence level 1-α′ selected for training.

[0118] The conformally trained classification model 320 receives input data 310 as model input and processes the input data 310 to generate classification output 325. The classification output 325 includes, for example, a predicted probability that the model input will be classified into each of a set of classes. The system then generates a fitness score 326 based on the classification output 325. The fitness score 326 may be defined according to any suitable conformal prediction formulation, such as the THR or APS formulations discussed with reference to FIG. 1.

[0119] In some implementations, the system 300 uses the same conformal prediction formulation (i.e., the same definition of fitness score) as was used to conformally train the classification model 320. However, this is not a necessary requirement. After the classification model 320 is conformally trained, any suitable conformal prediction formulation may be used in the conformal classification system 300. For example, even if the THR formulation was used to conformally train the classification model 320, the system 300 could still use a different conformal prediction formulation, such as the APS formulation, to define the fitness score 326 and predict the confidence set 345 based on the threshold 335.

[0120] The threshold 335 is determined by a calibration process that uses the calibration function 330 on the calibration training data set 312. The calibration process performed by the calibration function 330 is similar to the calibration process described with reference to the smooth calibration function 130 of FIG. 1. However, the calibration function 330 is generally not a smooth function with respect to the input. Rather, the calibration function 330 is generally a non-differentiable function with respect to the input to the function, i.e., the classification output with respect to the calibration set 312. For example, in a THR or APS formulation, the quantile operation would be a regular non-differentiable quantile operation instead of a smoothed quantile operation to calculate the threshold 335 based on the confidence level 316 using the calibration function 330.

[0121] In some implementations, the confidence level 316(1−α) for determining the calibrated threshold 335 coincides with the confidence level 1−α′ used to conformally train the classification model 320. However, this is not strictly required. In some implementations, the confidence level 316(1−α) can be a different value than the confidence level 1−α′ used to conformally train the classification model 320. Thus, once the classification model 320 is conformally trained, various conformal calibrations and predictions using different types of conformal prediction formulations and different confidence levels can be used with the trained classification model 320 to predict the confidence set 345.

[0122] After determining the threshold 335, the system can use the threshold to select classifications to be included in a predicted confidence set 345 based on the predicted probabilities of the classifications manifested by the classification output 325. For example, if the THR formulation is used, the system can determine the confidence set 345 using Equation 2. In another example, if the APS formulation is used, the system can determine the confidence set 345 using Equations 5 and 6.

[0123] 4 is a flow diagram illustrating an exemplary process 400 for performing conformal training of a classification model. For convenience, process 400 is described as being performed by one or more computer systems located at one or more locations. For example, a conformal training system suitably programmed in accordance with this specification, such as the conformal training system 100 described with reference to FIG. 1, may perform process 400.

[0124] In step 410, the system obtains calibration and prediction training examples.

[0125] In some implementations, the system takes a batch of training examples and divides the batch, for example by random sampling, into calibration training examples and prediction training examples, each of which includes a respective training model input and a respective classification label for each of the training model inputs.

[0126] In step 420, the system determines a threshold value based on the calibration training examples. In particular, for each calibration training example, the system processes each training model input of each calibration training example using the classification model according to the current values of the model parameters to generate a respective classification output for each training model input. The system then determines a threshold value based on at least the classification output generated for the calibration training example, the classification label of the calibration training example, and a confidence level.

[0127] In particular, to determine the threshold, the system can process the classification output, the classification labels of the calibration training examples, and the confidence values with a smooth calibration function that outputs a threshold. The smooth calibration function is differentiable with respect to the classification output and, therefore, with respect to the model parameters of the classification model.

[0128] For example, for each calibration training example, the system can calculate a respective fitness score based on the respective classification output and the respective classification label using an appropriate formulation (e.g., a THR or APS formulation), and then perform a smoothed implementation of a quantile operation on the fitness scores and confidence levels to determine a threshold.

[0129] In step 430, the system generates data characterizing the predicted confidence set based on the threshold and the predicted training examples.

[0130] In some implementations, for each predicted training example, the system processes each model input for each predicted training example using the classification model according to the current values of the model parameters to generate a respective classification output. The system then processes the input, including each classification output and a threshold, with a smooth prediction function to generate a respective predicted output. In particular, each predicted output characterizes a predicted confidence set that includes one or more output classifications selected from a set of classifications such that the probability that the true classification of each model input is included in the predicted confidence set is equal to or greater than a confidence level.

[0131] The smooth prediction function is differentiable with respect to each of the inputs to the smooth prediction function, i.e., the classification output and the threshold. Since the classification output and the threshold are each differentiable with respect to the model parameters of the classification model, the predicted output is also differentiable with respect to the model parameters of the classification model.

[0132] In one particular example, to generate the predicted outputs, the system calculates a respective fitness score for each classification output using a formulation selected in a smooth calibration process, and processes the fitness scores and thresholds using a smooth prediction function to generate a confidence score that characterizes the probability that each of a set of classifications is included in the confidence set.

[0133] In step 440, the system updates the model parameters of the classification model based at least on the predicted confidence set.

[0134] In some implementations, the system calculates gradients with respect to the model parameters of a training loss calculated based on at least each predicted output. In particular, the training loss includes an inefficiency loss that measures a value that characterizes the size of a predicted confidence set characterized by the predicted output. For example, the value that characterizes the size of the predicted confidence set may be calculated based on a sum of confidence scores that characterize the probability that each classification is included in the confidence set.

[0135] In some implementations, the training loss further includes a classification loss that measures the classification error of the output classifications contained in the predicted confidence set.

[0136] For example, the classification loss may include a first term that measures the error of not including, for each respective predicted training example, the true classification indicated by the respective classification label of the respective predicted training example in the predicted confidence set characterized by the predicted output.

[0137] The classification loss may further include a second term that measures the error of including a specified classification that is not the true classification in the respective output confidence set for each respective predicted training example. For example, the specified classification may be a classification opposite to the true classification, which is determined as a classification based on a priori information.

[0138] The system can shape the inefficiency loss and / or classification loss to adapt to different application scenarios. For example, the system can implement the inefficiency loss as a weighted size calculated based on the size of each predicted confidence set scaled by a weighting factor for a set of classifications. For example, the system can assign a larger value of the weighting factor to a particular classification that is relatively less important to detect.

[0139] In another example, the system can include one or more additional terms in the classification loss to penalize two or more classifications being included together in the predicted confidence set, which can be useful in scenarios where certain classifications may contradict each other or can easily be confused.

[0140] After the gradients of the training loss with respect to the model parameters are determined, the system can use the gradients to update the current values of the model parameters. The system can use any suitable backpropagation-based machine learning technique to update the parameters of the classification model (e.g., a classification neural network), for example, using an Adam or AdaGrad optimizer. In some implementations, the system can apply stochastic gradient descent to multiple batches of training examples and repeat the above steps for each batch.

[0141] 5 is a flow diagram illustrating an exemplary process for performing conformal classification. For convenience, process 500 is described as being performed by one or more computer systems located at one or more locations. For example, a conformal classification system suitably programmed in accordance with this specification, such as the conformal classification system 300 described with reference to FIG. 3, can perform process 500.

[0142] In step 510, the system obtains model inputs, which may include any type of data to be analyzed, such as images, text, audio signals, sensor measurements, event logs, etc.

[0143] In step 520, the system processes the model inputs using the conformally trained classification model to generate classification outputs. The output data may reveal classification information for the model inputs, such as one or more object categories for objects depicted by the input images, one or more text categories for the input text, one or more status categories for the system, one or more health state categories for a person, etc. In particular, the classification output indicates, for each particular classification in a set of classifications, the predicted probability of the model inputs for a particular classification.

[0144] The classification model was conformally trained by a conformal training system, such as system 100 described in connection with FIG. 1. That is, the classification model was trained end-to-end with a smooth calibration function (which determines the threshold) and a smooth conformal prediction function. The parameters of the classification model were optimized to minimize a training loss (e.g., including an inefficiency loss) calculated for a predicted confidence set based on the particular confidence level 1-α′ selected for training.

[0145] In step 530, the system obtains a threshold value predicted by a non-smooth calibration function according to a confidence level. The threshold value was determined by a calibration process using the calibration function on a calibration training data set. A non-smooth calibration function is generally non-differentiable with respect to the input to the function, i.e., the classification output with respect to the calibration set. For example, in a THR or APS formulation, the quantile operation becomes a regular quantile operation instead of a smoothed quantile operation to calculate a threshold value based on a confidence level using a non-smooth calibration function.

[0146] In step 540, the system generates a confidence set based on the model output and a threshold value. In particular, the system can compare the threshold value to the fitness scores calculated based on the model output and select classifications with corresponding fitness scores that satisfy the conditions on the comparison results (e.g., according to Equation 2 or Equations 5 and 6) to be included in the output confidence set.

[0147] Figure 6 shows the performance comparison between a classification model trained using conformal training (ConfTr) and a baseline classification model without conformal training (Basel.) in predicting the confidence set.

[0148] Conformal training is based on the THRLP formulation with a significance level of α = 0.01. After training, the trained classification model is used with a conformal prediction wrapper that uses either the THR or APS formulation to predict the confidence set.

[0149] Figure 6 summarizes a measure of the inefficiency, i.e., predicted confidence set size, of various methods based on five test datasets. As shown by the data in Figure 6, conformal training (ConfTr) significantly improves the predicted confidence set inefficiency compared to the baseline. Furthermore, the classification loss (+L class ) has generally been shown to further improve inefficiencies.

[0150] Furthermore, Figure 6 shows that even if the conformal predictor formulation used in the conformal prediction wrapper does not match the conformal predictor formulation used in conformal training (e.g., THR or APS vs. THRL), conformal training still provides significant improvement in inefficiency. Thus, once a classification model is conformally trained, it can be used with different conformal prediction wrappers depending on the application to effectively predict confidence sets.

[0151] This specification uses the term "configured" in reference to systems and computer program components. Configuring one or more computer systems to perform a particular operation or action means that the system has installed thereon software, firmware, hardware, or a combination thereof that causes the system to perform the operation or action during operation. Configuring one or more computer programs to perform a particular operation or action means that the one or more programs contain instructions that, when executed by a data processing apparatus, cause the apparatus to perform the operation or action. Embodiments and functional operations of the subject matter described herein can be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware, including the structures disclosed herein and their structural equivalents, or a combination of one or more of these. Embodiments of the subject matter described herein can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by or to control the operation of a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of these. Alternatively or additionally, the program instructions may be encoded onto an artificially generated propagated signal, such as a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0152] The term "data processing apparatus" refers to data processing hardware and encompasses all kinds of apparatuses, devices, and machines for processing data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. An apparatus can also be or further include special-purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). Optionally, in addition to hardware, an apparatus can include code that creates an execution environment for a computer program, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

[0153] A computer program, which may also be referred to or described as a program, software, software application, app, module, software module, script, or code, can be written in any form of programming language, including compiled or interpreted, or declarative or procedural, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but does not necessarily, correspond to a file in a file system. A program can be stored as part of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, a single file dedicated to the program in question, or multiple organized files, e.g., files storing one or more modules, subprograms, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and connected together by a data communications network.

[0154] The term "database" is used broadly herein to refer to any collection of data, which need not be structured in any particular way, or at all, and which may be stored on storage devices in one or more locations. Thus, for example, an index database may contain multiple collections of data, each of which may be organized and accessed in a different way.

[0155] Similarly, the term "engine" is used broadly herein to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Generally, an engine is implemented as one or more software modules or components installed on one or more computers at one or more locations. In some cases, one or more computers are dedicated to a particular engine, and in other cases, multiple engines can be installed and running on the same computer or computers.

[0156] The processes and logic flows described herein may be performed by one or more programmable computers executing one or more computer programs to perform functions by performing operations on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or a combination of special purpose logic circuitry and one or more programmed computers.

[0157] A computer suitable for running a computer program can be based on a general-purpose or special-purpose microprocessor, or both, or any other type of central processing unit. Generally, the central processing unit receives instructions and data from a read-only memory or a random-access memory, or both. The essential elements of a computer are a central processing unit for performing or executing instructions, and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by, or incorporated in, special-purpose logic circuitry. Generally, a computer also includes one or more mass storage devices, e.g., magnetic, magneto-optical, or optical disks, for storing data, or is operatively coupled to receive data from, transfer data to, or both of, the mass storage devices. However, a computer need not have such devices. Additionally, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name just a few.

[0158] Computer-readable media suitable for storing computer program instructions and data include, by way of example, all forms of non-volatile memory, media, and memory devices, including semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices, magnetic disks, e.g., internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0159] To provide for user interaction, embodiments of the subject matter described herein can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user, and a keyboard and pointing device, e.g., a mouse or trackball, through which the user can provide input to the computer. Other types of devices can also be used to provide for user interaction; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback, and input from the user can be received in any form, including acoustic, speech, or tactile input. Additionally, a computer can interact with a user by sending documents to and receiving documents from a device used by the user, e.g., by sending a web page to a web browser on the user's device in response to a request received from the web browser. A computer can also interact with a user by sending text messages or other forms of messages to a personal device, e.g., a smartphone, running a messaging application, and receiving a reply message from the user in return.

[0160] A data processing device for implementing machine learning models may also include dedicated hardware accelerator units, for example for handling the often computationally intensive parts of machine learning training or generation, i.e., inference, workloads.

[0161] The machine learning model can be implemented and deployed using a machine learning framework, for example, the TensorFlow framework, the Microsoft Cognitive Toolkit framework, the Apache Singa framework, or the Apache MXNet framework.

[0162] Embodiments of the subject matter described herein can be implemented in a computing system that includes a back-end component, e.g., a data server, or includes a middleware component, e.g., an application server, or includes a front-end component, e.g., a client computer having a graphical user interface, web browser, or app through which a user can interact with an implementation of the subject matter described herein, or includes any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communications network. Examples of communications networks include local area networks (LANs) and wide area networks (WANs), e.g., the Internet.

[0163] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server sends data, e.g., HTML pages, to a user device for the purpose of, e.g., displaying data to and receiving user input from a user interacting with the device acting as a client. Data generated at the user device, e.g., a result of user interaction, may be received from the device at the server.

[0164] While this specification contains many specific implementation details, these should not be considered limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features that are described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, while features may be described above as working in a particular combination, and may even be initially claimed as such, one or more features of a claimed combination may in some cases be deleted from the combination, and a claimed combination may be directed to a subcombination or a variation of a subcombination.

[0165] Similarly, while operations are illustrated in the figures and claimed in a particular order, this should not be understood as requiring such operations to be performed in the particular order shown or in sequential order, or that all of the operations shown be performed to achieve a desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the division of various system modules and components in the above-described embodiments should not be understood as requiring such division in all embodiments, and it should be understood that the described program components and systems generally can be integrated together in a single software product or packaged in multiple software products.

[0166] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As an example, the processes depicted in the accompanying figures do not necessarily require the particular order shown or sequential order to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. [Explanation of symbols]

[0167] 100 Conformal Training System 110 Training Examples 112 Calibration Training Examples, Calibration Training Example Sets 114 Prediction Training Examples 116 confidence levels 120 Classification Machine Learning Models 130 Smooth Calibration Functions 135 Threshold τ 140 Smooth Prediction Functions 145 Trust Score 150 Parameter Update Engine 200 Conformal Training Process 210a Model Input 212 calibration batches 214 prediction batches 220 Classification Model 225 Classification Output 230 Smooth Calibration Functions 240 Smooth Prediction Functions 300 Conformal Classification System 310 Input Data 312 Calibration Training Dataset 316 confidence levels 320 Classification Model 325 Classification Output 326 Suitability Score 330 Calibration Function 335 Threshold 345 Trusted Set

Claims

1. 1. A computer-implemented method for training a conformal classification machine learning model, comprising: the conformal classification machine learning model: a classification model having a plurality of classification model parameters, the classification model being configured to process the model inputs to generate, for each particular classification in a set of classifications, a classification output indicative of a predicted probability of the model input with respect to the particular classification; and a prediction function configured to process inputs including the classification outputs to generate data identifying a confidence set that includes, for the model input, one or more output classifications selected from the set of classifications such that the probability that the true classification of the model input is included in the confidence set is equal to or greater than a confidence level. Including, The method comprises: obtaining a set of calibration training examples, each calibration training example including a respective training model input and a respective classification label for said respective training model input; for each respective calibration training example in the set of calibration training examples, processing the respective training model inputs of each of the calibration training examples using the classification model according to current values of the classification model parameters to generate a respective classification output for the respective training model input; determining a threshold based on at least the classification output generated for the calibration training examples, the classification labels of the calibration training examples, and the confidence level; obtaining a set of predicted training examples, each predicted training example including a respective training model input and a respective classification label for said respective training model input; For each respective predicted training example in the set of predicted training examples, performing processing the respective training model inputs for each of the predictive training examples using the classification model according to the current values of the classification model parameters to generate respective classification outputs; processing inputs including each of the classification outputs and the threshold with the prediction function to generate a respective predicted output, the prediction function being differentiable with respect to each of the classification outputs; determining, for at least each respective predicted training example, gradients with respect to the classification model parameters of training losses, the training losses including an inefficiency loss measuring a value indicative of the size of a respective output confidence set in the respective predicted output generated for each said predicted training example; updating the current values of the parameters of the classification model parameters using the gradients; A computer-implemented method comprising:

2. the respective prediction outputs include a confidence score for each classification in the set of classifications; The method of claim 1 , wherein each of the confidence scores characterizes a respective probability that each of the classifications is included in the predicted confidence set.

3. The method of claim 1 , wherein the classification model comprises a neural network.

4. The method of claim 1 , wherein the training loss further comprises a classification loss that measures a classification error of the output classifications included in the predicted confidence set characterized by the respective predicted outputs.

5. 5. The method of claim 4, wherein the classification loss includes, for each respective predicted training example, a first term that measures the error of not including the true classification in the predicted confidence set characterized by the predicted output.

6. 5. The method of claim 4, wherein the classification loss includes, for each respective predicted training example, a second term that measures the error of including a specified classification other than the true classification in the respective output confidence set.

7. The method of claim 6 , wherein the assigned classification is determined as a classification opposite to the true classification based on a priori information.

8. 2. The method of claim 1 , wherein the prediction function comprises a smoothed implementation of a threshold function applied to a difference between a first value calculated based on the classification output and a second value calculated based on the threshold.

9. 9. The method of claim 8, wherein for each respective predicted training example in the set of predicted training examples, the first value is calculated as the predicted probability indicated by the respective classification output for a particular classification.

10. 9. The method of claim 8, wherein for each respective predicted training example in the set of predicted training examples, the first value is calculated as a sum of a plurality of predicted probabilities indicated by the respective classification outputs for a plurality of classifications from the set of classifications.

11. The method of claim 8 , wherein the smoothed threshold function is a sigmoid function.

12. determining the threshold value 2. The method of claim 1, comprising processing the classification output from the classification model, the classification labels of the calibration training examples, and confidence values with a calibration function that outputs the threshold, wherein the calibration function is differentiable with respect to the classification output.

13. The calibration function is For each respective calibration training example in the set of calibration training examples, calculating a respective fitness score based on the respective classification output and the respective classification label; performing a smoothed implementation of a quantile operation on the fitness scores and the confidence levels; The method of claim 12 , configured to:

14. obtaining a batch of training examples; randomly sampling the batch of training examples to generate the set of calibration training examples and the set of prediction training examples; The method of claim 1 further comprising:

15. the batch of training examples is a first batch of training examples; The method comprises: obtaining additional batches of training examples; iteratively training the conformal classification machine learning model on each of the additional batches of training examples; 15. The method of claim 14, further comprising:

16. The method of claim 1 , wherein the inefficiency loss is a weighted size measure calculated based on the size of each predicted confidence set scaled by a weighting factor for the set of classifications.

17. the model inputs include sensor data; the classification corresponds to an object category; the model inputs include medical data including physiological measurements of a patient; each of said classifications corresponds to a different diagnosis of said patient; and / or the model inputs include observations of the environment; The method of claim 1 , wherein each of the classifications corresponds to a different state of the environment.

18. 1. A computer-implemented method for performing conformal classification, comprising: obtaining the classification model trained using the method of claim 1; obtaining model inputs; processing the model inputs with the classification model to generate, for each particular classification in a set of classifications, a classification output indicating a predicted probability of the model input with respect to the particular classification; obtaining a threshold value predicted by a non-smooth calibration function according to a confidence value based on the calibration dataset; processing the model outputs and the threshold to generate a confidence set including one or more output classifications selected from the set of classifications; A computer-implemented method comprising:

19. Based at least in part on the confidence set, controlling a robot or vehicle, wherein the model input includes sensor data and the classification corresponds to an object category; controlling an agent to perform a task in an environment, wherein the model inputs include observations of the environment and the classifications each correspond to a different state of the environment; 20. The method of claim 18, further comprising performing one or more of:

20. one or more computers; one or more storage devices storing instructions that, when executed by said one or more computers, cause said one or more computers to perform each of the operations of the method of any one of claims 1 to 19; Including, the system.

21. One or more computer-readable storage media having stored thereon instructions that, when executed by one or more computers, cause the one or more computers to perform each of the operations of the method of any one of claims 1 to 19.

Images