Machine learning fairness with limited protected attributes

Abstract

The two-level model allows for improved fairness of the machine learning model in the event of limited access to categories of information (e.g., protected attributes) in underlying data used to train the model. The first stage is used for training a protected attribute estimation model through active learning. A second stage is used to supplement data that does not contain protected attributes with estimates from the trained protected attribute estimation model, and to allow for training the deviation mitigation model with increased smaller deviations.

Description

[0001]

[0002] This application claims priority to U.S. Provisional Application No. 63 / 507,381, filed June 9, 2023, the contents of which are incorporated herein by reference in their entirety. Background Technology

[0003] Machine learning models rely on large amounts of data to provide predictions. Generally, the provided data is considered a complete or near-complete dataset, where the values ​​for various data points within the dataset are known. This availability of information allows the machine learning model to make predictions. Furthermore, a robust dataset allows for the analysis of the machine learning model to understand whether the trained model is biased towards certain values ​​within the data points. For example, the choice of machine learning model or technique may lead to bias towards a particular population within the dataset, while different choices may lead to bias towards different populations. Once the model or technique is trained, the potential bias can be evaluated based on the dataset. However, this approach assumes that all information about the underlying dataset on which training took place is available.

[0004] Furthermore, the increasing prevalence of machine learning models across all aspects of decision-making has brought fairness in machine learning into sharp focus. For example, decisions such as credit availability, loan approval, hiring, and even criminal sentencing can rely on machine learning. Fairness in machine learning can refer to the process of eliminating or correcting potential biases in machine learning models to prevent or correct possible outcomes caused by any characteristics related to the data. For instance, there may be emerging or inherent biases related to race, ethnicity, gender, sexual orientation, disability, veteran status, or other characteristics that may be relevant to the data. Therefore, ensuring fairness in machine learning models is crucial for promoting the positive impacts of machine learning while mitigating its negative impacts on individuals, entities, and society.

[0005] Additionally, one of the major challenges in implementing and determining whether a machine learning model is fair is the limited availability of certain values ​​or protected attributes (PAs) (such as gender, race, or age) due to privacy, consent, or legal barriers. Because of this scarcity of information, it can hinder the development of fair machine learning algorithms, as it is difficult to control and mitigate bias without the necessary data on protected attributes. Furthermore, many countries and regulations prohibit the collection of such data.

[0006] Despite hurdles regarding access to PAs, existing solutions for mitigating bias and ensuring fairness assume the availability of PAs and incorporate them into machine learning models to avoid bias. Therefore, new frameworks are needed to achieve fairness in resource-constrained situations, such as when limited or unknown information about PAs is available. Summary of the Invention

[0007] The disclosed implementation includes an upstream model that can learn a predictive model for protected attributes (PAs) using limited data, and then synthesizes a new PA for use in the downstream target task. The estimated PA can effectively replace the original PA, thus enabling the development of fair machine learning models that operate on limited access to PAs. Furthermore, selecting high-quality examples and processing them with fairness in mind can help train competitive and fair models from relatively small datasets.

[0008] This paper provides systems, methods, and computer-readable media that enable fairness in machine learning applications while using a limited number of protected attributes. In some examples, a two-stage framework (also known as the first stage (or upstream stage, PA estimation stage, or bias estimation stage) and the second stage (also known as the downstream stage, fairness model generation stage, or downstream fairness stage)) can leverage active learning techniques to predict protected attributes. The framework can use these predictions, based on limited labeled data, during the machine learning training process to achieve fairness in the target task. In some examples, the first stage (upstream) can employ active learning to accurately predict protected attributes using limited labeled data. Additionally, in some examples, the second stage (downstream) can incrementally sample low-bias training examples based on the estimates provided from the first stage for fairness learning. The framework can improve both the predictive performance and fairness aspects on the target task. In some examples, sample selection strategies can be used to sample data from the available dataset. Sample selection strategies can further improve fairness while only accessing a limited number of PAs.

[0009] The disclosed implementation includes a method for correcting algorithmic bias in a machine learning framework. The method may include: obtaining a first dataset containing protected attributes from at least one class of protected attributes; obtaining a second dataset not containing any protected attributes from at least one class of protected attributes; training a protected attribute estimation model using the first dataset, wherein the protected attribute estimation model estimates information associated with at least one protected attribute; and using the protected attribute estimation model to predict at least one protected attribute for data within the second dataset. The method may further include any combination of the following: The protected attribute estimation model is trained iteratively. Each iteration may involve an active learning process for improving the protected attribute estimation model in each iteration. Additional data points containing protected attributes may be added to the first dataset to update the training of the protected attributes. Additional data points may be obtained by annotating points from the second dataset using protected attributes. Additional data points may be selected from a ranked list of data points in the second dataset. The ranking may be based on an empirically defined bias. The protected attribute estimation model can provide a confidence measure associated with the predicted protected attribute. An expanded dataset can be generated by augmenting a second dataset with the protected attributes predicted from the protected attribute estimation model.

[0010] The disclosed implementation includes a method comprising: obtaining an augmented dataset comprising data points having protected attributes predicted from an estimation model; training multiple bias mitigation models based on a subset of the augmented dataset; obtaining a fairness metric and an accuracy metric for each of the multiple bias mitigation models; and selecting a model from the multiple bias mitigation models based on at least one of the fairness metric and the accuracy metric. A subset of the augmented dataset may be generated based on statistics of the protected attributes of the augmented dataset. The statistics may be generated by the estimation model. The statistics may include a confidence metric associated with the predicted protected attributes. A subset of the augmented dataset may be generated based on the range of the confidence metric.

[0011] The disclosed implementation includes a method with a first phase and a second phase. The first phase may include: obtaining a known dataset of the protected attribute (PA) and an unknown dataset of PA; training a bias estimation model based on data points from the known dataset of PA; generating an estimate of the protected attribute based on the trained bias estimation model; and expanding the unknown dataset of PA to generate an expanded dataset. The second phase may include: obtaining the expanded dataset; generating multiple subsets of the expanded dataset; training multiple bias mitigation models based on the multiple subsets of the expanded dataset; and selecting one of the trained bias mitigation models as the final mitigation model. Additional data points can be added to the known dataset of PA through an active learning process. The active learning process can be performed by annotating data points in the unknown dataset of PA using the protected attribute. A ranking entropy model can be used to select additional data points during training in the second phase.

[0012] Implementations may include a system comprising one or more processors coupled to a computer-readable medium, the computer-readable medium including instructions that, when executed by the one or more processors, perform any of the methods described above. Attached Figure Description

[0013] Figure 1A An example dataset with protected attributes is shown.

[0014] Figure 1B This demonstrates a two-stage framework for achieving fairness in machine learning applications.

[0015] Figure 2 An example framework for generating protected attribute estimation models is shown.

[0016] Figure 3 An example framework for generating and selecting one fairness model from multiple fairness models is shown.

[0017] Figure 4 An example method related to the protected attribute estimation model is shown.

[0018] Figure 5 Example methods related to generating fair models are shown.

[0019] Figure 6 , Figure 7 and Figure 8 The performance and fairness aspects related to machine learning models are shown.

[0020] Figure 9 An example computing system according to an embodiment of the present disclosure is shown.

[0021]

[0022] A “server computer” can include a powerful computer or a cluster of computers. For example, a server computer can include a mainframe, a small cluster of computers, or a group of servers operating as a single unit. In one example, a server computer can include a database server coupled to a web server. A server computer can include one or more computing devices and can use any of a variety of computing architectures, arrangements, and compilations to serve requests to one or more “client computers.”

[0023] "Memory" can include any suitable one or more devices capable of storing electronic data. Suitable memory can include non-transient computer-readable media storing instructions executable by a processor to implement the desired method. Examples of memory include one or more memory chips, disk drives, etc. Such memory can be operated using any suitable electrical, optical, and / or magnetic modes of operation.

[0024] "Processor" can include any suitable one or more data computing devices. A processor can include one or more microprocessors that work together to achieve the desired functionality. A processor can include a CPU that includes at least one high-speed data processor sufficient to execute program components for performing user and / or system-generated requests. A CPU can be a microprocessor such as AMD's Athlon, Duron, and / or Opteron; IBM and / or Motorola's PowerPC; IBM and Sony's Cell processors; Intel's Celeron, Itanium, Pentium, Xenon, and / or XScale; and / or similar processors.

[0025] A "uniform distribution" can refer to a probability distribution in which all possible values ​​associated with a random variable are equally likely (e.g., a probability density function or probability mass function). Fair dicing is an example of a system corresponding to a uniform distribution (because any two dices have equal probabilities). The term "non-uniform distribution" can refer to a probability distribution in which all possible values ​​or intervals are not equally likely. The Gaussian distribution is an example of a non-uniform distribution.

[0026] "Classification" can refer to a process by which something (such as a data value, feature vector, etc.) is associated with a specific category of things. For example, an image can be classified as an image of a dog. "Anomaly detection" can refer to a classification process by which things are categorized as normal or abnormal. An "anomaly" can refer to something unusual, rarely observed, or unexpected. For example, in the context of email communication, spam emails might be considered anomalous, while non-spam emails might be considered normal. Machine learning models can be used to perform both classification and anomaly detection.

[0027] The terms "artificial intelligence model" or "machine learning model" can include models that can be used to predict outcomes to achieve predefined goals. Machine learning models can be developed using a learning process, where training data is classified based on known or inferred patterns.

[0028] Machine learning can include artificial intelligence processes in which software applications can be trained to make accurate predictions through learning. Predictions can be generated by applying input data to a predictive model formed based on statistical analysis performed on aggregated data. Training data can be used to train the model so that it can be used to make accurate predictions. Predictions can be, for example, image classification (e.g., identifying images of cats on the internet), or, as another example, recommendations (e.g., movies a user might like or restaurants a consumer might prefer).

[0029] "Machine learning models" can include applications of artificial intelligence that provide systems with the ability to learn automatically and improve based on experience without being explicitly programmed. A machine learning model can include a set of software routines and parameters that can predict the output of a process based on feature vectors or other input data (e.g., identifying a computer network attacker, authenticating a computer, providing appropriate recommendations based on a user's search query, etc.). The structure of the software routines (e.g., the number of subroutines and the relationships between them) and / or the values ​​of the parameters can be determined during training, which can use the actual results of the process being modeled, such as the identification of different categories of input data. Examples of machine learning models include support vector machines (SVMs), models that classify data by establishing gaps or boundaries between inputs of different categories, and neural networks, i.e., a collection of artificial "neurons" that perform functions by activating in response to inputs. Models can include linear regression, logistic regression, convolutional neural networks (CNNs), deep recurrent neural networks (e.g., fully connected recurrent neural networks (RNNs), gated recurrent units (GRUs), long short-term memory (LSTMs)), transformation-based methods (e.g., XLNet, BERT, XLM, RoBERTa), hidden Markov models (HMMs), linear discriminant analysis (LDA), k-means clustering, density-based spatial clustering for noisy applications (DBSCAN), random forest algorithms, adaptive boosting (AdaBoost), extreme gradient boosting (XGBoost), support vector machines (SVMs), or any model described herein. A machine learning model can be trained using "training data" (e.g., to identify patterns in the training data) and then applied when this training is used for its intended purpose.

[0030] You can use "training data" to train a machine learning model (e.g., to identify patterns in the training data), and then apply this training when it is used for its intended purpose.

[0031] A machine learning model can be defined by "model parameters," which can include numerical values ​​that define how the machine learning model performs its function. Training a machine learning model can include an iterative process of determining a set of model parameters that achieve the optimal performance of the model. Supervised learning models can be trained in various ways using a variety of cost / loss functions that define the error from known labels (e.g., least squares and absolute difference from known classifications) and various optimization techniques, such as backpropagation, steepest descent, conjugate gradients, and Newton and quasi-Newton techniques. Detailed Implementation

[0032] The implementation plan includes methods and systems for correcting algorithmic biases in machine learning or artificial intelligence frameworks.

[0033] In some examples, the method may include any combination of the following features: The method may include obtaining a first dataset containing features or information related to at least one protected attribute. The first dataset may be a finite dataset for training purposes and smaller than a second dataset. A second dataset may be obtained, wherein the second dataset does not contain classifications, information, or labels related to the protected attribute. A protected attribute estimation model may be trained using the first dataset. At least one protected attribute may be estimated using data within the second dataset.

[0034] This disclosed technique comprises an upstream- and downstream-based framework that leverages active learning to predict protected attributes from limited labeled data and uses these predictions in the learning process to achieve fairness in the target task. The upstream model employs active learning to make accurate protected attribute predictions using limited labeled data, while the downstream model incrementally samples low-biased training samples based on the upstream estimates for equitable learning.

[0035] The framework can include two phases. The first phase (also known as the upstream phase or PA estimation model generation phase) can include training the protected attribute estimation model. The upstream phase can utilize active learning to generate the protected attribute estimation model. The first phase or upstream phase can include information about... Figure 2 Any step described in the steps.

[0036] The second stage may include generating multiple fairness models using the selected PA estimation model. In some examples, the second stage may be considered a downstream stage. The second stage or downstream stage may include information about... Figure 3 Any step described in the steps.

[0037] The final model selection can be used to analyze data in a production environment while minimizing bias. Therefore, the final selected model is generated using limited or restricted amounts of data that include protected attributes, resulting in a low-bias model.

[0038] Experiments on benchmark datasets demonstrate the effectiveness of the two-stage framework in both prediction performance and fairness.

[0039] I. Fairness in Machine Learning

[0040] It may be necessary to train a machine learning model while ensuring it is unbiased when certain features relevant to the dataset are unavailable. This problem is exacerbated when the size of the data with certain features (e.g., protected attributes) is very small (e.g., not large enough, small compared to the size of the entire dataset, not large enough to produce statistically meaningful training, etc.).

[0041] Describing fairness in the context of machine learning is useful. Fairness generally refers to correcting for biases in decision-making processes (e.g., outputs) implemented using machine learning models. Decisions made by a computer after a machine learning process may be considered unfair if they are based on variables considered sensitive (also called features or attributes), such as race, gender, sex, or religion. Fairness can be viewed from a group perspective (e.g., whether a particular group is not discriminated against) and an individual perspective (e.g., whether similar individuals are treated similarly when sensitive variables are excluded). Sometimes, when used for automated decision-making, ML algorithms can unfairly influence people, such as in legal, academic, or other settings.

[0042] Machine learning processes can become unfair (also known as biased) for a variety of reasons. These reasons can include problems with the underlying training data, such as skewed samples (e.g., compounded bias in the dataset due to initial bias in the dataset), limited features (e.g., features that are less informative for some groups), sample size differences (e.g., sample sizes for some groups may be smaller than sample sizes for the remaining groups or the largest group in the data), and the presence of surrogates (e.g., other features associated with the sensitive features that may lead to unfairness).

[0043] It should be understood that various quantitative expressions of "fairness" are possible. As an example, fairness can be defined based on the proportion of a particular outcome received by the class. For instance, for a binary category (e.g., the color of a toy, such as green or blue), if one color receives less than a certain percentage of positive outcomes than the other, a fairness criterion can be considered violated. The threshold can be a defined percentage of the other group. For example, the threshold can be a fixed percentage (e.g., for a particular category, the proportion of green toys should be at least 80% of the proportion of blue toys). Specific proportions can be based on legislation or other rules. For example, Title VII of the Civil Rights Act of 1964 states that a differential effect exists when the ratio of the probabilities of a particular outcome between two groups is less than 80%. The definition of fairness can be generalized to multiple features within a dataset (e.g., blue, green, purple, red, pink, etc.).

[0044] II. Examples of datasets with protected attributes

[0045] Figure 1 illustrates an example dataset 100. Dataset 100 is a hypothetical dataset for illustrative purposes and describes features associated with a variety of toys. It should be understood that dataset 100 can contain any number of rows (e.g., data points) and columns (e.g., each column is associated with a feature). Dataset 100 describes various features associated with toys identified by IDs 1 through 10 in columns 110. For each toy, the toy may have a name (Toy_Name in column 111), color (column 112), shape (column 113), manufacturer (column 114), material type (Material_Type in column 115), age group (Age_Group in column 116), price (column 117), country of origin (column 118), supply chain method (column 119), and demand (column 120). Demand (which is low, medium, or high) can be the target task or output on which the dataset can be trained.

[0046] Once trained, a machine learning model can classify a new toy into one of three demand categories. Additionally, the machine learning model can provide statistical information associated with the output, such as a confidence metric. For example, the classification could be low, medium, or high demand for the toy. In other examples, the classification can be binary, such as a yes or no output from the model.

[0047] Specific features within dataset 100 can be considered protected features. Protected features may also be referred to as protected attributes (PAs) in this paper. For example, the country of origin of a toy might be considered a protected attribute. Ideally, in a “fair” model, such a protected attribute should have minimal or no predictive effect on the trained machine learning model. As another example, the supply chain approach chosen for a toy could be a protected attribute. Similarly, for example, the supply chain approach might be considered independent of the strength of demand for a particular toy. Therefore, the supply chain approach could also be considered a protected attribute. Whether a specific feature has predictive value regarding the outcome can be considered in its selection as a protected attribute.

[0048] Figure 1 also shows dataset 199. In this example, dataset 199 is derived from dataset 100 by removing elements from dataset 100. Therefore, in dataset 199, the protected attributes are only available for the three data points corresponding to IDs 1, 5, and 6. Because the training data containing the protected attributes is finite, it is difficult to robustly train a machine learning model based on dataset 199 while also ensuring that the predictive performance of the protected attributes is minimized. For example, a finite number of points with protected attributes will lead to inaccurate or unrobust models.

[0049] The fairness of models trained on example datasets 100 and 199 can be checked. Various approaches can be taken to ensure that a trained model is fair. For example, a fairness metric can be measured against the algorithm during its training. If a fairness metric is violated, the objective function may be penalized to better reflect the described fairness criterion. In other words, constraints can be imposed on the training of the objective function to ensure it adheres to a fairness criterion. This can also be called a constraint-based optimization approach. However, constraint-based optimization methods rely on the ability to perform operations among group members and require group membership information for all data instances in the dataset. Therefore, while this approach might work for dataset 100, it is useless for dataset 199 because the dataset is incomplete. Thus, this solution, rooted in computing fairness metrics, is not suitable for fairness problems when considering restrictions on access to protected attributes. These solutions are provided using the two-stage framework discussed in this paper.

[0050] Therefore, as explained further below, a two-stage framework using active learning can be used to ensure fairness, where only a small subset of the data contains the protected attribute. As further explained below, in the first stage of the framework, to overcome the problem of classification with limited data, an estimation model for estimating the protected attribute can be trained. This model can also generate probabilities (e.g., probability measures associated with the estimation) for each estimated protected attribute. In the second stage, data points with probabilities close to 50% (where it is difficult to determine which class a data point belongs to) can be used to train the model. The range of probabilities can be expanded to train additional models, from which one can be selected as the final model (e.g., a model used for future classification in a production environment).

[0051] III. Overview of the Two-Stage Model

[0052] An overview of the two-stage model is provided before the additional discussion of each stage. Figure 1B A framework 150 is shown, comprising an upstream stage 151 (also known as the first stage or the protected attribute (PA) estimation stage) and a downstream stage 152 (also known as the second stage or the fairness model generation stage). Reference Figure 1B The variables discussed are described in more detail below.

[0053] First, referring to upstream level 151, dataset D can be used to train the bias estimation model (specifying the symbol M). be (And also known as the PA estimation model). The dataset D can consist of two subsets—a first subset containing protected attribute information (specified as D). pal The first subset, also known as the PA known dataset, and the second subset, which does not contain information related to the protected attributes (specified as D), are a subset of the PA known dataset and a subset of the PA known dataset. pau This dataset is also known as the PA unknown dataset. The training of the bias estimation model is discussed further in this paper.

[0054] Once trained, the bias estimation model can predict for the unknown dataset (D) of PA. pau The data points within the dataset are protected attributes, and quantitative information about those predictions is also provided (e.g., confidence levels in the predictions, expressed as probabilities). The information output by the bias estimation model can be appended to the dataset without protected attributes and used in downstream stage 152. Once appended, this can be referred to as the augmented dataset (specified by the notation D). pauPL The expanded dataset may also optionally include statistical information related to the predictions obtained from the bias estimation model.

[0055] In the downstream stage 152, the augmented dataset (D) can be used. pauPLMultiple models can be trained. A subset of the augmented dataset can be created through a selection process. Each such subset of the augmented dataset can be used to train a different bias mitigation model. The selection process can be based on, for example, the confidence range of each prediction from a bias estimation model. For example, it can generate... A subset of ... bmi One of the generated "i" bias mitigation models can be selected based on performance criteria (e.g., for use in a production environment). This model can be designated with the symbol M. bm .

[0056] Two-stage models can be advantageous because they allow for bias correction (e.g., by training a bias-mitigated model) even when the underlying protected properties are unknown. Therefore, two-stage models allow for training fair models (e.g., bias-mitigated models) on small, known datasets of PA while maintaining robust performance. Furthermore, two-stage models may not require data from any known dataset of PA (D... pal The information is used to train the bias estimation model.

[0057] IV. Phase One: Protected Attribute (PA) Estimation Model

[0058] See below for reference. Figure 2 To further explain, the protected attribute (PA) estimation model can be trained on a subset of data containing protected attributes. The protected attribute estimation model can also be referred to as an upstream model or a bias estimation model. After training, the protected attribute estimation model can be used with input data that does not contain protected attributes (e.g., a dataset D with unknown PA). pau It takes the data as input and outputs the protected attributes for that data.

[0059] Training the PA estimation model can be accomplished through active learning. Active learning is a type of semi-supervised learning where model performance can be gradually improved. A smaller subset of the training data is initially used for training, while supplementary and more valuable data from the remainder of the dataset is used to supplement the model's training. During the training of the PA estimation model, an attempt is made to minimize the number of queries to the protected attribute. If the iterations of the trained protected attribute model are not robust or do not meet certain performance criteria, supplementary datasets can be requested. In some examples, human annotators can be used to label or provide supplementary data instances to provide additional data.

[0060] Figure 2 A framework 200 that can be used to generate PA estimation models is shown. Figure 2Datasets 210A, 210B, and 210C are shown, along with PA estimation models 220A, 220B, and 220C, respectively corresponding to and trained on each dataset. Active learning steps between datasets are also illustrated. For example, dataset 210B can be created by supplementing data in dataset 210A through an active learning process. Similarly, datasets 210A or 210B can be supplemented to create dataset 210C. Although in Figure 2 Only three datasets and PA estimation models are shown in the paper, but it should be understood that the process of generating datasets is iterative, and any number of datasets can be generated through active learning, and a corresponding number of PA estimation models can be trained.

[0061] Active learning allows algorithms to determine whether to acquire additional information about a dataset based on cost-benefit analysis. In active learning, the algorithm can proactively select a subset of examples to be labeled next from a pool of unlabeled data. Active learning can include stream-based selective sampling, pool-based sampling, and membership query synthesis. In stream-based selective sampling, decisions are made regarding a data instance and whether it is worthwhile to query for that data instance with a label. In pool-based sampling, it is determined which subset of data is most beneficial to insert into the next training set for the active learning cycle. In membership query synthesis, synthetic data can be generated to be included in the dataset. In some instances of active learning, human operators can use the desired output to label the data. Figure 2 The PA estimation model described herein has the expected output being one or more of the protected attributes for which the PA estimation model is being trained.

[0062] Datasets 210A, 210B, and 210C can be subsets of larger datasets that include information about protected attributes. As an example, referring to Figure 1, rows 1, 5, and 6 of dataset 199 can form dataset 210A. As shown in Figure 1, some information about one or more protected attributes (e.g., country of origin and supply chain methodology in columns 118 and 119 of dataset 199) may be missing. As previously explained, this information can be supplemented using active learning techniques. For example, a human operator could provide information about the supply chain methodology in row 1 and about the country of origin in row 5. This information can be included in dataset 210B, and training can be performed on dataset 210B to generate a PA estimation model 220B, which can be more accurate than PA estimation model 220A due to the availability of more data. Additional information can be added to transform dataset 210A or dataset 210B into dataset 230C by including selected examples for which protected attributes are added. For example, row 3 in Table 199, which lacks information related to the protected attribute, can be selected, and protected attribute data can be added to this row to transform this data point into one that can be used to train the PA estimation model. Thus, in this way, the dataset can be actively learned and recursively updated by providing additional data points to further train the PA estimation model.

[0063] PA estimation models 220A, 220B, and 220C can be machine learning models. For example, a PA estimation model can be a machine learning model trained to generate predicted classifications of values ​​for protected attributes. These output classifications can be one or more protected attributes. Additionally, PA estimation models 220A, 220B, and 220C can generate additional statistics along with the generated classifications. For example, measures related to the confidence of the classification can be provided by the PA estimation model. In some examples, additional columns can be appended to the table shown in Figure 1 to include statistical information.

[0064] Statistical performance of the PA estimation model can be generated during model training. This statistical information can include the probability that a specific data point contains the protected attribute. For example, the protected attribute estimation model can be tested on a validation dataset, and the model's confidence on these data instances can be ranked to identify the data points(s) where the model has the lowest confidence. Among the data instances with the lowest confidence, a subset of data (e.g., the last ten data instances) can be selected for annotation using an active learning framework. The training process can be repeated until a threshold for training (e.g., resource budget or model performance metric) is reached.

[0065] During training, the PA estimation model can be used in the second stage, as further described below.

[0066] The following provides an additional implementation scheme for training the PA estimation model. Before discussing the additional implementation scheme, the following notation can be further discussed.

[0067]

[0068] The following method can be used to train the PA estimation model (Mbe). be It is an ML model that acquires data D and produces results PA for each individual with a specific protected attribute PA.

[0069] The steps of the method:

[0070]

[0071] From group D pau Selecting samples can include using a hierarchical query approach. Examples of this approach include... The batch pattern active learning and hierarchical batch pattern active learning techniques described herein. Qu can be computed for each data point, and then the data points can be sorted based on the computed Qu values. This method can incorporate uncertainty and similarity (e.g., measured via Euclidean distance) between data points used for training by using the following equation:

[0072] , in:

[0073] This indicates a subset of data (e.g., group D). pal Specific examples or data points (within);

[0074] measure The similarity between a given example and the example pool; and

[0075] This indicates the uncertainty of the example, and It is used to combine uncertainty and similarity weights.

[0076] Training and selecting the final PA estimation model (M) be Afterwards, the PA estimation model can be used to generate protected attributes for unlabeled data. Statistical information can also be obtained from M. be Output. For example, D can be analyzed. pau Data instances in the dataset are used to generate dataset D. paupl D paupl Each data instance in the dataset can contain statistical information (e.g., uncertainty information, probability information, confidence values). As explained further below, in the second phase, the set D can be... paupl Sampling is performed to generate multiple models.

[0077] Phase 2: Training a Fair Model

[0078] During Phase Two (also known as the downstream phase, Phase Two, or Fair Model Training Phase), multiple machine learning models can be trained. The training of each of these models can be based on an augmented dataset (D). pauPL Different subsets of ) are included. The augmented dataset comprises the predicted PAs from the bias estimation models discussed above, which are then expanded into the PA unknown dataset (D). pau Machine learning models (which may be called fair models and are designated by the symbol M) bmi Each machine learning model in the set (where i is an index) can be measured for accuracy and fairness. One of several fair models can be selected for use in making predictions on the target task. The final selected fair model can be represented as M. bm The final model can be selected based on the requirements derived from the final model.

[0079] The fair model can be trained on an augmented dataset (D). paupl Execute on ) instead of on the PA dataset (D pal Training on either the full dataset (D) or the complete dataset (D) is not recommended, as training on either dataset will propagate any bias present in the data to the fair model. The propagation of bias can be mitigated by using an augmented dataset where statistics about each predicted PA are known.

[0080] A. Summary

[0081] Figure 3 A framework 300 for generating a bias mitigation model from a dataset D is shown. More specifically, the bias mitigation model can be generated from an augmented dataset (D). paupl ) 320 generated. Figure 3 The diagram shows PA estimation model 310 and the PA unknown dataset (D). pau ) 311, an expanded dataset 320, datasets generated from uncertainty sampling 330A, 330B, and 330C, multiple bias mitigation models 340A, 340B, and 340C, and a final bias mitigation model 350. The PA estimation model 310 can be... Figure 2 The bias estimation model described in (M) be Dataset 320 can be an augmented dataset (D). pauPL The expanded dataset may include information output from the PA estimation model 310 and the PA unknown dataset 311.

[0082] Datasets 330A, 330B, and 330C can be selected based on statistical information from dataset 320 (e.g., the uncertainty of predicted data points in the dataset cap). Datasets 330A, 330B, and 330C can be subsets of dataset 320. Bias mitigation models 340A, 340B, and 340C can be models trained on datasets 340A, 340B, and 340C, respectively. Although only three datasets and three corresponding models are shown, any number of datasets can be selected to train "i" potential models. The final bias mitigation model 350 can be selected from one or more models based on criteria such as performance, accuracy, and bias in the model. (About...) Figures 6 to 8 Examples of criteria that can be considered are further described, such as precision (measured by dividing the number of true positives by the total number of positive predictions) and F-score.

[0083] In Framework 300, model generation and training rely on the output of the model estimated from PA. Therefore, at this stage, it is not necessary to use the protected attributes initially known from dataset D (e.g., dataset D with known protected attributes). pal ). Attributes estimated solely from the PA estimation model can be used. Therefore, dataset D pauPL Along with other algorithms (such as the Difference Effect Removal (DIR) algorithm), it can be used for training. The training of each bias mitigation model (340A, 340B, 340C, etc.) can be based on modifications to the training data, rather than modifications to the underlying machine learning model.

[0084] B. Create a subset 320 from the expanded dataset.

[0085] The expanded dataset 320 (D) paupl Each data point in the model contains (i) features that are irrelevant to (e.g., features that are not protected attributes) and (ii) features derived from the bias estimation model (M). be Both the estimated protected attributes and the estimated protected attributes. Each estimated protected attribute output from the bias estimation model can be associated with statistical information. For example, for a binary protected attribute (e.g., circle or square), the statistical information could be the confidence level of the value estimated by the bias estimation model for the protected attribute. Datasets 330A, 330B, and 330C can be obtained from the expanded dataset 320 ( Each dataset can be generated based on statistical properties of the protected attributes, such as how confident the bias estimation model is about the predictions. Dataset 330A can include protected attributes that are more difficult to distinguish between group memberships due to their lower confidence in the predictions. Dataset 330B can include additional data points with higher confidence in their predictions about them. Dataset 330C can even include additional data points with much higher confidence in their predictions about them. In this way, each dataset can be used to train different bias mitigation models. The accuracy of the trained bias mitigation model can be increased by including additional data points. However, fairness may decrease as more biased points are included.

[0086] Confidence can be represented by values ​​from a range (such as between 0 and 1). For binary values, this range (e.g., 0 to 1) allows a value of 0 to mean that the estimated value is 100% confident that it is the first binary value (e.g., a circle), while a value of 1 could mean that the estimated value is 100% confident that it is the second binary value (e.g., a square). Such extreme values ​​imply perfect prediction. The confidence metric might mean that the biased estimation model has 0% confidence in the values ​​it has predicted (e.g., the biased estimation model is completely unsure whether the protected attribute should be round or square). 0 and Between and The confidence measure between 1 and 2 can similarly represent the confidence level for values ​​that are not extreme. For example, for A confidence metric might mean that there is a 75% confidence that the predicted value is the first binary value in the binary set (e.g., a circle).

[0087] The expanded dataset 320 (D) paupl Subsets of the data point in the augmented dataset 320 can be determined based on values ​​falling within the range of a confidence metric. As discussed above, each data point in the augmented dataset 320 will have a confidence metric associated with the protected attribute predicted for that data point. Based on the confidence metric, data points within the augmented dataset can be partitioned into subsets. For example, from the center value (e.g., Iteratively increasing datasets can be used to train bias-mitigation models. For example, the k-th subset may include datasets with... The confidence metric covers all points between q and k, where q is any real number and k is the index. Each subset can be nested within the next subset as k increases. Each subset can be used to train a bias-mitigation model.

[0088] The above is about indexes. The m-th subset will contain all data points from the n-th subset plus additional data points. However, the m-th subset will contain data points from far away... A wider range of confidence measures (corresponding to a 0% confidence level). Since a 0% confidence level implies a completely "neutral" value for the protected attribute, values ​​far from 0% may reflect bias for the predicted protected attribute. Therefore, including such values ​​will reduce the fairness of the resulting bias-mitigation model. However, for robust training, additional data points are needed to improve accuracy. Therefore, each bias-mitigation model trained on different datasets may contain different characteristics in terms of the trade-off between fairness and accuracy.

[0089] In some examples, a subset can be selected by increasing the number of data points within the confidence range rather than increasing the confidence range itself. Additionally, preprocessing steps can be used to ensure that robust samples of the data points are included in the subset.

[0090] C. Training bias mitigation model from dataset

[0091] Datasets 330A, 330B, and 330C can be obtained from the expanded dataset. Created in [the context of the dataset]. For example, each dataset can be based on intervals related to statistics of the generated protected attributes. For example, dataset 330A could include [the following]. The ones associated with the estimated protected attributes are The confidence metric covers all data points. Dataset 330B can include a larger range. For example, dataset 330B can include... The ones associated with it The dataset 330C contains all the data points for its confidence metric. Dataset 330C can contain a much larger range. For example, dataset 330C may already have all the data points in its confidence metric. This includes all data points. In some examples, preprocessing algorithms can be used to process data instances selected for datasets 330A, 330B, and 330C. For example, fairness preprocessing algorithms such as the Difference Effect Removal (DIR) can be used. The Difference Effect Removal (DIR) is a preprocessing technique that edits feature values ​​to increase group fairness while preserving the rank order within the group.

[0092] DIR techniques can reduce or eliminate inherent bias in a set of data to minimize the impact of this bias on algorithms trained using that data. Some DIR algorithms employ optimization methods to transform datasets with the aim of eliminating bias while preserving data utility as much as possible. Some DIR methods are rooted in the legal concept of the effects of bias, making them suitable for real-world applications where legal compliance is critical. In some examples, the same DIR techniques applied to training data can also be applied to generating data (e.g., data to be classified by machine learning models).

[0093] A dataset can be selected in this way to obtain a robust model. This is achieved by selecting training examples with low bias (e.g., close to the mean). (those examples of the probability) and utilize additional data points with more bias (e.g., in) Increasing the number of data points (within a larger range to the left and right) allows for training with a sufficient number of points to achieve good performance, while still limiting the number of data points that might introduce unwanted bias into the model. It can be expected that as the number of data points increases (e.g., as the dataset used for training becomes larger), the accuracy of the trained model will increase, but the bias may decrease.

[0094] Points where the protected attribute estimation model is less reliable may mean that the information or data content does not contribute to the bias because the protected attribute estimation model cannot accurately predict the protected attribute for that particular data instance. As an example, if the confidence measure (e.g., probability) for a particular toy being one of two values ​​is 1 / 2 (e.g., low or high), this means the PA estimation model is unsure which group the toy belongs to. A value of ½ in this example might mean a completely unbiased data point because the model cannot determine which value to assign to the protected attribute for that point.

[0095] The final model 350 can be selected from multiple trained models based on accuracy and fairness requirements. Any other metrics can be chosen as a basis for selecting the final model 350. The final model 350 can be selected for use in a production environment to determine the output for data requiring classification.

[0096] This paper describes additional information about training fair models. Before describing the mathematical problem, the following symbols and variables can be defined.

[0097]

[0098] Method: Training a fairness model

[0099]

[0100]

[0101] By classifying the training data based on the estimated bias from the bias estimation model, low-bias and high-bias estimates can be obtained empirically based on confidence values. Thresholds for what is included in the training dataset can be increased based on parameters such as those mentioned above. For example, in the binary case, the threshold parameter can be input as... The training dataset can be constructed by including additional data instances in each iteration. As discussed above, a fair model can be chosen (the final M selected).bm And then put it into the production environment.

[0102] VI. Example Method

[0103] Based on the implementation of the disclosed technology, the following are example methods related to data selection, active learning, and training of machine learning models (e.g., protected attribute estimation models (in the first stage), classification models (in the second stage)).

[0104] A. Generate a protected attribute estimation model

[0105] about Figure 4 Example method 400 is provided to generate an upstream estimation model. The upstream estimation model may also be referred to as a protected attribute estimation model. Example method 400 may utilize or combine any of the techniques described with respect to the disclosures herein.

[0106] In step 402, a first dataset can be obtained. The first dataset may contain features related to at least one protected attribute. In some examples, the first dataset may be selected from or generated from a larger dataset containing protected attributes. For example, the first dataset may be a known dataset of PA (D) that can be obtained from dataset D. pal This is described above.

[0107] In step 404, a second dataset can be obtained. The second dataset may not contain features, information, labels, or classifications related to at least one protected attribute. In some examples, the second dataset may be larger than the first dataset and represent data that will be classified as part of the target task. For example, the second dataset could be an unknown PA dataset (D) that can be obtained from dataset D. pau ,), as described above.

[0108] In step 406, the first dataset can be used to train a protected attribute estimation model. The protected attribute estimation model can be a machine learning model capable of estimating information or classifications related to at least one protected attribute. In some examples, the protected attribute estimation model can be trained for a single attribute or multiple attributes.

[0109] In step 408, the protected attribute estimation model can be updated. In some examples, a human expert can annotate or provide information about a selected number of examples from the second dataset to allow for the next iteration or version of the protected attribute estimation model. Step 408 can be iterated or repeated as needed. In some examples, annotation can be simulated. For example, to replicate the role of a human expert or human operator in annotating supplementary data, supplementary labels can be exposed to simulate interactive annotation. For example, a subset of the labeled dataset can be used to train the PA estimation model (e.g., 30% of the labeled data). The remainder of the dataset can be considered "unlabeled," even though the PA for the data is known. After training on the subset, the PA estimation model can be used to make predictions about the protected attributes for the remaining data (e.g., the other 70% of the labeled data). For these predictions, the data points with the highest uncertainty can be selected. Instead of having a human expert label or annotate these points (which are known but hidden from the PA estimation model), these data points (along with the known protected attributes) can be provided to further train the PA estimation model by adding those specific instances to the training dataset.

[0110] In some examples, data points from a second dataset (e.g., the PA unknown dataset) can be selected and annotated using human experts to update the first dataset (e.g., the PA known dataset). The updated first dataset can then be used for iterative training. In some examples, data points selected from the second dataset can be included in the first dataset.

[0111] In step 410, at least one protected attribute can be estimated for the data within the second dataset. The estimation can be based on a trained protected attribute estimation model. The estimation (also known as a prediction) made by the protected attribute estimation model can also provide statistical information related to the prediction.

[0112] In some examples, method 400 can be considered an upstream estimation model and can be used in conjunction with a downstream model. In some examples, updates to the protected attribute estimation model can occur iteratively, based on criteria such as resources, budget, number of examples, or availability of human experts. In some examples, the method may further include sorting a selected number of examples from a second dataset by an empirically defined bias.

[0113] One output of method 400 can be a protected attribute model. This model can be generated from a dataset that does not contain any information related to the protected attributes (such as dataset D). pau Use it on ).

[0114] B. Generate a bias mitigation model

[0115] Figure 5 Method 500 is illustrated. Example method 500 can be used to generate multiple fairness models and select one of them as the final model based on fairness, bias, accuracy, or other metrics. Example method 500 can utilize or combine any techniques described in the disclosures herein (including the appendix). The generated fairness models can be considered downstream because they are based on the protected attributes predicted from sampling unlabeled datasets and are generated “downstream” from such unlabeled data. In some examples, the fairness model may be referred to as a bias mitigation model (M). bm Or M bmi ).

[0116] In box 502, a dataset containing the protected attributes predicted using the PA estimation model (e.g., an augmented dataset) can be obtained. In some examples, a dataset containing the predicted protected attributes (e.g., an augmented dataset (D)) is provided. paupl This can be achieved by using the unknown dataset (D) in PA. pau Dataset D is generated using the PA estimation model. paupl

[0117] It can also include statistical information about each predicted PA for each data instance. This information can be used to create the training set, as described below. The statistical information can be obtained from the PA estimation model.

[0118] In box 504, statistics from the dataset can be sampled to obtain a set of training data. Sampling can be performed based on a range associated with a statistical measure of the predicted protected attribute (e.g., the confidence level of the prediction expressed as a probability). For example, a small range (such as 48 to 52% confidence) can be selected at this step. In some examples, preprocessing algorithms can be applied to the training dataset. These can include fairness preprocessing algorithms, such as difference effect removers.

[0119] In box 506, the bias mitigation model can be trained using the training dataset modified by the preprocessing algorithm. The model's statistics and performance information can also be obtained in this step.

[0120] In box 508, fairness and accuracy measures can be obtained for one or more trained models. Accuracy measures can be measures of the precision of predictions made for a target task (e.g., target classification). Fairness measures can include one or more measures of fairness, including measures based on group fairness, group ignorance, demographic parity, disparity impact, equal chance, equal odds, parity of positive predictive values, parity of false positive rates, and parity of negative predictive values.

[0121] In some examples, a confusion matrix can be used to investigate one or more fairness metrics. A confusion matrix can also be used to determine accuracy metrics. Accuracy metrics can correspond to the number of correct predictions / the total number of predictions. A confusion matrix can include matrices representing true positives (the total number of outcomes in which the model correctly predicted the positive class), true negatives (the total number of outcomes in which the model correctly predicted the negative class), false positives (the total number of outcomes in which the model incorrectly predicted the positive class), and false negatives (the total number of outcomes in which the model incorrectly predicted the negative class) for a dataset. Various numbers in the matrix can be compared or evaluated to determine fairness and accuracy metrics.

[0122] Demographic parity can refer to the degree to which a protected attribute influences the outcome. For example, for a pair, the prediction should be statistically independent of the protected attribute. Various measures can be calculated based on demographic parity. For instance, if there are two values ​​(A and B) for a pair, we can define that for any outcome "O", the probability of that outcome should be the same for both pairs. Fairness measures can be calculated as the difference between these probabilities or the ratio between them. For a perfectly fair model, accuracy should be expected to be "1".

[0123] Differential impact might refer to a situation where one group has a higher probability of achieving a certain outcome than another group. In some examples, fairness measures can be a binary function that returns a value of 0 or 1 when the ratio between outcomes for the two groups exceeds a certain ratio. For example, if the probability of an outcome for the first group divided by the probability of the same outcome for the second group exceeds a threshold (e.g., ...), ... If the threshold is met, the fairness metric function can return 1 to indicate the presence of a disparity. If not, the fairness metric function can return 0. The threshold can be adjusted based on the use case.

[0124] Equal opportunity might refer to the true positive rate between two groups. A fairness measure can look at the difference in true positive rates between two (or more) groups. For example, a true positive might be loan repayment. Other variations based on equal opportunity can also form the basis for fairness measures. For example, the false positive rate can be measured between two groups, and the measure can be based on that false positive rate.

[0125] Positive predictive value parity and negative predictive value parity can refer to the situation where two groups have the same positive predictive value or the same negative predictive value. A positive predictive value (e.g., the probability of an event occurring) can be defined in a binary case. Fairness measures can be based on the variation in expected parity values ​​between the two cases.

[0126] The described example fairness metrics and calculation methods can be further modified based on requirements.

[0127] In box 509, an additional model can be generated by iterating through steps 504, 506, and 508. Iteration can be performed by expanding the statistical range to obtain additional data points. Therefore, an additional training dataset can be obtained to train the additional model. Step 509 can be performed when there may not be a sufficient number of examples or data points available based on the selected number, thus obtaining additional examples or data points.

[0128] In some examples, additional data points can be selected based on the criteria described in this paper, such as using a ranking entropy model. In the ranking entropy model, additional data points can be selected for the i-th iteration of the training dataset. Entropy Sort them. This could be a dataset containing both unestimated data attributes and protected attributes estimated according to a PA estimation model. It could also be a pair of high-entropy examples to low-entropy examples. The examples within the dataset are sorted to prune the dataset, selecting the most informative and unpredictable examples from the perspective of the target task. Furthermore, to consider the quality of the examples from the PA estimation model, we can... Select quantity k ed In some examples, the following equation can be used to determine the quantity: . and It has already been defined above. You can choose to ensure that the minimum number of samples is selected.

[0129] In some examples, the output of the protected attribute estimation model (such as the predicted features or groups (e.g., race or gender)) can be fed to fairness preprocessing algorithms, such as difference effect remover algorithms, to modify feature values ​​to improve the groups or feature values.

[0130] In box 509, the training process can be iterated to generate an additional fair model. In some examples, the statistical scope can be extended to include additional data points used for training. For example, as mentioned above... Figure 3 The interpretation can be extended to confidence metrics associated with the protected attribute. The iterative process of training a fair model can continue until a sufficient number of models are generated. In some examples, the iterative process can continue until a model with sufficient performance is determined. In other examples, the iterative process can continue until a threshold for a statistical range is reached (e.g., a percentage point of the confidence metric, the number of data points, or the percentage of data analyzed).

[0131] In box 510, one of the selected final models can be used in production or other environments. The selected model can reduce algorithmic bias or be chosen based on threshold criteria used for bias or fairness. The final model selection can be used in a production environment to analyze data while minimizing bias. The model selection can be determined based on, for example, a trade-off between model accuracy and expected fairness. In some examples, the model can be based on the expected target database or use case.

[0132] As can be understood from the diagram above, the final selected model is generated using a limited or restricted amount of data containing protected attributes, while simultaneously generating a low-bias model.

[0133] VII. Example Experiment Setup and Results

[0134] Referring to the example results will help to better understand the benefits of the disclosed technique. The advantages disclosed below are for illustrative purposes and should not be construed as being limited to the dataset discussed. The following will discuss... Figure 6 and Figure 7 Let me further explain these benefits.

[0135] Examples of experimental results were derived from data from the following two publicly available datasets: (1) the Adult Income dataset and (2) the Credit Card Customer Default (DCCC) dataset. These datasets and related tasks are further described below.

[0136] The first dataset is an adult income dataset. This dataset is derived from US Census data. The objective task for training the machine learning model on this dataset is a binary classification problem to predict whether an individual's annual income exceeds $50,000. The dataset includes features such as gender, race, marital status, and capital gains. For this specific dataset example, gender was selected as a protected attribute when predicting income levels.

[0137] For the DCCC dataset, the task is a binary classification problem: whether a user will default on a credit card payment. The dataset includes 23 attributes, such as age, payment history, bill amount, and previous payment amount. In this dataset, age is considered a protected attribute. For the protected attribute, users under 25 years old are in one group, and users over 25 years old are in another.

[0138] A. Results Overview

[0139] Figure 6 Example results are shown on the Adult Income dataset and the DCC dataset. Figure 6Table 610 is shown below. Table 610 shows models with or without the inclusion of baseline models, differential income removal models, reweighted models, and models generated using the methods described herein (marked as disclosed k). ed And publicly disclosed without k ed The results were obtained from two datasets (Adult Income Dataset and DCCC Dataset) in the case of a proxy.

[0140] As can be seen from Table 610, for both datasets, including surrogates within the studied features leads to higher precision but worse group fairness compared to removing features considered as surrogates, which is discussed further below. Therefore, Table 610 illustrates the correlation between surrogate features and target task classification. Thus, removing surrogate features may improve model fairness. The trends in Table 610 represent the difference effect (DI) fairness measure from baseline to... Difference Effect Removal (DIR) and Reweighting Techniques and Improvements to the published methods. Additionally, complex models such as RF and XGB produce higher accuracy but worse fairness compared to LR models.

[0141] Furthermore, the disclosed technology could lead to [targeting / targeting] and The agent settings improve fairness for both, which is explained further below. These improvements are derived from the estimation of the protected attributes in the first stage (upstream stage). Additionally, the DI metric obtained using DIR and the disclosed technique is better than that obtained using DIR and the original PA. This improvement can be attributed to the ability of the disclosed technique to select a smaller number of high-quality samples for training compared to using the entire original training dataset, indicating the potential of the disclosed technique to enhance the effectiveness of DIR.

[0142] B. Details of the experimental setup

[0143] This paper provides further details regarding the experimental setup related to Table 610. More specifically, Figure 6 The datasets referenced in the text are divided into training datasets, validation datasets, and test datasets. Figure 6 The results shown are for the test dataset. After tuning the hyperparameters of both the first-stage and second-stage models used on the validation data, the tuned models were used, and the reported results are based on the results from the test set.

[0144] For reference Figure 6 In terms of feature selection, two different feature sets can be considered. The first set (called "features with surrogate") The second group (referred to as "those without surrogates") includes attributes such as relationship status and marital status that can be used as proxies for protected attributes (PAs). These surrogate features are combined with regular features, which are dependent variables suitable for predicting downstream tasks. The proxy feature list contains only regular features. For the adult income dataset, the feature list includes regular features such as age, education, capital gains, capital losses, weekly hours, job category, and occupation; and proxy features such as marital status, relationship, and nationality. For the DCCC dataset, regular features include balance limit, payments, bill amount, and payment amount, while the proxy feature is education.

[0145] like Figure 6 As further shown, a “baseline” can be created against which results from the disclosed technical model are compared. The baseline can be developed without incorporating fairness into the model development. Two existing fairness preprocessing techniques, DIR (Difference Effect Remover) and reweighting, can also be considered for measuring their comparison results. For both DIR and reweighting, the entire training data, including the original PA, can be used as input. Conversely, the disclosed technique can utilize the estimated PA and may require only a smaller subset of the training data including the original PA.

[0146] Figure 6 The paper also demonstrates logistic regression (LR), random forest (RF), and XGBoost (XGB) techniques that can be used as part of a classification model for the first stage within an active learning framework. For example, the model can start with a seed dataset (labeled Dl) that may contain 0.1% of the training examples. Through active learning, new examples can be sampled from the remaining pool of instances Du. For example, sampling can continue until it constitutes 10% of the original training set. Additionally, various criteria and hyperparameters can be selected. In some examples, such as regarding... Figure 6 For the examples shown, the hyperparameters can be selected as follows. For LR "balanced" class weights, a maximum of 1000 iterations can be used to specify the solver's convergence and the reciprocal of the regularization strength can be set to 0.1. Penalty values. For RF, the number of trees can be set to 30, the maximum depth to 20, and the Gini criterion used. For XGB, logistic regression can be used as the criterion, and the maximum depth can be set to 6. These hyperparameters are exemplary.

[0147] In some examples, due to class imbalance in the dataset, precision and F1 score can be used as evaluation metrics to report classification performance. To assess fairness, the difference effect ratio (DI) can be used. For example, in... A DI value within a certain range can be considered fair, where 1 is the best possible outcome (e.g., is perfectly fair).

[0148] In some examples, to emphasize fairness estimates, “loose” versions of group fairness measures, such as Equal Opportunity Disparity (EOD) and Equal Opportunity (EOP), can be considered. Lower numbers may be better for both EOD and EOP, where 0 represents perfect fairness. To select the best upstream model, performance can be monitored on the validation set throughout the entire iteration of the upstream model. In some examples, and as explained earlier above, a “final” model can be selected from iterations of the trained model based on accuracy and DI. To achieve the final model (e.g., in the downstream stage), candidate models can be ranked in ascending order of accuracy first, followed by models ranked in descending order. This indicates the extent to which DI deviates from its best possible value.

[0149] C. Additional results from ablation studies

[0150] Figure 7 An ablation study is illustrated. Ablation studies investigate the performance of a machine learning system by removing certain components (e.g., features) to understand the contribution of those components (e.g., features) to the overall system. Figure 7 The figures show the number of data instances (e.g., data points) required to train the second stage (downstream model) using the exponential decay equation for Kd discussed above, with (solid line) and without (dashed line) conditions. Figure 710 shows the study with an agent, while Figure 720 shows the study without an agent. In both Figures 710 and 720, the x-axis (horizontal axis) represents the number of iterations, and the y-axis (vertical axis) represents the number of instances selected. In summary, Figure 7 Examples are shown where, compared to baseline, DIR, and reweighted methods, particularly for models such as RF and XGB, adopting the exponential decay equation for Kd results in a smaller number of choices while achieving similar or better results.

[0151] D. Additional Results

[0152] Figure 8 Additional graphs 810, 820, and 830 are shown, illustrating the improvements in three different machine learning models (logistic regression, random forest, and XGBoost) compared to the baseline measurement. Figure 8 In the charts, for all charts, changes in accuracy and discrepancies have a greater impact than improvements in either, with higher numbers reflecting improvements. For example, from... Figure 8As can be seen, an improvement in the difference effect ratio was observed across all machine learning models used for both datasets. Furthermore, the accuracy change was relatively small in most cases. Therefore, the graphs illustrate that the disclosed technique allows for improvements in the difference effect only for small changes.

[0153] E. Non-limiting overview of advantages

[0154] The advantages of the disclosed technology are summarized below.

[0155] Including proxy features can improve fairness. However, fairness (e.g., group fairness) is reduced during such inclusion. Therefore, the correlation between proxy features and the target task is proposed. Model fairness can be enhanced by removing proxy features. The disclosed technique consistently improves fairness in both settings (with and without proxies). As a concrete example, a substantial improvement in fairness is achieved in the case of an XGB model on an adult income dataset compared to other methods (baseline, DIR, reweighting), while accuracy remains competitive.

[0156] Another advantage is the trend that shows an overall improvement in fairness, measured by difference effects (DI), as the model addresses fairness, from the baseline to DIR and reweighting, and finally to the published technical approach.

[0157] Improvements were achieved using PA estimated from the upstream model, demonstrating the effectiveness of the PA estimation model. Even better results were achieved when the estimated PA was used instead of the original PA for further data preprocessing in the downstream task model. This indicates that the new framework is efficient because it selects fewer but higher-quality examples.

[0158] VIII. Information Privacy

[0159] As described above, one aspect of this technology is the collection and use of publicly available information to protect privacy and remove the influence of discrepancies regarding protected attributes. Therefore, the spirit of this invention is to promote fairness in the model by removing the influence of discrepancies on individuals and groups of individuals through predictions that do not rely on protected attributes and their proxies. The spirit of this invention is to remove any information that causes discrepancies due to proxies (e.g., postal codes) or protected attributes.

[0160] This disclosure recognizes that, in this technology, the use of such personal information data can be beneficial to the user. As an example, personal information data can be used to predict financial instruments or products that may be more interesting or useful to the user. As another example, a specific offer can be provided to the user based on personal information data aligned with the user's interests.

[0161] This disclosure envisions that, in some cases, the collected data may include unique identifiers or personal information that can be used to identify a specific person. Such personal information may include demographic data, location-based data, online identifiers, telephone numbers, email addresses, home addresses, dates of birth, or any other personal information. However, such information is not intended to identify a specific individual, but is generalized and anonymized for use.

[0162] Additionally, this disclosure also envisions implementation schemes in which users can selectively block the use or access to their personal information data. For example, a user may "opt out" of the collection of one or more pieces of personal information during service registration. In other examples, a user may be notified that the information provided is anonymous and not specifically linked to a particular user for the purpose of identifying that user. In still other examples, the information may be deleted after a specified period of time.

[0163] Furthermore, the intent of this disclosure is that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. This risk can be minimized by restricting data collection and deleting data that is no longer needed. Additionally, and where appropriate, user privacy can be protected by deidentifying data, including in certain health-related applications. Deidentification can be facilitated where appropriate by removing identifiers, controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods such as differential privacy.

[0164] This disclosure envisions that entities responsible for collecting, analyzing, disclosing, transmitting, storing, or otherwise using such personal information data will comply with well-established privacy policies and / or privacy practices, including all applicable laws and regulations. Non-restrictive examples of policies and practices that may be used include federal and state laws such as the Health Insurance Portability and Accountability Act (HIPAA) in the United States, the California Consumer Privacy Act (CCPA) in California, or the General Data Protection Regulation (GDPR) applicable in Europe.

[0165] Additionally, the intent of this disclosure is to limit the use of personal information. Therefore, privacy protection and enhancement techniques are described, such as by not using proxies targeting certain protected attributes and by not using protected attributes when making decisions.

[0166] IX. Training of Example Computational Systems and ML Models

[0167] A. Training of machine learning models

[0168] The process of training a machine learning model can involve determining a set of parameters that achieves "best" performance, typically based on the output of a loss or error function. A loss function generally correlates the expected or ideal performance of a machine learning model with its actual performance. For example, if a training dataset exists that includes input data paired with corresponding labels (indicating, for example, whether this input data corresponds to normal or outlier data), the loss or error function can associate or compare the classifications or labels produced by the machine learning model with known labels corresponding to the training dataset.

[0169] If the machine learning model produces labels that are the same as those associated with the training data, the loss function can take a small value (e.g., zero), while if the machine learning model produces labels that are completely different from those corresponding to the training dataset, the loss function can take a larger value (e.g., 1, values ​​greater than 1, infinity, etc.). In some cases, the training process may involve iteratively updating the model parameters during training rounds, batches, epochs, or other appropriate segments of the training process.

[0170] In each round, the computer system can use a loss or error function to evaluate the model's current performance with respect to a specific set of parameters. The system can then use metrics such as gradients and techniques such as stochastic gradient descent to update the model parameters based on the loss or error function. In summary, the system can predict which change to the model parameters will lead to the fastest decrease in the loss function, and then adjust the model parameters based on this prediction. This process can be repeated in subsequent training rounds, epochs, etc.

[0171] The computer system can execute this iterative process until training is complete. In some cases, the training process involves a set number of training rounds or periods. In other cases, training can continue until the model "converges," that is, the model shows little or no further improvement in successive training rounds, or the difference in the error or output of the loss function between successive training rounds approaches zero.

[0172] B. Example Computer System

[0173] Any computer system mentioned in this article can utilize any suitable number of subsystems. Figure 9 Examples of such subsystems in computer system 900 are shown. In some embodiments, the computer system includes a single computer device, wherein the subsystem may be a component of the computer device. In other embodiments, the computer system may include multiple computer devices, each of which is a subsystem and has internal components. The computer system may include desktop and laptop computers, tablet computers, mobile phones, and other mobile devices.

[0174] Figure 9The subsystems shown are interconnected via system bus 75. Additional subsystems are shown, such as printer 74, keyboard 78, storage device 79, monitor 76 (e.g., display screen, such as LED) coupled to display adapter 82, etc. Peripheral devices and I / O devices coupled to input / output (I / O) controller 71 can be connected to the computer system via various components known in the art, such as input / output (I / O) port 77 (e.g., USB, FireWire®). For example, I / O port 77 or external interface 81 (e.g., Ethernet, Wi-Fi, etc.) can be used to connect computer system 700 to a wide area network (e.g., the Internet), a mouse input device, or a scanner. The interconnection via system bus 75 allows central processing unit 406 to communicate with each subsystem and control the execution of multiple instructions from system memory 72 or storage device 79 (e.g., fixed disk, such as hard disk drive or optical disk), as well as the exchange of information between subsystems. System memory 72 and / or storage device 79 may contain computer-readable media. Another subsystem is data collection device 85, such as a camera, microphone, accelerometer, etc. Any data mentioned in this article may be output from one component to another and / or output to the user.

[0175] A computer system may include multiple identical components or subsystems connected together, for example, via an external interface 81, an internal interface, or via a removable storage device that can be attached to and removed from one component to another. In some embodiments, the computer system, subsystem, or apparatus may communicate over a network. In such cases, one computer may be considered a client, and another computer may be considered a server, where each computer may be part of the same computer system. The client and server may each include multiple systems, subsystems, or components. In various embodiments, the method may involve a variety of numbers of clients and / or servers, including at least 4, 20, 50, 40, 200, 500, 1000, or 4000 devices. The method may include a variety of numbers of communication messages between devices, including at least 40, 200, 500, 1000, 4000, 50000, 40000, 50000, or one million communication messages. Such communication may involve at least 1 MB, 10 MB, 100 MB, 1 GB, 10 GB, or 100 GB of data.

[0176] Any computer system mentioned herein can use any suitable number of subsystems. In some embodiments, the computer system includes a single computer device, wherein the subsystem may be a component of the computer device. In other embodiments, the computer system may include multiple computer devices, each of which is a subsystem and has internal components.

[0177] A computer system may include multiple components or subsystems connected together, for example, by external interfaces or internal interfaces. In some embodiments, the computer system, subsystem, or device may communicate via a network. In such cases, one computer may be considered a client, and another computer may be considered a server, where each computer may be part of the same computer system. The client and server may each include multiple systems, subsystems, or components.

[0178] It should be understood that any embodiment of this disclosure may be implemented using hardware (e.g., application-specific integrated circuits or field-programmable gate arrays) and / or computer software in the form of control logic, wherein the general-purpose programmable processor is modular or integrated. As used herein, the processor includes a single-core processor, a multi-core processor on the same integrated chip, or multiple processing units on a single circuit board or networked together. Based on the disclosure and teachings provided herein, those skilled in the art will know and understand other ways and / or methods of implementing embodiments of this disclosure using hardware and combinations of hardware and software.

[0179] Any software component or function described in this application may be implemented as processor-executable software code using any suitable computer language such as Java, C, C++, C#, Objective-C, Swift, or a scripting language such as Perl or Python, employing conventional or object-oriented techniques. This software code may be stored as a series of instructions or commands on a computer-readable medium for storage and / or transmission. Suitable media include random access memory (RAM), read-only memory (ROM), magnetic media (e.g., hard disk drive or floppy disk), or optical media (e.g., optical disc (CD) or digital versatile optical disc (DVD)), flash memory, and so on. The computer-readable medium may be any combination of such storage or transmission devices.

[0180] Such programs can also be encoded and transmitted using carrier signals adapted for transmission via wired, optical, and / or wireless networks conforming to various protocols, including the Internet. Therefore, computer-readable media according to embodiments of this disclosure can be created using data signals encoded with such programs. Computer-readable media encoded with program code can be packaged with compatible devices or provided separately from other devices (e.g., downloaded via the Internet). Any such computer-readable media can reside on or within a single computer product (e.g., a hard disk drive, CD, or an entire computer system) and can exist on or within different computer products within a system or network. A computer system may include a monitor, printer, or other suitable display for providing a user with any of the results mentioned herein.

[0181] Any method described herein can be performed wholly or partially by a computer system including one or more processors that can be configured to perform the steps. Therefore, embodiments relate to computer systems configured to perform steps of any method described herein, possibly having different components that perform corresponding steps or groups of corresponding steps. Although presented as numbered steps, the method steps herein may also be performed simultaneously or in different orders. Furthermore, portions of these steps may be used in conjunction with portions of other steps of other methods. Additionally, all or part of the steps may be optional. Furthermore, any step of any method may be performed using modules, circuits, or other means for performing these steps.

Claims

1. A method for correcting algorithmic bias in a machine learning framework, the method comprising: Obtain a first dataset, which contains protected attributes from at least one class of protected attributes; Obtain a second dataset that does not contain any protected attributes from the at least one class of protected attributes; The first dataset is used to train a protected attribute estimation model, wherein the protected attribute estimation model estimates information related to at least one protected attribute; as well as Use a protected attribute estimation model to predict at least one protected attribute for the data in the second dataset.

2. The method of claim 1, wherein the protected attribute estimation model is trained iteratively.

3. The method of claim 2, wherein each iteration involves an active learning process for improving the protected attribute estimation model in each iteration.

4. The method of claim 2, wherein additional data points of the protected attribute are added to the first dataset to update the training of the protected attribute.

5. The method of claim 4, wherein the additional data points are obtained by annotating points from the second dataset using protected attributes.

6. The method of claim 4, wherein the additional data points are selected from a sorted list of data points in the second dataset.

7. The method of claim 6, wherein the sorting is based on an empirically defined bias.

8. The method of claim 1, wherein the protected attribute estimation model provides a confidence measure associated with the predicted protected attribute.

9. The method of claim 1, wherein the expanded dataset is generated based on expanding the second dataset using the predicted protected attributes from the protected attribute estimation model.

10. A method comprising: Obtain an expanded dataset, which includes data points with protected attributes predicted from the estimation model; Multiple bias mitigation models were trained based on a subset of the augmented dataset; Obtain a fairness metric and an accuracy metric for each of the plurality of bias mitigation models; as well as A model is selected from the plurality of bias mitigation models based on at least one of the fairness metric and the accuracy metric.

11. The method of claim 10, wherein the subset of the expanded dataset is generated based on statistics of the protected attributes of the expanded dataset.

12. The method of claim 11, wherein the statistical information is generated by the estimation model.

13. The method of claim 12, wherein the statistical information is a confidence measure associated with the predicted protected attribute.

14. The method of claim 13, wherein the subset of the augmented dataset is generated based on a range of confidence metrics.

15. The method of claim 10, wherein the subset of the expanded dataset is generated based on statistics of the protected attributes of the expanded dataset.

16. A method comprising: (A) Execute the first phase, which includes: Obtain known datasets and unknown datasets for protected attributes (PAs); A bias estimation model is trained based on data points from the known dataset of the PA; Based on the trained bias estimation model, estimates of the protected attributes are generated; The unknown PA dataset is expanded to generate an expanded dataset; (B) Execute the second phase, which includes: Obtain the expanded dataset; Generate multiple subsets of the expanded dataset; Multiple bias mitigation models are trained based on the multiple subsets of the augmented dataset; and One of the trained bias mitigation models is selected as the final mitigation model.

17. The method of claim 16, wherein additional data points are added to the known dataset of the PA through an active learning process.

18. The method of claim 17, wherein the active learning process is performed by annotating data points in the unknown PA dataset with protected attributes.

19. The method of claim 17, further comprising using a sorting entropy model to select additional data points during training in the second phase.

20. A system comprising one or more processors coupled to a computer-readable medium, the computer-readable medium including instructions that, when executed by the one or more processors, perform the method as claimed in any one of claims 1 to 19.

Images