A method of reducing bias in an artificial intelligent system

Abstract

A method of reducing bias of the machine learning model output in an artificial intelligent system disclosed. The method includes firstly defining sensitive attributes in data of a dataset. Then a bia

Description

The present invention relates to a method of reducing bias in an artificial intelligent (AI) system and relates particularly, but not exclusively, to a system for identifying, measuring and reducing such bias. The rapid advancement of AI technology has enabled decisions to be automated and improved, leading to enhanced efficiency in various industries. However, the potential for algorithmic bias becomes a major concern when automating decisions due to insufficient data or complex decision-making criteria. Bias can lead to unfair outcomes and decrease model accuracy, often resulting in incorrect predictions. To ensure proper decision-making outcomes, effective bias detection, and mitigation are required. Broadly, AI bias mitigation strategies currently employed can be classified into three categories: pre-processing; inprocessing; and post-processing. Tai Feldman provides an indepth insight into these methods and their limitations in his paper "Removing Gender Bias in Deep Learning", where they found combining these three approaches together performed best. Each of these approaches has evolved into their own separate fields. Looking firstly at pre-processing, this approach aims to reduce feature selection bias by including or excluding certain features from the training data. An example of this is the "Optimised preprocessing" by Calmon et al technique that learns a probabilistic transformation thay egits the features and labels in the data with group fairness, individual distortion, and data fidelity constraints and objectives. Turning now to in-processing, this includes techniques such as fairness regularization, which adds a constraint to the model's output. Mingyang Wan et al. did an in-depth analysis of current In-Processing techniques in their paper "In Processing Techniques for Machine Learning Fairness". Their research identified a number of issues with current techniques, namely: 1. Sensitive attributes: These techniques heavily rely on the availability of sensitive attributes for training, which can be harder to process with newer regulations (ex: GDPR). This can reactivate bias from the dataset; 2. Fairness with Multiple Sensitive Attributes: Current techniques struggle to provide multiple-attribute fairness since they can be overly sensitive and result in inaccurate predictions. So, a fair model that is designed solely for women will still likely be unfair to black women. 3. Relationship between fairness and Model performance: Bias reduction in machine learning models is often seen as a "tradeoff". This is because attempting to eliminate bias can restrict the decision-making capabilities of the model, ultimately leading to inferior performance. Looking finally at post-processing, these techniques modify the output of a machine-learning model to reduce bias. They can be especially useful when the original model and data can't be changed, but its predictions can be adjusted. For example, the Equalized Odds Postprocessing presented by Hardt et. al. modifies the classifier's predictions to equalize false positive and false negative rates between different groups. Other methods include Calibration for the odds, which modifies the classifier's scores to satisfy certain fairness metrics. Each of these approaches have strengths and weaknesses and are suitable for different contexts. A key challenge is to develop methods that can effectively combine these approaches to balance accuracy and fairness, work with multiple sensitive attributes, and not rely heavily on the availability of sensitive attributes . Preferred embodiments of the present invention seek to overcome or alleviate the above described disadvantages of the prior art. According to an aspect of the present invention there is provided a method of reducing bias in an artificial intelligent system, comprising the steps: defining sensitive attributes in data of a dataset; defining a bias measurement function to calculate the difference in prediction outcomes between different groups within said data defined by said sensitive attributes by a model defined in an artificial intelligent system; defining a loss function using at least one first hyperparameter; generating counterfactual data for data with sensitive attributes; using a predetermined fairness threshold to determine whether prediction using counterfactual data exceeds said fairness threshold and if so, generate at least one second hyperparameter; and tuning said model of said artificial intelligent system using said first and second hyperparameters within an output of the loss function. The method may further comprise preprocessing said dataset. According to another aspect of the present invention there is provided a method of reducing bias in an artificial intelligent system, comprising: using an in-processing bias mitigation technique to determine a first hyperparameter for an artificial intelligent system; using a post-processing bias mitigation technique to determine a second hyperparameter for said artificial intelligent system; tuning said first and second hyperparameters to reduce bias; and applying said first and second hyperparameters to said artificial intelligent system. The present invention has significant advantages over the prior art. For example, the combined approach which generates the two hyperparameters addresses key limitations in the existing methodologies. In particular, the method of the present invention does not solely rely on the availability of sensitive attributes, making it more adaptable in light of regulations like the European Union's General Data Protection Regulation (GDPR). It effectively handles multiple sensitive attributes, allowing for more comprehensive and nuanced bias mitigation. Furthermore, the trade-off between fairness and model performance is adjustable, providing greater flexibility in optimizing the model based on the application's specific needs . Furthermore, the individual methodologies which generate the two hyperparameters each possess unique characteristics. However, the integration of both methods into a unified framework also yields a distinct and exceptional approach, which is the focus of this patent application. The part of the model that defines the loss function integrates a traditional loss function with a unique method for measuring and aggregating bias during the model's training phase. This unique calculation and aggregation of bias for each sensitive attribute group differentiates our invention from existing methods. Furthermore, the part of the model that defines the loss function employs a unique fairness constraint in the loss function, an innovative feature not found in previous counterfactual fairness concepts. Specifically, the framework imposes a penalty on the model when the bias measurement surpasses a certain threshold. This unique adaptation of counterfactual fairness within the loss function distinguishes our invention from existing methods in the field. However, the combination of these parts of the model enables the mitigation of both overt and subtle forms of bias and tackles bias issues more holistically than either approach alone. The synergy resulting from the combined application is greater than the sum of its parts. Preferred embodiments of the present invention will now be described, by way of example only, and not in any limitative sense with reference to the accompanying drawings in which:- Figure 1 is a flow chart outlining the steps of a first learning model used in the present invention; Figure 2 is a flow chart outlining the steps of a second learning model used in the present invention; and Figure 3 is flow chart showing how the two learning models are integrated together. The method of the present invention comprises two primary machine learning models, namely an Adaptive Bias Mitigation Network (ABMN) and a Counterfactual Impact Assessment (CIA). These models are combined into a single, unified algorithm designed to address fairness in machine learning prediction outcomes . The Adaptive Bias Mitigation Network (ABMN) is an inprocessing part of the combined approach to bias mitigation. It aims to directly minimize bias during the training of the model itself. The model is trained with a custom loss function that combines a traditional loss measure (like cross-entropy for classification tasks) and a bias measure. The bias measure computes and aggregates the amount of bias present in the predictions for each group of sensitive attributes (like gender or race) . In other words, while the model is learning to predict from the data, it is also learning to reduce the amount of bias in these predictions. The strength of this bias reduction is controlled by a hyperparameter X. Higher values of X force the model to prioritize bias minimization over the accuracy, and lower values do the opposite. This gives us a lever to balance the trade-off between fairness and accuracy according to the specific needs of our application. The ABMN model focuses on minimizing the bias in the machine learning model's predictions. The ABMN model is trained to optimize the following loss function: minimize L(K / (X)) +         Z-2,.... Zn)) subject to f(X') € F Where : f(X) is the prediction function, L(Y, f(X)) represents the loss function comparing the true labels Y to the predicted labels f(X), g(Z2,Z2, . . ., Zn) is a function computing a measure of bias for each combination of sensitive attribute values, B(g(Zi,Z2, . . ., Zn) ) is a function aggregating individual group bias measures into a single overall bias measure, X is a hyperparameter controlling the trade-off between accuracy and bias minimization, and F is the set of all feasible prediction functions. The Counterfactual Impact Assessment (CIA) is a postprocessing part of the combined approach to bias mitigation. The goal here is to ensure that changes to sensitive attributes would not significantly affect the model's decisions. This is a form of counterfactual fairness: if we were to "rerun" history with a person's sensitive attributes changed (for example, change their gender or race) , the decisions made by the model should not substantially change. To achieve this, a constraint is used in the loss function that penalizes the model when the bias measure exceeds a certain threshold (s). This threshold represents the maximum allowable difference in model decisions due to changes in sensitive attributes. As with the ABMN, a hyperparameter (A) allows the control of the trade-off between this type of fairness and accuracy. By combining the ABMN and CIA learning models, this addresses bias both directly during training and indirectly via counterfactual analysis. This approach a greater level of flexibility and robustness to bias that surpasses what is achievable with either component alone. The CIA learning model focuses on ensuring that the decisions remain the same when the sensitive attributes are modified. This is represented by the following function: minimize L(K f (X)) subject to Z-)..... Zn)) < e, and f(X) € F Where : f(X) is the prediction function, L(Y, f(X)) represents the loss function comparing the true labels Y to the predicted labels f(X), g (Z2, Z2, . . ., Zn) is a function computing a measure of bias for each combination of sensitive attribute values, B (g (Z2, Z2, . . ., Zn) ) is a function that aggregates individual group bias measures into a single overall bias measure, e is a fairness threshold that the overall bias measure must not exceed, and F is the set of all feasible prediction functions. In the present invention the ABMN and CIA learning models are integrated into a unified machine-learning algorithm. The combined model balances both fairness aspects by introducing two separate hyperparameters, X2 and X2, which independently control the influence of each fairness component: L(Y, JUf) + . Z>-.... ZY) + A2 • B'giZt, Z^..... Z'J) This combined model allows the adjustment of the relative emphasis on minimizing bias (via ABMN) versus ensuring counter!actual fairness (via CIA) . The selection of Ax and A2 values forms an important part of this step, as they must be determined based on the specifics of the data and fairness requirements, allowing us to optimize for both accuracy and fairness. The exact values for these hyperparameters are impacted by the nature of the data itself and can be determined by any suitable optimisation technique including, but not limited to grid search, random search, Bayesian optimization, and gradient-based optimization, among others. Factors that impact the decision as to which optimisation technique is most suitable include, but are not limited to, computational resources, time constraints, and the specific characteristics of the problem and data. Looking initially at figure 1, the first step in the method of the process of the present invention is pre-processing the data to ensure it is suitable for machine learning (labelled step SI) . This includes removing any irrelevant features and normalizing values where appropriate. Sensitive attributes are also identified at this stage in order to enable the calculation of bias measures. Next, the loss function of the ABMN model is defined (step S2) and the ABMN run to determine the degree of bias resulting as the model processes the data. That is, a bias measurement function is defined to calculate the difference in prediction outcomes between different groups within the data, those groups defined by said sensitive attributes. The loss function is thus used in conjunction with any suitable optimisation technique including, but not limited to grid search, random search, Bayesian optimization, and gradient-based optimization, to choose a suitable hyperparameter Ax that balances fairness and accuracy (step S3) using, for example, a grid search technique. Once the hyperparameter Ax has been determined, the AI model is retrained with the chosen Ai (step S4) . With further reference to figure 2, the CIA model is then used to check that the refined model, as retrained by the ABMN function. The counterfactual examples help the model generalize better to unseen situations, while the fairness constraint ensures that the model does not unfairly favour or disadvantage any particular group. Figure 2 illustrates the key steps involved in implementing the CIA approach, starting from the generation of counterfactual examples in step S5. This process generates extra data by taking and data which includes a sensitive attribute and creating an additional data entry for each alternative attribute. For example, if data entries include the sensitive attribute of sex, then all male data entries generate an equivalent female entry and all female data entries generate an equivalent male entry (assuming only male and female sexes are being considered). The retrained model is then rerun using the new counterfactual data to determine whether the predicted outcome is significantly different from the original outcome. The degree of difference in outcome is measured against the predetermined fairness constraint e (step S6) . If the difference exceeds the predetermined fairness constraint c then the model is retrained until the counterfactuals are producing a result that does not exceed the fairness constraint (step S7). After the application of the AMBN and CIA methods separately, as detailed in the preceding sections, the process focusses on the integration of these two methods with an incorporated fairness constraint. The flowchart in Figure 3 provides an illustrative overview of this process. It begins with the retrieval of the trained ABMN model and counterfactuals (step S8). This is then followed by applying the ABMN model to the counterfactuals (at step S9) and subsequently, the fairness constraint is checked (step S10) . If the constraint is not satisfied, the model parameters are adjusted (step Sil) and the process repeats. This iterative process ensures that the ABMN model is fine-tuned not only to maximize accuracy but also to ensure fairness, leading to a robust and equitable model. Having disclosed the working principles of the present invention, we now turn to look at an example of an implementation which relates to use of this model for job interview bias mitigation. The AI in this example is modelling the selection of candidates for interviewing, for example, from a data set of received job applications. The model can be used to assist in that shortlisting process. The first step is data collection and pre-processing, that is, to collect the data relevant to job interviews. A suitable dataset is the hiring data from a company's HR department, which includes candidate profiles, interview scores, and hiring decisions. This data is likely to include various features, for example, education, experience, skills, as well as sensitive attributes like gender, race, and age. During preprocessing, it is important to handle missing data, remove irrelevant features, and normalize numerical values. Sensitive attributes (such as gender, ethnicity and age) are also identified at this stage. It should be noted however that any preprocessing steps should not introduce additional bias into the data. The next stage is to create the model architecture. This utilises, by way of example only, a feed-forward neural network with three hidden layers, each containing 50 units, and using the ReLU activation function. This architecture is implemented using libraries including, but not limited to, PyTorch or TensorFlow. The following is an example using python: class Net(nn.Module): def__init__(self, input_size): super(Net, self).__init__() self.fcl = nn.Linear(input_size, 50) self.fc2 = nn. Linear(50, 50) self.fc3 = nn. Linear(50, 50) self.fc4 = nn.Linear(50, 1) def forward(self, x): x = torch.relu(self.fcl(x)) x = torch.relu(self.fc2(x)) x = torch.relu(self.fc3(x)) x = torch.sigmoid(self.fc4(x)) return x In the generation of bias measure functions, the function g (Zi, Z2, . . ., Zn) is defined to calculate the difference in prediction outcomes between different groups as they are defined by sensitive attributes. An example of an appropriate measure is the difference in mean predicted interview scores for different groups. def calculate_bias(predictions, sensitive_attribute_values): group_0_preds = predictions[sensitive_attribute_values == 0] group_l_preds = predictions[sensitive_attribute_values == 1] mean_pred_group_0 = group_0_preds.mean() mean_pred_group_l = group_l_preds.mean() bias = abs(mean_pred_group_0 - mean_pred_group_l) return bias Next the loss function is defined and for this binary classification task (hired or not hired), the Cross-Entropy Loss and the Adam optimisers are used as the loss function L(Y, f (X)) . criterion = nn.BCELoss() optimizer = optim.Adam(model.parameters()) Moving on now to the CIA part of the model, the counterfactual data generation takes place. For each example in the dataset, a new example is generated with the same features, but with modified sensitive attributes. For example, if there is a male candidate, a counterfactual example is created where all attributes remain the same, except that the candidate's gender is switched to female. These counterfactuals are generated using, for example, the DiCE library for Python. Having generated the counterfactual data, a suitable fairness threshold e must be determined. This threshold is, for example, set to a small value like 0.05, which means that the average disparity in predictions between groups must be no more than 5% to pass the CIA test. Next the hyperparameters and X2 in our combined loss function need to be optimized. This process is performed using, for example, a grid search, trying a range of values (for example, 0.01, 0.1, 1, 10, 100) for each hyperparameter and selecting the combination that provides the best trade-off between accuracy and fairness on a validation dataset. The Scikit-learn library is an example of a library that provides utilities for performing grid search. from sklearn.model_selection import GridSearchCV hyperparameters = { JlambdalJ: [0.01, 0.1, 1, 10, 100], Jlambda2J: [0.01, 0.1, 1, 10, 100] } grid_search = GridSearchCV(model, hyperparameters, cv=5) grid_search.fit(X_train, y_train) print("Best Hyperparameters: ", grid_search.best_params_) The dataset is split to provide training, validation and testing data. An example of a suitable data split is as follows, 70% for training, 15% for validation (used in the tuning of the hyperparameters) , and 15% for testing (used in the final model evaluation process). Once the data has been split, the final step is model training and evaluation. That is to train the model using the combined ABMN-CIA method, and then evaluate its performance and fairness on the test dataset by testing the loss. 5 It will be appreciated by persons skilled in the art that the above embodiments have been described by way of example only and not in any limitative sense, and that various alterations and modifications are possible without departure from the scope of the protection which is defined by the appended claims. For 10 example, other situations where the model of the present invention can be used include, but are not limited to, reviewing court judgements. In this instance, an AI model is created to, as accurately as possible, recreate the results of court cases based on input data. This data is then rerun through the model 15 together with the ABMN and CIA model of the present invention to determine whether bias is present in the result.

Claims

1. A method of reducing bias in an artificial intelligent system, comprising the steps:defining sensitive attributes in data of a dataset;defining a bias measurement function to calculate the difference in prediction outcomes between different groups within said data defined by said sensitive attributes by a model defined in an artificial intelligent system;defining a loss function using at least one first hyperparameter;generating counterfactual data for data with sensitiveattributes;using a predetermined fairness threshold to determine whether prediction using counterfactual data exceeds said fairness threshold and if so, generate at least one second hyperparameter; andtuning said model of said artificial intelligent system using said first and second hyperparameters within an output of the loss function.

2. A method according to claim 1, further comprising preprocessing said dataset.

3. A method of reducing bias in an artificial intelligent system, comprising:using an in-processing bias mitigation technique to determine a first hyperparameter for an artificial intelligent system;using a post-processing bias mitigation technique to determine a second hyperparameter for said artificial intelligent system;tuning said first and second hyperparameters to reduce bias; andapplying said first and second hyperparameters to said artificial intelligent system.

Images