Using machine learning models to suppress biased data

Abstract

A method (500) for suppressing biased data using a machine learning model (300) includes: receiving a cluster training dataset (130) including a known unbiased data population, and training a clustering model (211) to segment the received cluster training dataset into clusters (212) based on data characteristics of the known unbiased data population. Each cluster of the cluster training dataset includes a cluster weight (214). The method also includes: receiving a training dataset (302) for the machine learning model; and generating training dataset weights (218) corresponding to the training dataset for the machine learning model based on the clustering model. The method also includes adjusting each of the training dataset weights to match the corresponding cluster weight, and providing the adjusted training dataset as the unbiased training dataset to the machine learning model (206).

Description

Technical Field

[0001] The present disclosure relates to rejecting biased data using machine learning models. Background Art

[0002] Generally speaking, bias is the tendency of statistical data to overestimate or underestimate parameters. In this regard, collecting data and analyzing data usually include some inherent biases. These biases may be generated by the method of collection and analysis or the entity that performs collection and analysis. For example, the data research designed and performed by humans can cater to specific assumptions, human design constraints (for example, human ability), sampling constraints, etc. By catering to these factors, the data results of the research are likely to include various sampling errors, measurement errors, or more broadly based on the error of the sample that does not represent the research target group (target population). Because computer processing enables technology to collect data and analyze data at a rate incomparable with human activities, data processing technology must similarly overcome the problem of bias. Otherwise, data processing, especially for large blocks of data, may amplify the bias problem and produce results also incomparable with the bias generated by human activities. Summary of the Invention

[0003] One aspect of the present disclosure provides a method for suppressing biased data using a machine learning model. The method includes receiving a cluster training data set at data processing hardware, the cluster training data set including a known unbiased data population. The method also includes training a cluster model by the data processing hardware to segment the received cluster training data set into clusters based on data characteristics of the known unbiased data population. Each cluster of the cluster training data set includes a cluster weight. The method further includes receiving a training data set for the machine learning model at the data processing hardware; and generating a training data set weight corresponding to the training data set for the machine learning model based on the cluster model by the data processing hardware. The method also includes adjusting each of the training data set weights by the data processing hardware to match the corresponding cluster weight and form an adjusted training data set; and providing the adjusted training data set as an unbiased training data set to the machine learning model by the data processing hardware.

[0004] Implementations of the present disclosure may include one or more of the following optional features. In some implementations, the method includes providing the adjusted training data set as an unbiased training data set to the machine learning model and further includes training the machine learning model with the unbiased training data set. The method may include training the machine learning model with the unbiased training data set by data processing hardware or may include receiving a sample data set including at least one corresponding data feature at the data processing hardware. Here, the method may also include generating an unbiased prediction value based on the received sample data set using the trained machine learning model by the data processing hardware.

[0005] In some examples, adjusting each training dataset weight to match the corresponding cluster weight includes: for each training dataset weight, matching the training dataset weight to the corresponding cluster weight based on the common data characteristics; and removing data from the training dataset until the training dataset weight matches the corresponding cluster weight. In other examples, adjusting each training dataset weight to match the corresponding cluster weight includes: for each training dataset weight, matching the training dataset weight to the cluster weight based on the common data characteristics; and copying data from the training dataset until each training dataset weight matches the corresponding cluster weight.

[0006] In some configurations, for each training dataset weight, adjusting each training dataset weight to match the corresponding cluster weight includes matching the training dataset weight to the cluster weight based on the common data characteristics. When the training dataset weight is less than the corresponding cluster weight, the method may include associating an indication to increase the importance weight of training the machine learning model for the training data corresponding to the training dataset weight. Additionally or alternatively, for each training dataset weight, adjusting each training dataset weight to match the corresponding cluster weight may include matching the training dataset weight to the cluster weight based on the common data characteristics. Here, when the training dataset weight is greater than the corresponding cluster weight, the method may include associating an indication to decrease the importance weight of training the machine learning model for the training data corresponding to the training dataset weight.

[0007] In some implementations, adjusting each of the training dataset weights to match the corresponding cluster weight includes, for each training dataset weight, matching the training dataset weight to the corresponding cluster weight based on the common data characteristic. When the training dataset weight is less than the corresponding cluster weight, the method includes associating an instruction to increase an importance weight for training the machine learning model for the training data corresponding to the training dataset weight, and when the training dataset weight is greater than the corresponding cluster weight, the method includes associating an instruction to decrease an importance weight for training the machine learning model for the training data corresponding to the training dataset weight.

[0008] In some examples, when training a clustering model, the method includes segmenting a received cluster training dataset into clusters based on data characteristics of a known unbiased data population. In this example, for each cluster in the clustering based on the data characteristics of the known unbiased data population, the method includes determining a cluster weight for each cluster of the clustering model based on a ratio of the size of the corresponding cluster to the size of the known unbiased data population. In some implementations, an unsupervised machine learning algorithm segments the received cluster training dataset into clusters based on the data characteristics of the known unbiased data population.

[0009] Another aspect of the present disclosure provides a system for suppressing biased data using a machine learning model. The system includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. The operations include receiving a cluster training data set that includes a known unbiased data population. The operations also include training a clustering model to segment the received cluster training data set into clusters based on data characteristics of known unbiased data clusters, each cluster of the cluster training data set including cluster weights. The operations further include receiving a training data set for the machine learning model; and generating training data set weights corresponding to the training data set for the machine learning model based on the clustering model. The operations also include adjusting each of the training data set weights to match the corresponding cluster weights and providing the adjusted training data set to the machine learning model as an unbiased training data set.

[0010] This aspect may include one or more of the following optional features. In some configurations, providing the adjusted training dataset as an unbiased training dataset to the machine learning model includes training the machine learning model with the unbiased training dataset. The operations may also include: training the machine learning model with the unbiased training dataset; receiving a sample dataset including at least one corresponding data feature; and using the machine learning model to generate an unbiased prediction value based on the received sample dataset.

[0011] In some implementations, adjusting each training dataset weight to match the corresponding cluster weight further includes: for each training dataset weight, matching the training dataset weight to the corresponding cluster weight based on the common data characteristics; and removing data from the training dataset until the training dataset weight matches the corresponding cluster weight. In other examples, adjusting each training dataset weight to match the corresponding cluster weight includes: for each training dataset weight, matching the training dataset weight to the cluster weight based on the common data characteristics; and copying data from the training dataset until each training dataset weight matches the corresponding cluster weight.

[0012] In some examples, adjusting each training dataset weight to match the corresponding cluster weight includes, for each training dataset weight, matching the training dataset weight to the cluster weight based on the common data characteristics. In this example, when the corresponding training dataset weight is less than the corresponding cluster weight, the operation includes associating an instruction to increase the importance weight of training the machine learning model for the training data corresponding to the training dataset weight. In other examples, adjusting each training dataset weight to match the corresponding cluster weight may include matching the training dataset weight to the cluster weight based on the common data characteristics. In this example, when the corresponding training dataset weight is greater than the corresponding cluster weight, the operation includes associating an instruction to decrease the importance weight of training the machine learning model for the training data corresponding to the training dataset weight.

[0013] Additionally or alternatively, adjusting each training dataset weight to match the corresponding cluster weight may include, for each training dataset weight, matching the training dataset weight to the corresponding cluster weight based on the common data characteristics. Here, when the corresponding training dataset weight is less than the corresponding cluster weight, the association indicates increasing the importance weight of the training of the machine learning model for the training data corresponding to the training dataset weight, and when the training dataset weight is greater than the corresponding cluster weight, the association indicates decreasing the importance weight of the training of the machine learning model for the training data corresponding to the training dataset weight.

[0014] In some configurations, the operation of training the clustering model may include segmenting the received cluster training dataset into clusters based on data characteristics of a known unbiased data population; and for each cluster in the clusters based on the data characteristics of the known unbiased data population, determining a cluster weight for each cluster of the clustering model based on a ratio of the size of the corresponding cluster to the size of the known unbiased data population. In some examples, the unsupervised machine learning algorithm segments the received cluster training dataset into clusters based on the data characteristics of the known unbiased data population.

[0015] The details of one or more embodiments of the disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a diagram of an example machine learning environment.

[0017] Figure 2A is a schematic diagram of example processing stages for a bias suppression model.

[0018] Figure 2B is Figure 2A Schematic diagram of an example bias suppression model during the training phase.

[0019] Figure 2C is Figure 2A Schematic diagram of an example bias suppression model during the bias-free phase of .

[0020] Figure 2D and 2E is Figure 2A Schematic diagram of example adjustments made by the bias suppression model during the bias-free phase.

[0021] Figure 3 is a diagram of example processing stages of a machine learning model that incorporates unbiased training data from a bias suppression model.

[0022] Figure 4 is a diagram of an example bias scoring model used to generate a bias score for a dataset.

[0023] Figure 5 is a flowchart of an example method for making a machine learning model unbiased within a machine learning environment.

[0024] Figure 6 is a schematic diagram of an example computing device that can be used to implement the systems and methods described herein.

[0025] Like reference numbers in the various drawings indicate like elements. DETAILED DESCRIPTION

[0026] Figure 1is an example of a machine learning environment 10. The machine learning environment 10 generally includes a distributed system 100 having resources 110 accessible via a network 120 (e.g., a remote system such as a cloud environment), a bias suppression model 200, and a machine learning model 300. The resources 110 are accessible to the bias suppression model 200 and / or the machine learning model 300 for training the bias suppression model 200 and / or the machine learning model 300, and for performing the machine learning functions disclosed herein. The distributed system 100 can be any computer processing system having computing resources (e.g., resources 110) capable of executing the bias suppression model 200 and / or the machine learning model 300. In some examples, the bias suppression model 200 and / or the machine learning model 300 is executed on a device that can access the distributed system 100 via the network 120 or otherwise communicate with the distributed system 100. For example, the device can execute a web-based application associated with the distributed system 100.

[0027] Typically, the resources 110 of the distributed system 100 may include hardware resources 110h, 110h 1-i and software resources 110s, 110s 1-i The hardware resources 110 h include data processing hardware 112 and memory hardware 114 . The software resources 110 s may include software applications, software services, application programming interfaces (APIs), etc. The software resources 110 s may reside on the hardware resources 110 h (e.g., stored in the memory hardware 114 ), or may include instructions executed on the data processing hardware 112 .

[0028] Software applications (i.e., software resources 110s) can refer to computer software that enables a computing device to perform tasks. In some examples, software applications can be referred to as "applications," "apps," or "programs." Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.

[0029] Memory hardware 114 is non-transitory memory that can be a physical device used to store programs (e.g., sequences of instructions) or data (e.g., program state information) on a temporary or permanent basis for use by data processing hardware 112. Memory hardware 114 can be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., commonly used for firmware, such as bootloaders), and disk or tape. Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and phase change memory (PCM).

[0030] In the example shown, the bias suppression model 200 is executed in conjunction with the resource 110 to address bias issues to prevent them from adversely affecting the machine learning model 300. In other words, the bias suppression model 200 is configured to prevent the machine learning model 300 from being trained on a machine learning (ML) training dataset 302 that includes biased data by: generating / outputting an unbiased training dataset associated with the ML training dataset 302 for use in training the ML model 300, but with the biased data removed. By preventing the machine learning model 300 from being trained on biased data within the ML training dataset 302, the machine learning model 300 is not affected by the biased data and is therefore able to generate unbiased predictions 310 ( Figure 3 ). Thus, the bias suppression model 200 corresponds to a filter that removes / adjusts biased data within the ML training dataset 302 prior to training the ML model 300 by outputting / generating an unbiased training dataset 206 for use in training the ML model 300.

[0031] Figure 2A The bias suppression model 200 is shown during execution of a first training phase 202 and a second unbiased phase 204 following the first training phase 202. During the training phase 202, the bias suppression model 200 receives the clustered training dataset 130 and outputs cluster weights 214. During the unbiased phase 204, the bias suppression model 200 receives the ML training dataset 302 and uses the cluster weights 214 output from the training phase 202 to output an unbiased training dataset 206 having biased data removed from the ML training dataset 302.

[0032] Here, the term "weight" (e.g., bias cluster weights 214, 214a-n and training data set weights 218, 218a-n) refers to a value, such as a ratio, that is mapped to a unique cluster formed by the clustering process. For a population, each cluster can be associated with a portion of the population, so the value of the portion can be a weight associated with the cluster (e.g., a subset of the population). In other words, by clustering the population into subsets, each subset inherently has characteristics (e.g., weights) specific to the population. More generally, clusters such as bias clusters 212 or training clusters 216 refer to groups of people that can be used to group training data related to those people. Groups of people can include people who share a continuous range of variable values in their training data (e.g., a cluster for Asian females aged 25-27 might include one training example of an Asian female aged 25, another training example of an Asian female aged 26, and other training examples that share this set of values).

[0033] In other implementations, clustering includes people whose training data is clustered by a clustering algorithm (e.g., a clustering model). The clustering algorithm places people into groups that the algorithm considers similar based on having shorter distances between people (or characteristics of people). Grouping by shorter distances can avoid the exponential growth of the number of clusters as the number of variable values in the corresponding group increases. Clustering can be performed based on the variable of interest (e.g., a deviation variable) and / or other variables to determine the distances between training data (e.g., people). For example, clustering is performed based on other variables, but the final determination for clustering the data is based on the variable of interest (e.g., a deviation variable). As an example, the clustering process groups 18-year-old and 19-year-old male Austrians and Germans into a single cluster because it identifies similarities based on a defined metric (e.g., language used, related interests, frequency of being connected in a social network or being a member of the same organization). As another example to illustrate the large number of potential clustering approaches, the clustering process may have four separate groups covering these categories: (1) 18-year-old Austrians; (2) 18-year-old Germans; (3) 19-year-old Austrians; and (4) 19-year-old Germans.

[0034] Further references Figure 2ADuring the training phase 202, the bias suppression model 200 receives a clustered training dataset 130 corresponding to a known unbiased data population. The known unbiased data population can be a target population with an accurate probability distribution for the bias-sensitive variable. By using the known unbiased data population, the bias suppression model 200 avoids using a disproportionate amount of data related to the bias-sensitive variable for training. Bias-sensitive variables 132 are variables that, when over-represented or under-represented in a data sample of the target population, result in an increased likelihood of biased predictions for samples from the target population. In other words, small deviations from an accurate representation of a bias-sensitive variable are likely to produce a skewed predictive analysis. Therefore, when a machine learning model such as the machine learning model 300 is constructed (i.e., trained) without an accurate training dataset for the bias-sensitive variable, the machine learning model can inherently produce biased predictions and biased computational analysis. Some examples of bias-sensitive variables include race, gender, age, nationality, religious affiliation, political affiliation, wealth, etc.

[0035] In some examples, the target population is the complete data set for a given variable or set of variables. Here, the bias suppression model 200 and / or the machine learning model 300 can be trained and / or make predictions corresponding to the target population (e.g., the population corresponding to the cluster training data set 130). As a basic example, the machine learning model 300 can be configured to predict the value of the target population as the population of California. In order to accurately make predictions about the population of California, each model 200, 300 is trained on data associated with the population of California.

[0036] After training the bias suppression model 200 on the received cluster training dataset 130, the bias suppression model 200 is configured to adjust the ML training dataset 302 intended for use in training the ML model 300 during the unbias phase 204. By adjusting the ML training dataset 302 before training the ML model 300, the bias suppression model 200 generates an unbiased training dataset 206 and provides the unbiased training dataset 206 to the ML model 300. In other words, during the unbias phase 204, the bias suppression model 200 converts the ML training dataset 302 (e.g., which may potentially include biased data) into an unbiased training dataset 206 based on the training of the bias suppression model 200 on the cluster training dataset 130 during the training phase 202. In some examples, the bias suppression model 200 is trained with more than one cluster training dataset 130. For example, the bias suppression model 200 is dynamically trained on new or updated cluster training datasets 130 to continuously account for changes in the cluster training datasets 130 over time. The training phase 202 and the unbiasing phase 204 may be performed serially or concurrently, or some combination of the two.

[0037] Figure 2B An example of a bias suppression model 200 during a training phase 202 is shown. Here, the bias suppression model 200 receives a cluster training dataset 130 comprising a known unbiased data population. Although in some implementations, an entity 140, such as an administrator of the bias suppression model 200 or a user with access to the bias suppression model 200 (e.g., a user interested in a particular set of bias features), can define bias features corresponding to bias-sensitive variables. Here, the entity or designer of the bias suppression model 200 does not supply bias features and / or bias-sensitive variables to the bias suppression model 200. Instead, the bias suppression model 200, via the segmenter 210, uses a cluster model 211 to model the cluster training dataset 130 in order to identify biased or unbiased data. In some configurations, the cluster training dataset 130 includes a complete target population dataset. For example, the bias suppression model 200 may receive a complete dataset of U.S. demographic data as the cluster training dataset 130.

[0038] The bias suppression model 200 includes a segmenter 210 and a regulator 220. The segmenter 210 is configured to segment the data set into clusters 212, 212a-n using a clustering model 211 (interchangeably referred to as "clustering model 211"). During the training phase 202, the segmenter 210 trains the clustering model 211 to classify the data set into clusters 212, 212a-n based on the data characteristics of the known unbiased data population (in Figure 2B Shown as "DC a-n”) segments the received cluster training dataset 130 into clusters 212, 212a-n. For simplicity, these data characteristics include at least one corresponding bias-sensitive variable of the target population related to the cluster training dataset 130. In other words, some clusters 212 may be bias clusters associated with at least one corresponding bias-sensitive variable as a data characteristic, while other clusters 212 identify data characteristics that are not related to the bias-sensitive variable. In some embodiments, the cluster model 211 includes a clustering algorithm such that the cluster model 211 is trained during the training phase 202 based on the received Unsupervised learning is performed on the cluster training dataset 130. Unsupervised learning refers to a process in which learning occurs using data that does not include any labels associated with the data (e.g., pre-labeled bias-sensitive variables). By performing unsupervised learning based on the received cluster training dataset 130, the clustering model 211 is trained to identify the probability distribution of an unbiased dataset (via a known unbiased data population) in terms of data characteristics. For example, the clustering model 211 is trained to generate at least one cluster 212 having data characteristics representing bias-sensitive variables and / or combinations of bias-sensitive variables.

[0039] As an example, clustering model 211 clusters each bias-sensitive variable of race, gender, and age into data characteristics of known unbiased groups. Therefore, here, each cluster 212 can correspond to a corresponding bias-sensitive variable combination. As an example, using the data characteristics of race, gender, and age, at least one cluster 212 corresponds to a type of race (e.g., black, white, Hispanic, etc.), a type of gender (e.g., male, female), and a type of age category (e.g., 19-30, 31-44, 45-59, 60+, etc.). When segmenter 210 segments cluster training dataset 130 into clusters 212 using clustering model 211, segmenter 210 is also configured to determine a corresponding cluster weight 214 for each cluster 212, so that clusters 212, 212a-n have associated cluster weights 214, 214a-n. In some examples, cluster weight 214 represents the population portion of cluster 212 with respect to the target population (e.g., the population of cluster training dataset 130). For example, the cluster weights 214 can represent a ratio of the size of the corresponding cluster 212 to the size of the target population of the training dataset 130. In some examples, to determine each cluster weight 214, the segmenter 210 determines the population portion of each cluster 212 and divides each population portion by the maximum population portion of all clusters 212 (e.g., each cluster weight 214 is less than 1). In other examples, to determine each cluster weight 214, the segmenter 210 determines the population portion of each cluster 212 and divides each population portion by the lowest population portion of all clusters 212 (e.g., each cluster weight 214 is greater than 1).

[0040] In some configurations, the segmenter 210 communicates the cluster weights 214 for the clusters 212 to the adjuster 220 during the training phase 202. For example, the adjuster 220 includes a data store 222 of the cluster weights 214. In other examples, the segmenter 210 stores the cluster weights 214 (e.g., in a data store of the segmenter 210) for access by the adjuster 220 during the unbiased phase 204.

[0041] Figure 2C An example of the bias suppression model 200 during the unbiased phase 204 is illustrated. During the unbiased phase 204, the bias suppression model 200 receives an ML training dataset 302 intended for use in training the ML model 300. For example, the training dataset 302 may include an original training dataset that may be potentially biased (e.g., including biased data). In some implementations, the training dataset 302 is a sample of a target population and, therefore, may not accurately reflect the bias-sensitive variables 132 of the target population. For example, the target population may have a racial composition of 25% white, while the training dataset 302 may indicate a sampled racial composition of 45% white. Therefore, to prevent the ML model 300 from being trained on the ML training data 302 that inaccurately reflects the bias-sensitive variables of the target population, the bias suppression model 200 seeks to adjust for this bias (e.g., a 20% difference) using the segmenter 210 and the adjuster 220 during the unbiased phase 204.

[0042] Similar to how the segmenter 210 Figure 2BAfter segmenting the biased training dataset 130 into biased clusters 212 during the training phase 202, the segmenter 210 is configured to segment the received ML training dataset 302 into training clusters 216 during the unbiased phase 204. The segmenter 210 segments the training dataset 302 by providing the training dataset 302 to a trained clustering model 211. Based on the training from the training phase 202, the clustering model 211 has learned how to segment a dataset such as the training dataset 302 into clusters (e.g., clusters 212a-n or training clusters 216a-n). During the unbiased phase 204, the clustering model 211 generates training clusters 216, 216a-n based on the received training dataset 302 intended for use in the machine learning model 300. Here, at least one training cluster 216 is associated with at least one corresponding bias-sensitive variable of the target population. The segmenter 210 is also configured to generate a corresponding training dataset weight 218, 218a-n for each segmented training cluster 216, such that the segmented training cluster 216 includes an associated training dataset weight 218. In some examples, the corresponding training dataset weight 218 represents the population portion of the training cluster 216 with respect to the sample population associated with the training dataset 302. For example, the training dataset weight 218 can represent a ratio of the size of the corresponding training cluster 216 to the size of the sample population of the training dataset 302. In some examples, to determine each training dataset weight 218, the segmenter 210 determines the population portion of each training cluster 216 and divides each population portion by the maximum population portion of the training cluster 216 (e.g., each training dataset weight 218 is less than 1). In other examples, to determine each training dataset weight 218, the segmenter 210 determines the population portion of each training cluster 216 and divides each population portion by the lowest population portion of the training clusters 216a-n (e.g., each training dataset weight 218 is greater than 1).

[0043] The adjuster 220 is configured to adjust the training dataset weights 218 to match the probability distribution of the data characteristics of the target population (i.e., the bias-sensitive variables). In some implementations, the adjuster 220 performs a process 226 that adjusts the training dataset weights 218 by comparing the training dataset weights 218 with the cluster weights 214. For example, Figures 2C-2EThe adjuster 220 is depicted executing a process 226 to retrieve and compare the cluster weights 218 from the cluster weight data store 222 and the training dataset weights 218 from the training weight data store 224, and based on the comparison, adjust the training dataset weights 218. For example, based on a difference in the comparison between the corresponding training dataset weights 218 and the cluster weights 214, the adjuster 220 can adjust the corresponding training dataset weights 218 to match the corresponding cluster weights 214. Thus, the process 226 executed by the adjuster 220 for adjusting the training dataset weights 218 generates / outputs the adjusted training dataset weights, or more generally, the adjusted training dataset 208, that form the unbiased training dataset 206 for training the ML model 300.

[0044] In some implementations, the adjuster 220 performs the process 226 by first matching one or more training dataset weights 218a-n with one or more cluster weights 214a-n based on matching data characteristics, such as bias-sensitive variables. For example, when the training dataset weights 218 and the cluster weights 214 each share a common data characteristic (i.e., a bias-sensitive variable) or a combination of data characteristics, the adjuster 220 can adjust the training dataset weights 218 with the matching (i.e., corresponding) cluster weights 214 to output corresponding adjusted training dataset weights and / or an adjusted training dataset 208.

[0045] refer to Figure 2D, adjuster 220 compares the training dataset weights 218 and bias cluster weights 214 for combinations that share a common data feature (e.g., a bias-sensitive variable) or data characteristics. When the ML training dataset 302 over-represents the bias-sensitive variable, the training dataset weight 218 exceeds (e.g., is greater than) the cluster weight 214 for the data characteristic corresponding to the bias-sensitive variable (e.g., the training dataset 302 indicates a racial composition of more than 20% white). In response to such over-representation, process 226 performed by adjuster 220 may correspond to a data removal adjustment process that adjusts the training dataset weights 218 by removing data from the training dataset 302 until the training dataset weights 218 match the cluster weights 214. On the other hand, when the training dataset 302 under-represents the bias-sensitive variable, the training dataset weights 218 are less than the cluster weights 214 for the data characteristic corresponding to the bias-sensitive variable (e.g., the training dataset 302 indicates a racial composition of less than 20% black). In response to this underrepresentation, the process 226 executed on the adjuster 220 can correspond to a data copying process that adjusts the training dataset weights 218 by copying data from the training dataset 302 until the training dataset weights 218 match the cluster weights 214. In some implementations, the adjuster 220 randomly copies or removes data from the training dataset 302 to maintain the integrity of the training dataset 302. This can avoid further bias associated with non-random selective copying or removal.

[0046] same Figure 2C In contrast to the process 226 of removing / adding data to / from the training dataset 302 until the training dataset weights 218 match the bias cluster weights 214, Figure 2E 2. The process 226 of adjusting the importance weight 228 associated with each training dataset weight 218 is shown as being performed on the adjuster 220. Specifically, the process associates the importance weight 228 with the data of the training dataset 302 corresponding to the associated training dataset weight 218. The importance weight 228 is provided to the training phase 304 ( Figure 3 ) provides an indication to understand how much weight to give to the underlying data corresponding to the training dataset weight 218 while training the machine learning model 300. In some examples, when the training dataset weight 218 is greater than the cluster weight 214, the adjuster 220 associates an indication to decrease the importance weight 228 of the training of the machine learning model 300 for the training data corresponding to the corresponding training dataset weight 218. In other examples, when the training dataset weight 218 is less than the cluster weight 214, the adjuster 220 associates an indication to increase the importance weight 228 of the training of the machine learning model 300 for the training data corresponding to the corresponding training dataset weight 218.

[0047] As Figures 2A-2E As shown, the bias suppression model 200 generates an unbiased training dataset 206 for training the machine learning model 300. Figure 3 300 is an example of a machine learning model 300 trained on an unbiased training dataset 206. A machine learning model, such as the machine learning model 300, is typically taught (or trained) based on a dataset and a result set to predict its own output based on input data similar to the dataset. In some implementations, much like the bias suppression model 200, the machine learning model 300 first undergoes training during a training phase 304 and then undergoes a prediction phase (e.g., inference) 306, which receives a sample dataset 308 as input and outputs unbiased predicted values 310. During the prediction phase 306, the machine learning model 300 receives a sample dataset 308, such as a sample dataset that includes at least one bias-sensitive variable, and utilizes associated machine learning functionality trained on the unbiased training dataset 206 to generate unbiased predicted values 310 based on the received sample dataset 308.

[0048] In some examples, the machine learning model 300 is trained with more than one unbiased training dataset 206. For example, the machine learning model 300 is trained dynamically during operation to continuously learn from a dynamically changing dataset. In other words, the training 304 and prediction 306 phases can operate continuously or simultaneously, or some combination of the two.

[0049] Figure 4 is an example of a bias scoring model 400. The bias scoring model 400 can be used in conjunction with the bias suppression model 200 or separately from the bias suppression model 200. For example, the bias scoring model 400 can evaluate a training dataset 302 intended to train the machine learning model 300 before providing the training dataset 302 to the bias suppression model 200 (i.e., these models 200, 300 negotiate with the bias scoring model 400). In these examples, when the bias scoring model 400 suppresses the training dataset 302, indicating that the training dataset 302 is too biased to continue training the machine learning model 300, the bias scoring model 400 can communicate the suppressed training dataset 302 to the bias suppression model 200 to unbias the suppressed training dataset 426 and form the unbiased training dataset 206, as described above with reference to FIG. Figures 2A-2E described.

[0050] Similar to the bias suppression model 200, cluster model 211, and / or machine learning model 300, the bias scoring model 400 undergoes a training phase 402 to train the bias scoring model 400 to score a dataset, and once trained, the dataset is scored in a scoring phase 404 based on the training from the training phase 402. During the training phase 402, the bias scoring model 400 receives one or more bias scoring training datasets 410. Each bias scoring training dataset 410 includes data such as bias data 412 and / or unbiased data 414, as well as a bias score 416. For example, the bias score 416 is a numerical representation of the bias in the dataset. In some examples, the bias score 416 and / or the bias scoring training dataset 410 originates from a rater 140. The rater 140 can be an administrator within the machine learning environment 10 (e.g., an administrator of the models 200, 211, 300) or a user concerned about bias on the machine learning model 300. In some examples, the rater 140 is one or more entities / sources (i.e., a committee) or another machine learning model that has been trained to compile and / or score a dataset. During the training phase 402 , the bias scoring model 400 receives one or more bias scoring training datasets 410 and learns to generate bias scores 416 for the datasets.

[0051] Once trained, or in the case where the bias scoring model 400 is trained continuously in parallel with the scoring stage 404, the bias scoring model 400 receives (e.g., intercepts) a training dataset 302 intended for use with the machine learning model 300. Based on this training, the bias scoring model 400 performs a scoring process 420, in which the bias scoring model 400 generates a bias score 416 for the training dataset 302. As part of the scoring process 420, the bias scoring model 400 determines whether the bias score 416 for the training dataset 302 meets a score threshold 422. Here, the score threshold 422 indicates the confidence level that the dataset is unbiased or has negligible bias for the purpose of making predictions at the machine learning model 300. For example, the score threshold 422 is an acceptable bias score value.

[0052] When the bias score 416 of the training dataset 302 satisfies the score threshold 422 (e.g., exceeds an acceptable bias score value), the bias scoring model 400 approves the training data 302 as an approved training dataset 424. In some examples, the approved training dataset 424 includes an approval indicator that the machine learning model 300 can recognize, causing the machine learning model to proceed to generate unbiased prediction values 310 (e.g., Figure 3). When the bias score 416 of the training dataset 302 fails to meet the score threshold 422 (e.g., is less than the acceptable bias score value), the bias scoring model 400 suppresses the training dataset 302. The suppressed training dataset 426 may include a suppression indicator to inform the machine learning model 300 not to use the suppressed training dataset 302 for training. As shown by the dashed box and arrow in Figure 4 As indicated in , the bias scoring model 400 can communicate (i.e., provide) the suppressed training dataset 302 to the bias suppression model 200 so that the bias suppression model 200 can convert the suppressed training dataset 302 into an unbiased training dataset 206, as described above with references 2A-2E.

[0053] The bias suppression model 200 machine learning model 300 and / or the bias scoring model 400 can be any type of machine learning model (e.g., supervised, unsupervised, reinforcement, ensemble / decision tree, deep learning, neural network, recursive, linear, etc.) that employs at least one machine learning algorithm to perform the functionality of any of the models 200, 300, 400 described herein. Broadly speaking, the machine learning algorithm may involve supervised learning, unsupervised learning, active learning, or some hybrid combination of these types of learning algorithms. Some specific examples of these broad algorithms include machine learning algorithms such as linear regression algorithms, logistic regression algorithms, decision tree-based algorithms, support vector machine algorithms, naive Bayes classifiers, k-nearest neighbor algorithms, dimensionality reduction algorithms, gradient boosting algorithms, and the like.

[0054] Figure 5 1 is an example method 500 that includes operations for unbiasing a machine learning model 300 within a machine learning environment 10. At operation 502, the method 500 receives a cluster training dataset 130. The cluster training dataset 130 includes a known unbiased data population. At operation 504, the method 500 trains a cluster model 211 to segment the received cluster training dataset 130 into clusters 212 based on data characteristics of the known unbiased data population. Each cluster 212 of the clusters 212a-n includes a cluster weight 214. At operation 506, the method 500 receives a training dataset 302 for the machine learning model 300. At operation 508, the method 500 generates training dataset weights 218a-n corresponding to the training dataset 302 for the machine learning model 300 based on the cluster model 211. At operation 510, the method 500 adjusts each training dataset weight 218 of the training dataset weights 218a-n to match the corresponding cluster weight 214. At operation 512 , the method 500 provides the adjusted training dataset 208 to the machine learning model 300 as the unbiased training dataset 206 .

[0055] Figure 6is a schematic diagram of an example computing device 600 that can be used to implement the systems and methods described in this document (e.g., bias suppression model 200 and / or machine learning model 300). Computing device 600 is intended to represent various forms of digital computers, such as laptops, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown herein, their connections and relationships, and their functions are illustrative only and are not meant to limit the embodiments of the inventions described and / or claimed in this document.

[0056] Computing device 600 includes a processor 610, memory 620, storage device 630, a high-speed interface / controller 640 connected to memory 620 and a high-speed expansion port 650, and a low-speed interface / controller 660 connected to a low-speed bus 670 and storage device 630. Each of components 610, 620, 630, 640, 650, and 660 is interconnected using various buses and can be mounted on a common motherboard or in other suitable ways. Processor 610 is capable of processing instructions for execution within computing device 600, including instructions stored in memory 620 or on storage device 630, to display graphical information for a graphical user interface (GUI) on an external input / output device such as a display 680 coupled to high-speed interface 640. In other embodiments, multiple processors and / or multiple buses, as well as multiple memories and memory types, can be used as appropriate. Furthermore, multiple computing devices 600 can be connected, with each device providing a portion of the necessary operations (e.g., as a server group, a blade server group, or a multi-processor system).

[0057] The memory 620 stores information non-transitorily within the computing device 600. The memory 620 can be a computer-readable medium, a volatile memory unit, or a non-volatile memory unit. The non-transitory memory 620 can be a physical device used to store programs (e.g., instruction sequences) or data (e.g., program state information) on a temporary or permanent basis for use by the computing device 600. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., commonly used for firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.

[0058] The storage device 630 can provide mass storage for the computing device 600. In some embodiments, the storage device 630 is a computer-readable medium. In various embodiments, the storage device 630 can be a floppy disk device, a hard disk device, an optical disk device, or a magnetic tape device, a flash memory device, or other similar solid-state storage device, or an array of devices, including devices in a storage area network or other configuration. In another embodiment, a computer program product is tangibly embodied as an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as the memory 620, the storage device 630, or a memory on the processor 610.

[0059] The high-speed controller 640 manages bandwidth-intensive operations of the computing device 600, while the low-speed controller 660 manages less bandwidth-intensive operations. This division of responsibilities is illustrative only. In some embodiments, the high-speed controller 640 is coupled to the memory 620, the display 680 (e.g., via a graphics processor or accelerator), and the high-speed expansion ports 650 that can accept various expansion cards (not shown). In some embodiments, the low-speed controller 660 is coupled to the storage device 630 and the low-speed expansion ports 690. The low-speed expansion ports 690, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input / output devices, such as a keyboard, pointing device, scanner, or networking equipment, such as a switch or router, for example, via a network adapter.

[0060] As shown, computing device 600 can be implemented in many different forms. For example, it can be implemented as a standard server 600a or multiple times in a group of such servers 600a, as a laptop computer 600b, or as part of a rack server system 600c.

[0061] Various embodiments of the systems and techniques described herein can be implemented in digital electronic and / or optical circuitry, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be special purpose or general purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to these devices.

[0062] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages and / or in assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus, and / or device (e.g., a disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0063] The processes and logic flows described in this specification can be performed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data and generating outputs. The processes and logic flows can also be performed by dedicated logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits). For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include one or more large-capacity storage devices for storing data or be operably coupled to a large-capacity storage device to receive data therefrom or transmit data thereto, or both, such as a magnetic disk, a magneto-optical disk, or an optical disk. However, a computer does not necessarily have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and storage devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0064] To provide for interaction with a user, one or more aspects of the present disclosure can be implemented on a computer having a display device, such as a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or a touch screen for displaying information to the user, and an optional keyboard and pointing device, such as a mouse and trackball, through which the user can provide input to the computer. Other types of devices can also be used to provide for interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including sound, voice, or tactile input. In addition, the computer can interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on the user's client device in response to a request received from the web browser.

[0065] A number of embodiments have been described. However, it will be appreciated that various modifications can be made without departing from the spirit and scope of this disclosure. Accordingly, other embodiments are within the scope of the appended claims.

Claims

1. A method for suppressing biased data, comprising: receiving, at data processing hardware, a cluster training dataset comprising a known unbiased population of data; training, by the data processing hardware, a clustering model to segment the received cluster training data set into clusters based on data characteristics of the known unbiased data population, the data characteristics including human characteristics or attributes that, when over-represented or under-represented in data samples of a target population, result in an increased likelihood of biased predictions for samples from the target population, each cluster of the cluster training data set including a cluster weight; receiving, at the data processing hardware, a training data set for a machine learning model; generating, by the data processing hardware, a training data set weight corresponding to the training data set for the machine learning model based on the clustering model; adjusting, by the data processing hardware, each of the training dataset weights to match a corresponding cluster weight and form an adjusted training dataset; as well as The adjusted training dataset is provided to the machine learning model as an unbiased training dataset by the data processing hardware. 2 . The method of claim 1 , wherein the data characteristics include at least one of: race, gender, age, and nationality.

3. The method of claim 1 , wherein providing the adjusted training dataset as the unbiased training dataset to the machine learning model comprises: The machine learning model is trained using the unbiased training dataset.

4. The method according to claim 1, further comprising: training the machine learning model using the unbiased training dataset by the data processing hardware; receiving, at the data processing hardware, a sample data set including at least one corresponding data characteristic; as well as The data processing hardware uses the trained machine learning model to generate unbiased prediction values based on the received sample data set.

5. The method of claim 1 , wherein adjusting each training dataset weight to match the corresponding cluster weight comprises, for each training dataset weight: matching the training dataset weights to the corresponding cluster weights based on common data characteristics; and Data is removed from the training dataset until the training dataset weights match the corresponding cluster weights.

6. The method of claim 1 , wherein adjusting each training dataset weight to match the corresponding cluster weight comprises, for each training dataset weight: matching the training dataset weights to the corresponding cluster weights based on common data characteristics; and Data is copied from the training dataset until the training dataset weights match the corresponding cluster weights.

7. The method according to claim 1, wherein Adjusting each training dataset weight to match the corresponding cluster weight includes, for each training dataset weight: matching the training dataset weights to the cluster weights based on common data characteristics; as well as When the training dataset weight is less than the corresponding cluster weight, an importance weight associated with the training dataset weight is associated, the importance weight providing an indication to the training phase of the machine learning model and indicating to increase the training of the machine learning model for the training data corresponding to the training dataset weight.

8. The method of claim 1 , wherein adjusting each training dataset weight to match the corresponding cluster weight comprises, for each training dataset weight: matching the training dataset weights to the cluster weights based on common data characteristics; and When the training dataset weight is greater than the corresponding cluster weight, an importance weight associated with the training dataset weight is associated, and the importance weight provides an indication to the training phase of the machine learning model and indicates to reduce the training of the machine learning model for the training data corresponding to the training dataset weight.

9. The method of claim 1 , wherein training the clustering model further comprises: Segmenting the received cluster training data set into clusters based on data characteristics of the known unbiased data population; as well as For each of the clusters based on the data characteristics of the known unbiased data population, the cluster weight for each cluster of a cluster model is determined based on a ratio of a size of the corresponding cluster to a size of the known unbiased data population.

10. The method according to any one of claims 1-9, wherein an unsupervised machine learning algorithm segments the received cluster training data set into clusters based on data characteristics of the known unbiased data population.

11. A system for suppressing biased data, comprising: Data processing hardware; and Memory hardware in communication with the data processing hardware, the memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations comprising: receiving a cluster training dataset comprising a known unbiased data population; training a clustering model to segment the received cluster training data set into clusters based on data characteristics of the known unbiased data population, the data characteristics including human characteristics or attributes that, when over-represented or under-represented in data samples of a target population, result in an increased likelihood of biased predictions for samples from the target population, each cluster of the cluster training data set including a cluster weight; Receive a training dataset for a machine learning model; generating a training dataset weight corresponding to the training dataset for the machine learning model based on the clustering model; adjusting each of the training dataset weights to match the corresponding cluster weight and form an adjusted training dataset; and The adjusted training dataset is provided to the machine learning model as an unbiased training dataset.

12. The system of claim 11, wherein the data characteristics include at least one of: race, gender, age, and nationality.

13. The system of claim 11 , wherein providing the adjusted training dataset as the unbiased training dataset to the machine learning model comprises: The machine learning model is trained using the unbiased training dataset.

14. The system of claim 11, wherein the operations further comprise: Training the machine learning model using the unbiased training dataset; receiving a sample data set including at least one corresponding data characteristic; as well as Use the trained machine learning model to generate unbiased predictions based on the received sample dataset.

15. The system of claim 11 , wherein adjusting each training dataset weight to match the corresponding cluster weight comprises, for each training dataset weight: matching the training dataset weights to the corresponding cluster weights based on common data characteristics; and Data is removed from the training dataset until the training dataset weights match the corresponding cluster weights.

16. The system of claim 11 , wherein adjusting each training dataset weight to match the corresponding cluster weight comprises, for each training dataset weight: matching the training dataset weights to the corresponding cluster weights based on common data characteristics; and Data is copied from the training dataset until the training dataset weights match the corresponding cluster weights.

17. The system of claim 11 , wherein adjusting each training dataset to match the corresponding cluster weight comprises, for each training dataset weight: matching the training dataset weights to the cluster weights based on common data characteristics; and When the training dataset weight is less than the corresponding cluster weight, an importance weight associated with the training dataset weight is associated, the importance weight providing an indication to the training phase of the machine learning model and indicating to increase the training of the machine learning model for the training data corresponding to the training dataset weight.

18. The system of claim 11 , wherein adjusting each training dataset weight to match the corresponding cluster weight comprises, for each training dataset weight: matching the training dataset weights to the cluster weights based on common data characteristics; and When the training dataset weight is greater than the corresponding cluster weight, an importance weight associated with the training dataset weight is associated, and the importance weight provides an indication to the training phase of the machine learning model and indicates to reduce the training of the machine learning model for the training data corresponding to the training dataset weight.

19. The system of claim 11, wherein the operation of training the clustering model further comprises: Segmenting the received cluster training data set into clusters based on data characteristics of the known unbiased data population; as well as For each of the clusters based on the data characteristics of the known unbiased data population, the cluster weight for each cluster of the cluster model is determined based on a ratio of a size of the corresponding cluster to a size of the known unbiased data population.

20. The system of any one of claims 11-19, wherein an unsupervised machine learning algorithm segments the received cluster training data set into clusters based on data characteristics of the known unbiased data population.

Images