Use of machine learning models to suppress bias data

Abstract

The invention relates to suppression of deviation data using a machine learning model. A method for suppressing bias data using a machine learning model includes receiving a clustering training dataset including a known unbiased data population, and training a clustering model to segment the received clustering training dataset into clusters based on data characteristics of the known unbiased data population. Each cluster of the cluster training data set includes a cluster weight. The method further includes receiving a training data set for the machine learning model; and generating a training data set weight corresponding to the training data set for the machine learning model based on the clustering model. The method further includes adjusting each of the training dataset weights to match a respective cluster weight, and providing the adjusted training dataset as an unbiased training dataset to the machine learning model.

Description

Description of the case This application is a divisional application of Chinese invention patent application No. 201980057266.0, filed on August 26, 2019. Technical Field

[0001] The present disclosure relates to rejecting biased data using machine learning models. Background Art

[0002] Generally speaking, bias is the tendency of statistical data to overestimate or underestimate parameters. In this regard, data collection and analysis usually include some inherent biases. These biases may be generated by the method of collection and analysis or the entity that performs collection and analysis. For example, data research designed and performed by humans can cater to specific assumptions, human design constraints (e.g., human ability), sampling constraints, etc. By catering to these elements, the data results of the research are likely to include various sampling errors, measurement errors, or more broadly based on the errors of samples that do not represent the target population of the research. Since computer processing enables technology to collect data and analyze data at a rate incomparable with human activities, data processing technology must also overcome the problem of bias. Otherwise, data processing, especially for large pieces of data, may amplify the problem of bias and produce results that are also incomparable with the bias generated by human activities. Summary of the invention

[0003] One aspect of the present disclosure provides a method for suppressing biased data using a machine learning model. The method includes receiving a cluster training data set at data processing hardware, the cluster training data set including a known unbiased data population. The method also includes training a cluster model by the data processing hardware to segment the received cluster training data set into clusters based on data characteristics of the known unbiased data population. Each cluster of the cluster training data set includes a cluster weight. The method further includes receiving a training data set for the machine learning model at the data processing hardware; and generating a training data set weight corresponding to the training data set for the machine learning model based on the cluster model by the data processing hardware. The method also includes adjusting each training data set weight in the training data set weights by the data processing hardware to match the corresponding cluster weight and form an adjusted training data set; and providing the adjusted training data set to the machine learning model as an unbiased training data set by the data processing hardware.

[0004] Implementations of the present disclosure may include one or more of the following optional features. In some implementations, the method includes providing the adjusted training data set as an unbiased training data set to the machine learning model and also includes training the machine learning model with the unbiased training data set. The method may include training the machine learning model with the unbiased training data set by data processing hardware or may include receiving a sample data set including at least one corresponding data feature at the data processing hardware. Here, the method may also include generating an unbiased prediction value based on the received sample data set using the trained machine learning model by the data processing hardware.

[0005] In some examples, adjusting each training data set weight to match the corresponding cluster weight includes: for each training data set weight, matching the training data set weight to the corresponding cluster weight based on the common data characteristics; and removing data from the training data set until the training data set weight matches the corresponding cluster weight. In other examples, adjusting each training data set weight to match the corresponding cluster weight includes: for each training data set weight, matching the training data set weight to the cluster weight based on the common data characteristics; and copying data from the training data set until each training data set weight matches the corresponding cluster weight.

[0006] In some configurations, for each training data set weight, adjusting each training data set weight to match the corresponding cluster weight includes matching the training data set weight to the cluster weight based on the common data characteristics. When the training data set weight is less than the corresponding cluster weight, the method may include associating an indication to increase the importance weight of training of the machine learning model for the training data corresponding to the training data set weight. Additionally or alternatively, for each training data set weight, adjusting each training data set weight to match the corresponding cluster weight may include matching the training data set weight to the cluster weight based on the common data characteristics. Here, when the training data set weight is greater than the corresponding cluster weight, the method may include associating an indication to decrease the importance weight of training of the machine learning model for the training data corresponding to the training data set weight.

[0007] In some implementations, adjusting each of the training dataset weights to match the corresponding cluster weight includes, for each training dataset weight, matching the training dataset weight to the corresponding cluster weight based on the common data characteristics. When the training dataset weight is less than the corresponding cluster weight, the method includes associating an indication to increase the importance weight of training of the machine learning model for the training data corresponding to the training dataset weight, and when the training dataset weight is greater than the corresponding cluster weight, the method includes associating an indication to decrease the importance weight of training of the machine learning model for the training data corresponding to the training dataset weight.

[0008] In some examples, when training the clustering model, the method includes segmenting the received clustering training data set into clusters based on data characteristics of a known unbiased data population. In this example, for each cluster in the clustering based on the data characteristics of the known unbiased data population, the method includes determining a cluster weight for each cluster of the clustering model based on a ratio of the size of the corresponding cluster to the size of the known unbiased data population. In some embodiments, the unsupervised machine learning algorithm segments the received clustering training data set into clusters based on the data characteristics of the known unbiased data population.

[0009] Another aspect of the present disclosure provides a system for suppressing biased data using a machine learning model. The system includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. The operation includes receiving a cluster training data set, the cluster training data set including a known unbiased data population. The operation also includes training a clustering model to segment the received cluster training data set into clusters based on data characteristics of known unbiased data clusters, each cluster of the cluster training data set including cluster weights. The operation further includes receiving a training data set for the machine learning model; and generating training data set weights corresponding to the training data set for the machine learning model based on the clustering model. The operation also includes adjusting each training data set weight in the training data set weights to match the corresponding cluster weights and providing the adjusted training data set to the machine learning model as an unbiased training data set.

[0010] This aspect may include one or more of the following optional features. In some configurations, providing the adjusted training data set as an unbiased training data set to the machine learning model includes training the machine learning model with the unbiased training data set. The operations may also include: training the machine learning model with the unbiased training data set; receiving a sample data set including at least one corresponding data feature; and using the machine learning model to generate an unbiased prediction value based on the received sample data set.

[0011] In some implementations, adjusting each training data set weight to match the corresponding cluster weight further includes: for each training data set weight, matching the training data set weight to the corresponding cluster weight based on the common data characteristics; and removing data from the training data set until the training data set weight matches the corresponding cluster weight. In other examples, adjusting each training data set weight to match the corresponding cluster weight includes: for each training data set weight, matching the training data set weight to the cluster weight based on the common data characteristics; and copying data from the training data set until each training data set weight matches the corresponding cluster weight.

[0012] In some examples, the operation of adjusting each training data set to match the corresponding cluster weight includes, for each training data set weight, matching the training data set weight to the cluster weight based on the common data characteristics. In this example, when the corresponding training data set weight is less than the corresponding cluster weight, the operation includes associating an indication to increase the importance weight of the training of the machine learning model for the training data corresponding to the training data set weight. In other examples, the operation of adjusting each training data set weight to match the corresponding cluster weight may include matching the training data set weight to the cluster weight based on the common data characteristics. In this example, when the corresponding training data set weight is greater than the corresponding cluster weight, the operation includes associating an indication to decrease the importance weight of the training of the machine learning model for the training data corresponding to the training data set weight.

[0013] Additionally or alternatively, adjusting each training data set weight to match the corresponding cluster weight may include, for each training data set weight, matching the training data set weight to the corresponding cluster weight based on the common data characteristics. Here, when the corresponding training data set weight is less than the corresponding cluster weight, the association indicates increasing the importance weight of the training of the machine learning model for the training data corresponding to the training data set weight, and when the training data set weight is greater than the corresponding cluster weight, the association indicates decreasing the importance weight of the training of the machine learning model for the training data corresponding to the training data set weight.

[0014] In some configurations, the operation of training the clustering model may include segmenting the received clustering training data set into clusters based on data characteristics of a known unbiased data population; and for each cluster in the clustering based on the data characteristics of the known unbiased data population, determining a cluster weight for each cluster of the clustering model based on a ratio of the size of the corresponding cluster to the size of the known unbiased data population. In some examples, the unsupervised machine learning algorithm segments the received clustering training data set into clusters based on data characteristics of the known unbiased data population.

[0015] The details of one or more embodiments of the present disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a diagram of an example machine learning environment.

[0017] Figure 2A is a schematic diagram of example processing stages for a bias suppression model.

[0018] Figure 2B is Figure 2A Schematic diagram of an example bias suppression model during the training phase of .

[0019] Figure 2C is Figure 2A Schematic diagram of an example bias suppression model during the bias-free phase of .

[0020] Figure 2D and 2E is Figure 2A Schematic diagram of example adjustments made by the bias suppression model during the bias-free phase of .

[0021] Figure 3 is a schematic diagram of example processing stages of a machine learning model that incorporates unbiased training data from a bias suppression model.

[0022] Figure 4 is a diagram of an example bias scoring model used to generate a bias score for a dataset.

[0023] Figure 5 is a flowchart of an example method for making a machine learning model unbiased within a machine learning environment.

[0024] Figure 6 is a schematic diagram of an example computing device that can be used to implement the systems and methods described herein.

[0025] Like reference numbers in the various drawings indicate like elements. DETAILED DESCRIPTION

[0026] Figure 11 is an example of a machine learning environment 10. The machine learning environment 10 generally includes a distributed system 100 (e.g., a remote system such as a cloud environment) having a resource 110 accessible via a network 120, a bias suppression model 200, and a machine learning model 300. The resource 110 is accessible to the bias suppression model 200 and / or the machine learning model 300 for training the bias suppression model 200 and / or the machine learning model 300, and for performing the machine learning functions disclosed herein. The distributed system 100 can be any computer processing system having computing resources (e.g., resources 110) capable of executing the bias suppression model 200 and / or the machine learning model 300. In some examples, the bias suppression model 200 and / or the machine learning model 300 is executed on a device that can access the distributed system 100 via the network 120 or otherwise communicate with the distributed system 100. For example, the device can execute a web-based application associated with the distributed system 100.

[0027] Typically, the resources 110 of the distributed system 100 may include hardware resources 110h, 110h 1-i and software resources 110s, 110s 1-i The hardware resources 110h include data processing hardware 112 and memory hardware 114. The software resources 110s may include software applications, software services, application programming interfaces (APIs), etc. The software resources 110s may reside on the hardware resources 110h (e.g., stored in the memory hardware 114), or may include instructions executed on the data processing hardware 112.

[0028] Software applications (i.e., software resources 110s) may refer to computer software that enables a computing device to perform tasks. In some examples, software applications may be referred to as "applications," "apps," or "programs." Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.

[0029] Memory hardware 114 is non-transitory memory that can be a physical device used to store programs (e.g., sequences of instructions) or data (e.g., program state information) on a temporary or permanent basis for use by data processing hardware 112. Memory hardware 114 can be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., commonly used for firmware, such as boot programs), and disk or tape. Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and phase change memory (PCM).

[0030] In the example shown, the bias suppression model 200 is executed in conjunction with the resource 110 to address the bias issue to prevent the machine learning model 300 from being adversely affected. In other words, the bias suppression model 200 is configured to prevent the machine learning model 300 from being trained on a machine learning (ML) training dataset 302 that includes biased data by: generating / outputting an unbiased training dataset associated with the ML training dataset 302 for use in training the ML model 300, but with the biased data removed. By preventing the machine learning model 300 from being trained on the biased data within the ML training dataset 302, the machine learning model 300 is not affected by the biased data and is therefore able to generate unbiased prediction values ​​310 ( Figure 3 ). Thus, the bias suppression model 200 corresponds to a filter that removes / adjusts biased data within the ML training dataset 302 prior to training the ML model 300 by outputting / generating an unbiased training dataset 206 for use in training the ML model 300.

[0031] Figure 2A The bias suppression model 200 is shown during execution of a first training phase 202 and a second unbiased phase 204 following the first training phase 202. During the training phase 202, the bias suppression model 200 receives the clustered training dataset 130 and outputs the clustering weights 214. During the unbiased phase 204, the bias suppression model 200 receives the ML training dataset 302 and uses the clustering weights 214 output from the training phase 202 to output an unbiased training dataset 206 having biased data removed from the ML training dataset 302.

[0032] Here, the term "weight" (e.g., the deviation cluster weights 214, 214a-n and the training data set weights 218, 218a-n) refers to a value, such as a ratio, mapped to a unique cluster formed by the process of clustering. For a population, each cluster can be related to a portion of the population, so the value of the portion can be a weight associated with the cluster (e.g., a subset of the population). In other words, by clustering the population into subsets, each subset inherently has characteristics (e.g., weights) for the population. More generally, clusters such as the deviation cluster 212 or the training cluster 216 refer to groups of people that can be used to group training data related to those people. Groups of people can include people who share a continuous range of variable values ​​in their training data (e.g., a cluster for Asian females aged 25-27 may include one training example of an Asian female aged 25, another training example of an Asian female aged 26, and other training examples that share this set of values).

[0033] In other implementations, clustering includes people whose training data is clustered by a clustering algorithm (e.g., a clustering model). The clustering algorithm places people into groups that the algorithm considers similar based on having a shorter distance between people (or characteristics of people). Grouping by shorter distances can avoid the exponential growth of the number of clusters as the number of variable values ​​grows in the corresponding group. Clustering can be performed based on the variable of interest (e.g., a deviation variable) and / or other variables to determine the distance between training data (e.g., people). For example, clustering is performed based on other variables, but the final determination for clustering the data is based on the variable of interest (e.g., a deviation variable). As an example, the clustering process groups 18-year-old and 19-year-old male Austrians and Germans into a single cluster because it identifies similarities based on a defined metric (e.g., language used, related interests, frequency of being connected in a social network or being a member of the same organization). As another example to illustrate the large number of potential clustering approaches, the clustering process may have four separate groups covering these categories: (1) 18-year-old Austrians; (2) 18-year-old Germans; (3) 19-year-old Austrians; and (4) 19-year-old Germans.

[0034] Further references Figure 2ADuring the training phase 202, the bias suppression model 200 receives a clustered training data set 130 corresponding to a known unbiased data population. The known unbiased data population may be a target population with an accurate probability distribution of bias-sensitive variables. With the known unbiased data population, the bias suppression model 200 avoids training with a disproportionate amount of data related to bias-sensitive variables. Bias-sensitive variables 132 refer to variables that, when over-represented or under-represented in a data sample of the target population, result in an increased likelihood of biased predictions from samples of the target population. In other words, a small deviation from an accurate representation of a bias-sensitive variable is likely to produce a skewed predictive analysis. Therefore, when a machine learning model such as the machine learning model 300 is constructed (i.e., trained) without an accurate training data set for the bias-sensitive variables, the machine learning model may inherently produce biased predictions and biased computational analysis. Some examples of bias-sensitive variables include race, gender, sex, age, nationality, religious affiliation, political affiliation, affluence, and the like.

[0035] In some examples, the target population is the complete data set for a given variable or set of variables. Here, the bias suppression model 200 and / or the machine learning model 300 can be trained and / or make predictions corresponding to the target population (e.g., the population corresponding to the cluster training data set 130). As a basic example, the machine learning model 300 can be configured to predict the value of the target population as the population in California. In order to accurately make predictions about the population in California, each model 200, 300 is trained on data associated with the population in California.

[0036] After training the bias suppression model 200 on the received clustered training dataset 130, the bias suppression model 200 is configured to adjust the ML training dataset 302 intended for use in training the ML model 300 during the unbiased phase 204. By adjusting the ML training dataset 302 before training the ML model 300, the bias suppression model 200 generates an unbiased training dataset 206 and provides the unbiased training dataset 206 to the ML model 300. In other words, during the unbiased phase 204, the bias suppression model 200 converts the ML training dataset 302 (e.g., which may potentially include biased data) into the unbiased training dataset 206 based on the training of the bias suppression model 200 on the clustered training dataset 130 during the training phase 202. In some examples, the bias suppression model 200 is trained with more than one clustered training dataset 130. For example, the bias suppression model 200 is dynamically trained on new or updated clustered training datasets 130 to continuously account for changes in the clustered training datasets 130 over time. The training phase 202 and the unbiasing phase 204 may be performed consecutively or simultaneously, or some combination of both.

[0037] Figure 2B An example of a bias suppression model 200 during a training phase 202 is illustrated. Here, the bias suppression model 200 receives a cluster training dataset 130 that includes a known unbiased data population. Although in some implementations, an entity 140, such as an administrator of the bias suppression model 200 or a user who has access to the bias suppression model 200 (e.g., a user who is interested in a particular set of bias features) can define bias features corresponding to bias sensitive variables. Here, the entity or designer of the bias suppression model 200 does not supply bias features and / or bias sensitive variables to the bias suppression model 200. Instead, the bias suppression model 200 via the segmenter 210 uses a cluster model 211 to model the cluster training dataset 130 in order to identify biased or unbiased data. In some configurations, the cluster training dataset 130 includes a complete target population dataset. For example, the bias suppression model 200 can receive a complete dataset of U.S. demographic data as the cluster training dataset 130.

[0038] The bias suppression model 200 includes a segmenter 210 and a regulator 220. The segmenter 210 is configured to segment the data set into clusters 212, 212a-n using a clustering model 211 (interchangeably referred to as "clustering model 211"). During the training phase 202, the segmenter 210 trains the clustering model 211 to classify the data set into clusters 212, 212a-n based on the data characteristics of the known unbiased data population (in Figure 2B Shown as "DC a-n”) segments the received cluster training data set 130 into clusters 212, 212a-n. For simplicity, the data features include at least one corresponding bias-sensitive variable of the target group related to the cluster training data set 130. In other words, some clusters 212 may be bias clusters associated with at least one corresponding bias-sensitive variable as a data feature, while other clusters 212 identify data features that are not related to the bias-sensitive variable. In some embodiments, the cluster model 211 includes a clustering algorithm, so that the cluster model 211 is based on the received data during the training phase 202. The clustering model 211 is trained to identify the probability distribution of an unbiased data set in terms of data characteristics (via a known unbiased data population). For example, the clustering model 211 is trained to generate at least one cluster 212 having data characteristics representing a bias-sensitive variable and / or a combination of bias-sensitive variables.

[0039] As an example, the clustering model 211 clusters each bias-sensitive variable of race, gender, and age into data characteristics of known unbiased groups. Therefore, here, each cluster 212 can correspond to a corresponding bias-sensitive variable combination. As an example, using the data characteristics of race, gender, and age, at least one cluster 212 corresponds to a type of race (e.g., black, white, Hispanic, etc.), a type of gender (e.g., male, female), and a type of age category (e.g., 19-30, 31-44, 45-59, 60+, etc.). When the segmenter 210 segments the cluster training data set 130 into clusters 212 using the clustering model 211, the segmenter 210 is also configured to determine a corresponding cluster weight 214 for each cluster 212, so that the clusters 212, 212a-n have associated cluster weights 214, 214a-n. In some examples, the cluster weight 214 represents the group portion of the cluster 212 of the target group (e.g., the group of the cluster training data set 130). For example, the cluster weights 214 may represent a ratio of the size of the corresponding cluster 212 to the size of the target population of the training data set 130. In some examples, to determine each cluster weight 214, the segmenter 210 determines the population portion of each cluster 212 and divides each population portion by the maximum population portion of all clusters 212 (e.g., each cluster weight 214 is less than 1). In other examples, to determine each cluster weight 214, the segmenter 210 determines the population portion of each cluster 212 and divides each population portion by the lowest population portion of all clusters 212 (e.g., each cluster weight 214 is greater than 1).

[0040] In some configurations, the segmenter 210 transmits the cluster weights 214 of the clusters 212 to the adjuster 220 during the training phase 202. For example, the adjuster 220 includes a data store 222 of the cluster weights 214. In other examples, the segmenter 210 stores the cluster weights 214 (e.g., in a data store of the segmenter 210) for access by the adjuster 220 during the unbiased phase 204.

[0041] Figure 2C An example of a bias suppression model 200 during an unbiased phase 204 is illustrated. During the unbiased phase 204, the bias suppression model 200 receives an ML training data set 302 intended for use in training an ML model 300. For example, the training data set 302 may include an original training data set that may be potentially biased (e.g., including biased data). In some implementations, the training data set 302 is a sample of a target population, and therefore, may not accurately reflect the bias-sensitive variables 132 of the target population. For example, the target population may have a racial composition of 25% whites, whereas the training data set 302 may indicate a sampled racial composition of 45% whites. Therefore, in order to prevent the ML model 300 from being trained on the ML training data 302 that inaccurately reflects the bias-sensitive variables of the target population, the bias suppression model 200 seeks to adjust this bias (e.g., 20% difference) with the segmenter 210 and the adjuster 220 during the unbiased phase 204.

[0042] Similar to how segmenter 210 Figure 2BThe segmenter 210 is configured to segment the biased training dataset 130 into biased clusters 212 during the training phase 202 of the machine learning model 300, and the segmenter 210 is configured to segment the received ML training dataset 302 into training clusters 216 during the unbiased phase 204. The segmenter 210 segments the training dataset 302 by providing the training dataset 302 to the trained clustering model 211. Based on the training from the training phase 202, the clustering model 211 has learned how to segment a dataset such as the training dataset 302 into clusters (e.g., clusters 212a-n or training clusters 216a-n). During the unbiased phase 204, the clustering model 211 generates the training clusters 216, 216a-n based on the received training dataset 302 intended for use in the machine learning model 300. Here, at least one training cluster 216 is associated with at least one corresponding bias-sensitive variable of the target population. The segmenter 210 is also configured to generate a corresponding training data set weight 218, 218a-n for each segmented training cluster 216, such that the segmented training cluster 216 includes an associated training data set weight 218. In some examples, the corresponding training data set weight 218 represents a population portion of the training cluster 216 for the sample population associated with the training data set 302. For example, the training data set weight 218 can represent a ratio of the size of the corresponding training cluster 216 to the size of the sample population of the training data set 302. In some examples, to determine each training data set weight 218, the segmenter 210 determines the population portion of each training cluster 216 and divides each population portion by the maximum population portion of the training cluster 216 (e.g., each training data set weight 218 is less than 1). In other examples, to determine each training data set weight 218, the segmenter 210 determines the population portion of each training cluster 216 and divides each population portion by the lowest population portion of the training clusters 216a-n (e.g., each training data set weight 218 is greater than 1).

[0043] The adjuster 220 is configured to adjust the training data set weights 218 to match the probability distribution of the data characteristics of the target population (i.e., the bias-sensitive variables). In some implementations, the adjuster 220 performs a process 226 that adjusts the training data set weights 218 by comparing the training data set weights 218 with the cluster weights 214. For example, Figure 2C-2EThe adjuster 220 is depicted executing a process 226 to retrieve and compare the cluster weights 218 from the cluster weight data store 222 and the training data set weights 218 from the training weight data store 224, and based on the comparison, adjust the training data set weights 218. For example, based on the comparison difference between the corresponding training data set weights 218 and the cluster weights 214, the adjuster 220 may adjust the corresponding training data set weights 218 to match the corresponding cluster weights 214. Thus, the process 226 executed by the adjuster 220 for adjusting the training data set weights 218 generates / outputs the adjusted training data set weights, or more generally the adjusted training data set 208, that form the unbiased training data set 206 for training the ML model 300.

[0044] In some implementations, the adjuster 220 performs the process 226 by first matching one or more training dataset weights 218a-n with one or more cluster weights 214a-n based on matching data characteristics, such as bias-sensitive variables. For example, when the training dataset weights 218 and the cluster weights 214 each share a common data characteristic (i.e., bias-sensitive variable) or a combination of data characteristics, the adjuster 220 can adjust the training dataset weights 218 with the matching (i.e., corresponding) cluster weights 214 to output corresponding adjusted training dataset weights and / or adjusted training dataset 208.

[0045] refer to Figure 2D, the adjuster 220 compares the training dataset weights 218 and the bias cluster weights 214 of the combination that share a common data feature (e.g., bias-sensitive variable) or data characteristics. When the ML training dataset 302 over-represents the bias-sensitive variable, the training dataset weight 218 exceeds (e.g., is greater than) the cluster weight 214 of the data characteristic corresponding to the bias-sensitive variable (e.g., the training dataset 302 indicates a white racial composition of more than 20%). In response to such over-representation, the process 226 performed by the adjuster 220 may correspond to a data removal adjustment process that adjusts the training dataset weights 218 by removing data from the training dataset 302 until the training dataset weights 218 match the cluster weights 214. On the other hand, when the training dataset 302 under-represents the bias-sensitive variable, the training dataset weights 218 are less than the cluster weights 214 of the data characteristic corresponding to the bias-sensitive variable (e.g., the training dataset 302 indicates a black racial composition of less than 20%). In response to this under-representation, the process 226 executed on the adjuster 220 can correspond to a data copying process that adjusts the training dataset weights 218 by copying data from the training dataset 302 until the training dataset weights 218 match the cluster weights 214. In some implementations, the adjuster 220 randomly copies or removes data from the training dataset 302 to maintain the integrity of the training dataset 302. This can avoid further bias associated with non-random selective copying or removal.

[0046] same Figure 2C In contrast to the process 226 of removing / adding data to / from the training dataset 302 until the training dataset weights 218 match the bias cluster weights 214, Figure 2E 26 is shown performing on the adjuster 220 to adjust the importance weight 228 associated with each training data set weight 218. Specifically, the process associates the importance weight 228 with the data of the training data set 302 corresponding to the associated training data set weight 218. The importance weight 228 is provided to the training phase 304 ( Figure 3 ) provides instructions to understand how much weight to provide to the underlying data corresponding to the training data set weight 218 while training the machine learning model 300. In some examples, when the training data set weight 218 is greater than the cluster weight 214, the adjuster 220 associates an instruction to reduce the importance weight 228 of the training of the machine learning model 300 for the training data corresponding to the corresponding training data set weight 218. In other examples, when the training data set weight 218 is less than the cluster weight 214, the adjuster 220 associates an instruction to increase the importance weight 228 of the training of the machine learning model 300 for the training data corresponding to the corresponding training data set weight 218.

[0047] If Figure 2A-2E As shown, the bias suppression model 200 generates an unbiased training dataset 206 for training the machine learning model 300. Figure 3 300 is an example of a machine learning model 300 trained on an unbiased training dataset 206. A machine learning model, such as the machine learning model 300, is typically taught (or trained) based on a dataset and a result set to predict its own output based on input data similar to the dataset. In some implementations, much like the bias suppression model 200, the machine learning model 300 first undergoes training during a training phase 304 and then undergoes a prediction phase (e.g., inference) 306 that receives a sample dataset 308 as input and outputs an unbiased prediction value 310. During the prediction phase 306, the machine learning model 300 receives a sample dataset 308, such as a sample dataset that includes at least one bias-sensitive variable, and utilizes an associated machine learning functionality trained on the unbiased training dataset 206 to generate an unbiased prediction value 310 based on the received sample dataset 308.

[0048] In some examples, the machine learning model 300 is trained with more than one unbiased training data set 206. For example, the machine learning model 300 is dynamically trained during operation to continuously learn dynamically changing data sets. In other words, the stages of training 304 and prediction 306 can operate continuously or simultaneously, or some combination of the two.

[0049] Figure 4 is an example of a bias scoring model 400. The bias scoring model 400 can be used in conjunction with the bias suppression model 200 or separately from the bias suppression model 200. For example, the bias scoring model 400 can evaluate a training data set 302 intended to train the machine learning model 300 before providing the training data set 302 to the bias suppression model 200 (i.e., these models 200, 300 negotiate with the bias scoring model 400). In these examples, when the bias scoring model 400 suppresses the training data set 302, indicating that the training data set 302 is too biased to continue training the machine learning model 300, the bias scoring model 400 can communicate the suppressed training data set 302 to the bias suppression model 200 to unbias the suppressed training data set 426 and form an unbiased training data set 206, as described above with reference to FIG. Figure 2A-2E as described.

[0050] Similar to the bias suppression model 200, the cluster model 211, and / or the machine learning model 300, the bias scoring model 400 undergoes a training phase 402 to train the bias scoring model 400 to score a data set, and once trained, the data set is scored in a scoring phase 404 based on the training from the training phase 402. During the training phase 402, the bias scoring model 400 receives one or more bias scoring training data sets 410. Each bias scoring training data set 410 includes data such as bias data 412 and / or unbiased data 414 and a bias score 416. For example, the bias score 416 is a numerical representation of the bias in the data set. In some examples, the bias score 416 and / or the bias scoring training data set 410 originates from a scorer 140. The scorer 140 can be an administrator within the machine learning environment 10 (e.g., an administrator of the models 200, 211, 300) or a user concerned about the bias on the machine learning model 300. In some examples, the scorer 140 is one or more entities / sources (i.e., a committee) or another machine learning model that has been trained to compile and / or score a data set. During the training phase 402 , the bias scoring model 400 receives one or more bias scoring training datasets 410 and learns to generate bias scores 416 for the datasets.

[0051] Once trained, or in the case where the bias scoring model 400 is continuously trained in parallel with the scoring stage 404, the bias scoring model 400 receives (e.g., intercepts) a training data set 302 intended for use with the machine learning model 300. Based on this training, the bias scoring model 400 performs a scoring process 420, in which the bias scoring model 400 generates a bias score 416 for the training data set 302. As part of the scoring process 420, the bias scoring model 400 determines whether the bias score 416 for the training data set 302 satisfies a score threshold 422. Here, the score threshold 422 indicates the confidence that the data set is unbiased or has negligible bias for the purpose of making predictions at the machine learning model 300. For example, the score threshold 422 is an acceptable bias score value.

[0052] When the bias score 416 of the training data set 302 satisfies the score threshold 422 (e.g., exceeds an acceptable bias score value), the bias scoring model 400 approves the training data set 302 as an approved training data set 424. In some examples, the approved training data set 424 includes an approval indicator that the machine learning model 300 can recognize, so that the machine learning model proceeds to generate unbiased prediction values ​​310 (e.g., Figure 3). When the bias score 416 of the training dataset 302 fails to meet the score threshold 422 (e.g., is less than an acceptable bias score value), the bias scoring model 400 suppresses the training dataset 302. The suppressed training dataset 426 may include a suppression indicator to inform the machine learning model 300 not to use the suppressed training dataset 302 for training. As shown by the dashed box and arrow in Figure 4 As indicated in , the bias scoring model 400 can communicate (i.e., provide) the suppressed training dataset 302 to the bias suppression model 200 so that the bias suppression model 200 can convert the suppressed training dataset 302 into an unbiased training dataset 206, as described above in references 2A-2E.

[0053] The bias suppression model 200 machine learning model 300 and / or the bias scoring model 400 can be any type of machine learning model (e.g., supervised, unsupervised, reinforcement, ensemble / decision tree, deep learning, neural network, recursive, linear, etc.) that employs at least one machine learning algorithm to perform the functionality of any of the models 200, 300, 400 described herein. Broadly speaking, the machine learning algorithm may involve supervised learning, unsupervised learning, active learning, or some hybrid combination of these types of learning algorithms. Some specific examples of these broad algorithms include machine learning algorithms such as linear regression algorithms, logistic regression algorithms, decision tree-based algorithms, support vector machine algorithms, naive Bayes classifiers, k-nearest neighbor algorithms, dimensionality reduction algorithms, gradient boosting algorithms, and the like.

[0054] Figure 5 5 is an example method 500 having operations for making a machine learning model 300 within a machine learning environment 10 unbiased. At operation 502, the method 500 receives a cluster training data set 130. The cluster training data set 130 includes a known unbiased data population. At operation 504, the method 500 trains a cluster model 211 to segment the received cluster training data set 130 into clusters 212 based on data characteristics of the known unbiased data population. Each cluster 212 in the clusters 212a-n includes a cluster weight 214. At operation 506, the method 500 receives a training data set 302 for the machine learning model 300. At operation 508, the method 500 generates training data set weights 218a-n, which correspond to the training data set 302 of the machine learning model 300 based on the cluster model 211. At operation 510, the method 500 adjusts each training data set weight 218 in the training data set weights 218a-n to match the corresponding cluster weight 214. At operation 512 , the method 500 provides the adjusted training dataset 208 to the machine learning model 300 as an unbiased training dataset 206 .

[0055] Figure 66 is a schematic diagram of an example computing device 600 that can be used to implement the systems and methods described in this document (e.g., bias suppression model 200 and / or machine learning model 300). Computing device 600 is intended to represent various forms of digital computers, such as laptops, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not meant to limit the implementation of the inventions described and / or claimed in this document.

[0056] The computing device 600 includes a processor 610, a memory 620, a storage device 630, a high-speed interface / controller 640 connected to the memory 620 and a high-speed expansion port 650, and a low-speed interface / controller 660 connected to a low-speed bus 670 and the storage device 630. Each of the components 610, 620, 630, 640, 650, and 660 is interconnected using various buses and can be mounted on a common motherboard or in other appropriate ways. The processor 610 is capable of processing instructions for execution within the computing device 600, including instructions stored in the memory 620 or stored on the storage device 630, to display graphical information for a graphical user interface (GUI) on an external input / output device such as a display 680 coupled to the high-speed interface 640. In other embodiments, multiple processors and / or multiple buses, as well as multiple memories and memory types, can be used appropriately. Moreover, multiple computing devices 600 can be connected, each of which provides a portion of the necessary operations (for example, as a server group, a blade server group, or a multi-processor system).

[0057] Memory 620 stores information non-temporarily within computing device 600. Memory 620 may be a computer-readable medium, a volatile memory unit, or a non-volatile memory unit. Non-temporary memory 620 may be a physical device used to store programs (e.g., sequences of instructions) or data (e.g., program state information) on a temporary or permanent basis for use by computing device 600. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.

[0058] The storage device 630 can provide mass storage for the computing device 600. In some embodiments, the storage device 630 is a computer readable medium. In various embodiments, the storage device 630 can be a floppy disk device, a hard disk device, an optical disk device or a tape device, a flash memory or other similar solid-state storage device, or a device array, including a device in a storage area network or other configuration. In other embodiments, a computer program product is tangibly embodied as an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine readable medium, such as a memory 620, a storage device 630, or a memory on a processor 610.

[0059] The high-speed controller 640 manages bandwidth-intensive operations of the computing device 600, while the low-speed controller 660 manages less bandwidth-intensive operations. This division of responsibilities is illustrative only. In some embodiments, the high-speed controller 640 is coupled to a memory 620, a display 680 (e.g., through a graphics processor or accelerator), and a high-speed expansion port 650 that can accept various expansion cards (not shown). In some embodiments, the low-speed controller 660 is coupled to a storage device 630 and a low-speed expansion port 690. The low-speed expansion port 690, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a networking device, such as a switch or a router, for example, through a network adapter.

[0060] As shown, computing device 600 may be implemented in a variety of different forms. For example, it may be implemented as a standard server 600a or multiple times in a group of such servers 600a, as a laptop computer 600b, or as part of a rack server system 600c.

[0061] Various implementations of the systems and techniques described herein can be implemented in digital electronic and / or optical circuitry, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be special purpose or general purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to these devices.

[0062] These computer programs (also referred to as programs, software, software applications or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages ​​and / or in assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0063] The process and logic flow described in this specification can be performed by executing one or more computer programs to perform functions by operating input data and generating outputs. The process and logic flow can also be performed by a dedicated logic circuit, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). For example, a processor suitable for executing a computer program includes a general-purpose and special-purpose microprocessor, and any one or more processors of any kind of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include one or more large-capacity storage devices for storing data or be operably coupled to a large-capacity storage device to receive data therefrom or transmit data to it, or both, and the large-capacity storage device is, for example, a magnetic disk, a magneto-optical disk, or an optical disk. However, a computer does not have to have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0064] To provide interaction with a user, one or more aspects of the present disclosure can be implemented on a computer having a display device, such as a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or a touch screen for displaying information to the user, and an optional keyboard and pointing device, such as a mouse and trackball, through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including sound, voice, or tactile input. In addition, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a web page to a web browser on the user's client device in response to a request received from the web browser.

[0065] Many embodiments have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of the present disclosure. Therefore, other embodiments are also within the scope of the appended claims.

Claims

1. A computer-implemented method, which when executed by data processing hardware causes the data processing hardware to perform operations comprising: Receiving a training data set for training a machine learning model, the training data set comprising one or more bias-sensitive variables; generating a bias score for the training dataset using a bias scoring model; determining whether the deviation score satisfies a threshold score; marking the training data set with an approval indicator when the deviation score satisfies the threshold score; as well as When the deviation score fails to satisfy the threshold score, the training data set is marked with a suppression indicator.

2. The method according to claim 1, wherein the operation further comprises: The bias scoring model is trained using one or more bias scoring training datasets.

3. The method of claim 2, wherein the bias scoring model is trained using feedback based on the bias scores. The method of claim 1 , wherein the deviation score is a numerical representation of the deviation. The method of claim 1 , wherein the threshold score comprises a user-configurable acceptable deviation score.

6. The method of claim 1, wherein the operation further comprises: When the deviation score meets the threshold score, the machine learning model is trained using the training dataset.

7. The method of claim 1, wherein the operations further comprise: When the bias score fails to satisfy the threshold score, an unbiased training dataset is generated from the training dataset using a bias suppression model.

8. The method of claim 7, wherein the operation further comprises: When the bias score fails to satisfy the threshold score, the machine learning model is trained using the unbiased training dataset.

9. The method according to claim 7, wherein: Generating the unbiased training data set includes: segmenting the training data set into a plurality of clusters, wherein at least one cluster of the plurality of clusters corresponds to a bias-sensitive variable of the one or more bias-sensitive variables; generating a weight for each cluster in the plurality of clusters; and The unbiased training dataset is generated based on a weighted plurality of clusters.

10. The method of claim 9, wherein the weight for each cluster is based on a probability distribution of data characteristics of the target group.

11. A system comprising: Data processing hardware; as well as Memory hardware in communication with the data processing hardware, the memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations comprising: Receiving a training data set for training a machine learning model, the training data set comprising one or more bias-sensitive variables; generating a bias score for the training dataset using a bias scoring model; determining whether the deviation score satisfies a threshold score; When the deviation score satisfies the threshold score, marking the training data set with an approval indicator; and When the deviation score fails to satisfy the threshold score, the training data set is marked with a suppression indicator.

12. The system of claim 11, wherein the operations further comprise: The bias scoring model is trained using one or more bias scoring training datasets.

13. The system of claim 12, wherein the bias scoring model is trained using feedback based on the bias scores. The system of claim 11 , wherein the deviation score is a numerical representation of the deviation.

15. The system of claim 11, wherein the threshold score comprises a user-configurable acceptable deviation score.

16. The system of claim 11, wherein the operations further comprise: When the deviation score meets the threshold score, the machine learning model is trained using the training dataset.

17. The system of claim 11, wherein the operations further comprise: When the bias score fails to satisfy the threshold score, an unbiased training dataset is generated from the training dataset using a bias suppression model.

18. The system of claim 17, wherein the operations further comprise: When the bias score fails to satisfy the threshold score, the machine learning model is trained using the unbiased training dataset.

19. The system of claim 17, wherein: Generating the unbiased training data set includes: segmenting the training data set into a plurality of clusters, wherein at least one cluster of the plurality of clusters corresponds to a bias-sensitive variable of the one or more bias-sensitive variables; generating a weight for each cluster in the plurality of clusters; and The unbiased training dataset is generated based on a weighted plurality of clusters.

20. The system of claim 19, wherein the weight for each cluster is based on a probability distribution of data characteristics of the target population.

21. A computer-implemented method, which when executed by data processing hardware causes the data processing hardware to perform operations comprising: Receiving a training data set for training a machine learning model, the training data set comprising one or more bias-sensitive variables; obtaining a target population for the training data set, the target population comprising a probability distribution of the one or more bias-sensitive variables of the training data set; identifying, from the training data set, over-represented variables among the one or more bias-sensitive variables based on the target population; determining a weight for the over-represented variable based on the target population, the weight adjusting the training data set to remove bias for the over-represented variable; as well as The machine learning model is trained using the training data set and the weights for the over-represented variables.

22. The method of claim 21, wherein the operations further comprise: A plurality of clusters based on the one or more bias-sensitive variables are generated based on the training data set.

23. The method of claim 22, wherein the operations further comprise: The weights are applied to corresponding clusters of the plurality of clusters associated with the over-represented variables.

24. The method according to claim 23, wherein: The weight comprises a ratio of a size of a corresponding cluster of the plurality of clusters associated with the over-represented variable to a size of the target population.

25. The method of claim 21, wherein the operations further comprise: The weights are stored in a data store of cluster weights.

26. The method according to claim 21, wherein: The one or more deviation-sensitive variables include one or more of the following: Race; Gender; and age.

27. The method of claim 21, wherein the operations further comprise: Based on identifying the over-represented variable among the one or more bias-sensitive variables from the training data set, data associated with the over-represented variable is removed from the training data set.

28. The method of claim 21, wherein: The operations also include training a bias suppression model using the training data set to identify deviant data.

29. The method according to claim 28, wherein: The operations also include: generating a bias score for the training dataset using the bias suppression model; determining that the deviation score satisfies a threshold score; and Based on determining that the deviation score satisfies the threshold score, the training data set is marked with a suppression indicator.

30. The method of claim 29, wherein the threshold score comprises a user-configurable acceptable deviation score.

31. A system comprising: Data processing hardware; as well as Memory hardware in communication with the data processing hardware, the memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations comprising: Receiving a training data set for training a machine learning model, the training data set comprising one or more bias-sensitive variables; obtaining a target population for the training data set, the target population comprising a probability distribution of the one or more bias-sensitive variables of the training data set; identifying, from the training data set, over-represented variables among the one or more bias-sensitive variables based on the target population; determining a weight for the over-represented variable based on the target population, the weight adjusting the training data set to remove bias for the over-represented variable; and The machine learning model is trained using the training data set and the weights for the over-represented variables.

32. The system of claim 31 , wherein the operations further comprise: A plurality of clusters based on the one or more bias-sensitive variables are generated based on the training data set.

33. The system of claim 32, wherein the operations further comprise: The weights are applied to corresponding clusters of the plurality of clusters associated with the over-represented variables.

34. The system of claim 33, wherein: The weight comprises a ratio of a size of a corresponding cluster of the plurality of clusters associated with the over-represented variable to a size of the target population.

35. The system of claim 31 , wherein the operations further comprise: The weights are stored in a data store of cluster weights.

36. The system of claim 31, wherein: The one or more deviation-sensitive variables include one or more of the following: Race; Gender; and age.

37. The system of claim 31 , wherein the operations further comprise: Based on identifying the over-represented variable among the one or more bias-sensitive variables from the training data set, data associated with the over-represented variable is removed from the training data set.

38. The system of claim 31, wherein: The operations also include training a bias suppression model using the training data set to identify deviant data.

39. The system of claim 38, wherein: The operations also include: generating a bias score for the training dataset using the bias suppression model; determining that the deviation score satisfies a threshold score; and Based on determining that the deviation score satisfies the threshold score, the training data set is marked with a suppression indicator.

40. The system of claim 39, wherein the threshold score comprises a user-configurable acceptable deviation score.

Images