Fairness evaluation program, fairness evaluation method, and information processing device

Abstract

This fairness evaluation program causes a computer to execute processes of: acquiring a plurality of pieces of data; identifying a first ratio of data, from the plurality of pieces of data, for which a first attribute of a plurality of attributes of the plurality of pieces of data has a first value; identifying a second ratio of data for which the first attribute has the first value from a first group for which a second attribute of the plurality of attributes has a second value, and a third ratio of data for which the first attribute has the first value from a second group for which the second attribute has a third value; and evaluating fairness relating to a third group for which the second attribute has the second value and a third attribute of the plurality of attributes has a fourth value, if a fourth ratio of data for which the first attribute has the first value from the third group satisfies a criterion based on the first ratio, the second ratio, and the third ratio.

Description

[Technical Field]

[0001] The present invention relates to a fairness evaluation program, a fairness evaluation method, and an information processing device. [Background technology]

[0002] Machine learning models trained using biased data may produce unfair inferences, such as discriminatory results. Bias is, for example, a bias in protected attributes such as gender. Protected attributes include gender, race, skin color, nationality, and age.

[0003] To evaluate the fairness of data, the fairness may be evaluated in subgroups contained in the data, where the subgroups are defined by a combination of the protection attribute and other attributes. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Pedreschi, D., Ruggieri, S. and Turini, F.: Discrimination-aware data mining, Proc. 14th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp.560-568, 2008 [Non-patent document 2] KAMIRAN, Faisal; ZLIOBAITE, Indre; CALDERS, Toon. Quantifying explainable discrimination and removing illegal discrimination in automated decision making. Knowledge and information systems, 2013. Summary of the Invention [Problem to be solved by the invention]

[0005] However, the number of subgroups increases as the number of attributes to be combined increases. When fairness is evaluated for all subgroups, the processing load of the calculation process also increases as the number of subgroups increases.

[0006] On the other hand, a method can be considered in which the fairness evaluation of subgroups within a subgroup that can be explained as fair is omitted and processing is terminated. However, there is a risk that the subgroups within a subgroup that can be explained as fair may contain potential fairness risks.

[0007] As such, the above-mentioned methods may make it difficult to reduce the computational load required to evaluate the fairness of data used for training in machine learning.

[0008] In one aspect, an object of the present invention is to reduce the computational load required to evaluate the fairness of data used for training in machine learning. [Means for solving the problem]

[0009] In one aspect, the fairness evaluation program may cause a computer to perform the following processes. The processes may include acquiring a plurality of data. The processes may include identifying a first proportion of data in which a first attribute of a plurality of attributes of the plurality of data is a first value. The processes may include identifying a second proportion of data in a first group in which a second attribute of the plurality of attributes is a second value, in which the first attribute is the first value, and a third proportion of data in a second group in which the second attribute is a third value, in which the first attribute is the first value. The processes may include performing a fairness evaluation on the third group in which the second attribute is the second value and a third attribute of the plurality of attributes is a fourth value, if a fourth proportion of data in which the first attribute is the first value satisfies a criterion based on the first proportion, the second proportion, and the third proportion. [Effects of the Invention]

[0010] In one aspect, the computational load for evaluating the fairness of data used for training in machine learning can be reduced. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram for explaining the fairness of the inference results of a machine learning model. [Figure 2] FIG. 2 is a diagram illustrating an example of data to be processed by the fairness evaluation device. [Figure 3] FIG. 10 is a diagram illustrating an example of a criterion for determining whether or not to perform a fairness evaluation by the fairness evaluation device. [Figure 4] FIG. 10 is a diagram illustrating another example of data to be processed by the fairness evaluation device. [Figure 5] FIG. 5 is a diagram showing an example of a plurality of subgroups in the data shown in FIG. 4. [Figure 6] 1 is a block diagram illustrating an example of a functional configuration of a fairness evaluation device according to an embodiment. [Figure 7]10 is a flowchart illustrating an example of the operation of the fairness evaluation device according to an embodiment. [Figure 8] 1 is a block diagram showing an example of the hardware (HW) configuration of a computer that realizes the functions of a fairness evaluation device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude various modifications or application of techniques not explicitly described below. For example, the present embodiment can be implemented with various modifications within the scope of the spirit thereof. In the drawings used in the following description, parts with the same reference numerals represent the same or similar parts unless otherwise specified.

[0013] [A] Description of fairness evaluation process according to one embodiment A method for evaluating the fairness of training data will be described below by taking a fairness evaluation device 1 (see FIG. 6) according to an embodiment as an example.

[0014] FIG. 1 is a diagram illustrating the fairness of inference results of a machine learning model. When a machine learning model is trained using a dataset containing unfair bias as training data, the trained machine learning model may output unfair inference results, such as inference results that cause unfair discrimination. The training data is an example of multiple data to be processed by the fairness evaluation device 1. Discrimination includes discrimination regarding protected attributes.

[0015] Protected attributes are sometimes called sensitive attributes. Examples of protected attributes include at least one of the following attributes: race, color, national origin, religion, sex, familial status, and disability status, as defined by the Fair Housing Act (FHA) in the United States. Other examples of protected attributes include at least one of the following attributes as defined by the Credit Card Nondiscrimination Act (ECNDA). O A) At least one attribute such as marital status, whether or not receiving public assistance, and age is included.

[0016] In order to prevent the machine learning model from outputting unfair inference results, the fairness evaluation device 1 performs a fairness evaluation on whether or not the training data contains bias regarding a specific protected attribute such as gender.

[0017] The fairness evaluation device 1 may generate training data for machine learning from which unfair bias has been removed by processing a portion of the training data based on the fairness evaluation result. This allows the original training data containing bias to be corrected, for example, expanded, into appropriate training data that does not contain bias. The fairness evaluation device 1 may use the training data generated in this way to train a machine learning model.

[0018] 2 is a diagram showing an example of data 111a to be processed by the fairness evaluation device 1. The fairness evaluation device 1 acquires the data 111a and protection attribute information from a device (not shown) that provides the data.

[0019] The data 111a is an example of a plurality of data (data sets) acquired by the fairness evaluation device 1. The data 111a may be processed to generate training data used for training a machine learning model. The data 111a includes a plurality of attributes. Each of the plurality of attributes may be a protected attribute or an unprotected attribute.

[0020] The protected attribute information is information indicating attributes that are protected attributes among a plurality of attributes. For example, the protected attribute information may be predetermined based on external knowledge such as laws. The non-protected attribute means an attribute other than the protected attribute among a plurality of attributes.

[0021] The fairness evaluation device 1 determines a first attribute (first attribute) from among a plurality of attributes included in the data 111a, and also determines a second attribute (second attribute) based on the protected attribute information.

[0022] The first attribute is, for example, a dependent variable. In the example of FIG. 2, the first attribute is an attribute indicating whether an applicant to a certain university passed or failed. The second attribute is, for example, one of the protected attributes. In FIG. 2, the second attribute is "gender." Note that attributes other than dependent variables, such as the second attribute, may also be called explanatory variables.

[0023] In one embodiment, among the values ​​of the first attribute (e.g., pass / fail), a value indicating "pass" may be positive, i.e., a positive example, and is an example of the first value. The fairness evaluation device 1 may, for example, identify the ratio of the first value to the multiple values ​​of the first attribute included in the data 111a (first ratio, positive example ratio). Note that the ratio of the first value may be included in the data 111a. As shown by reference symbol A1 in FIG. 2, the first ratio is "pass rate": 30% (0.3).

[0024] In one embodiment, as indicated by symbol A2 in Fig. 2, among the values ​​of the second attribute (e.g., gender), a value indicating "male" is an example of a second value, and a value indicating "female" is an example of a third value. The data 111a may include a first group 31a in which the second attribute has the second value, and a second group 31b in which the second attribute has the third value. The first group 31a is an example of a first group, and the second group 31b is an example of a second group.

[0025] The fairness evaluation device 1 identifies the proportion (positive example proportion) of the first group 31a in which the first attribute has the first value (in FIG. 2, "pass") as the second proportion. The fairness evaluation device 1 also identifies the proportion of the second group 31b in which the first attribute has the first value as the third proportion. In the example of FIG. 2, the second proportion is 36% (0.36), and the third proportion is 24% (0.24).

[0026] In Figure 2, when focusing on the pass rate for all faculties, there is a difference between the proportion of positive cases (pass rate) in the objective variable between men and women. Therefore, data 111a appears to contain a bias in the protected attribute of gender. However, when data 111a is divided into subgroups as shown below, it is clear that data 111a does not contain a bias in the protected attribute.

[0027] 2 shows subgroups 32 and 33 obtained by grouping the first group 31a and the second group 31b by a third attribute (e.g., "faculty"). The value of the third attribute (e.g., "faculty") of subgroup 32 is "pharmacy," and the value of the third attribute (e.g., "faculty") of subgroup 33 is "computer science."

[0028] Subgroup 32 may include subgroup 32a where the value of the second attribute is "male" and subgroup 32b where the value of the second attribute is "female." Subgroup 33 may include subgroup 33a where the value of the second attribute is "male" and subgroup 33b where the value of the second attribute is "female."

[0029] Each of the subgroups 32a and 33a is an example of a third group (third group) in which the second attribute (gender) is a second value (e.g., male) and the third attribute (e.g., "faculty") is a fourth value. Note that the second value of the second attribute ("gender") may be "female" and the third value may be "male." In this case, each of the subgroups 32b and 33b is an example of a third group in which the second attribute (gender) is a second value (e.g., female) and the third attribute (e.g., "faculty") is a fourth value.

[0030] The fairness evaluation device 1 may identify the proportion of the plurality of third groups in which the first attribute has the first value ("Pass" in FIG. 2) as the fourth proportion. In FIG. 2, the fourth proportion in the subgroup 32a is 20% (0.2), and the fourth proportion in the subgroup 33a is 40% (0.4). The fourth proportion in the subgroup 32b is 20% (0.2), and the fourth proportion in the subgroup 33b is 40% (0.4).

[0031] When the data 111a is divided into subgroups 32 and 33 in this way, there is no difference between the genders in the proportion of positive cases in each of the subgroups 32 and 33, and it can be said that the data 111a does not contain bias in the protection attributes.

[0032] Here, the fairness evaluation device 1 according to one embodiment determines whether to perform a fairness evaluation of the third group based on whether a fourth proportion of data in which the first attribute is a positive example among the plurality of third groups satisfies a criterion based on the first proportion, the second proportion, and the third proportion.

[0033] For example, the fairness evaluation device 1 performs a fairness evaluation for a subgroup of the plurality of subgroups whose fourth ratio satisfies the criterion. On the other hand, the fairness evaluation device 1 suppresses, for example, omits (does not perform) the evaluation of fairness for a subgroup of the plurality of subgroups whose fourth ratio does not satisfy the criterion.

[0034] Fig. 3 is a diagram showing an example of a criterion for determining whether or not to perform fairness evaluation by the fairness evaluation device 1. The vertical axis in Fig. 3 corresponds to the positive example rate.

[0035] In FIG. 3, the parameters C0, C1, and C2 may be calculated based on the following formulas (1) to (3). C0=P0(1) C1=P0+(|P1-P2|) / 2 (2) C2=P0-(|P1-P2|) / 2 (3)

[0036] In the above formulas (1) to (3), P0 represents the first ratio P0(Y = 1). Y is the target variable, and Y = 1 means that the first attribute shows the first value.

[0037] In the above formulas (2) and (3), P1 represents the second ratio P1(Y = 1|A = a1), and P2 represents the third ratio P2(Y = 1|A = a2). A = a1 means that the second attribute shows the second value. A = a2 means that the second attribute shows the third value (the second value ≠ the third value). Also, Y = 1|A = a1 means Y = 1 and A = a1, and Y = 1|A = a2 means Y = 1 and A = a2.

[0038] In the example shown in FIG. 2, the first ratio P0(Y = 1) = 0.3, the second ratio P1(Y = 1|A = a1) = 0.36, and the third ratio P2(Y = 1|A = a2) = 0.24. Therefore, C0 = 0.3, C1 = 0.3+(0.36 - 0.24) / 2 = 0.36, and C2 = 0.3-(0.36 - 0.24) / 2 = 0.24.

[0039] Note that the calculation of the parameters C0 to C2 based on the above formulas (1) to (3) may be considered to be performed based on the calculation of the numerical range C based on the following formula (4). Numerical range C = |P1 - P2| (4)

[0040] For example, the fairness evaluation device 1 may calculate the numerical range C based on the above formula (4), and calculate the upper limit value C1 and the lower limit value C2 of the numerical range C so that the parameter C0 of the above formula (1) is included (as an example, it becomes the median) within the range of the numerical range C.

[0041] In one embodiment, the fairness evaluation device 1 may determine that the fourth ratio meets the standard when the fourth ratio > C1 or the fourth ratio < C2, in other words, when the fourth ratio is outside the range of the numerical range C.

[0042] For example, among each of a plurality of subgroups (the third group), for a subgroup in which a fourth ratio (positive example ratio), which is the ratio of data where the first attribute is the first value, satisfies the criterion of "fourth ratio > C1 or fourth ratio < C2", a fairness evaluation is performed. On the other hand, among each of a plurality of subgroups (the third group), for a group in which the fourth ratio (positive example ratio) is "C1 ≧ fourth ratio ≧ C2", the execution of the fairness evaluation is suppressed.

[0043] Here, the first ratio P0(Y = 1) is an example of an index related to the estimated distribution of the positive example ratio in the subgroup. In this example, it is assumed that the estimated distribution of the positive example ratio in the subgroup follows a normal distribution, and it is assumed that the center of the normal distribution is the first ratio P0(Y = 1), which is the overall positive example ratio of the objective function.

[0044] According to the above assumption, while the positive example ratios of each subgroup are concentrated around the center of the normal distribution, the positive example ratios of subgroups containing unfair biases are assumed to be distributed away from the center of the normal distribution. Thus, for subgroups where the estimated distribution of the positive example ratio for each subgroup exists near the center, it is assumed that there is no unfair bias, so the omission of the fairness evaluation execution is allowed. Therefore, by suppressing the execution of the fairness evaluation for such subgroups, the computational processing load for evaluating the fairness of the data can be reduced.

[0045] As shown in the above formula (4), the second ratio P1 and the third ratio P2 used for calculating the numerical range C (the difference between the parameters C1 and C2) are an example of an index indicating the statistic of the bias occurring between the values of the protection attribute of interest (for example, between men and women). The larger the difference between the second ratio P1(Y = 1|A = a1) and the third ratio P2(Y = 1|A = a2), the larger the numerical range C becomes, and thus the number of subgroups for which the fairness evaluation is performed decreases. Thus, the larger the difference between the second ratio P1 and the third ratio P2, the more the effect of reducing the computational processing load can be enhanced.

[0046] 2 shows an example in which the third group is a group in which the second attribute is a second value (for example, gender is male) and another attribute (third attribute, for example, the department taken) has a certain value (fourth value, for example, the department of pharmacy). However, the other attribute (third attribute) is not limited to a single attribute, and may be a combination of multiple attributes.

[0047] 4 is a diagram showing another example of data 111b to be processed by the fairness evaluation device 1. The data 111b is another example of a plurality of data (data sets) to be acquired. The data 111b is an example of training data used for training a machine learning model.

[0048] 4 is an example of a data 111b that indicates, as a first attribute, whether an applicant is hired or not hired by a certain company. The data 111b also includes, as a second attribute (protected attribute), the attribute of gender. Furthermore, the data 111b may include, as attributes (explanatory variables), marital status (single or married) and the type of employment applied for (full-time or part-time).

[0049] Fig. 5 is a diagram showing an example of a plurality of subgroups in the data 111b shown in Fig. 4. As indicated by reference symbol B1 in Fig. 5, the data 41 (all data 111b) indicating the first attribute (hiring success or failure) is divided into a first group 41a in which the second attribute (for example, "gender") has a second value (male) and a second group 41b in which the second attribute has a third value (female).

[0050] Furthermore, an "adoption rate" is identified as a first rate P0 (proportion of positive cases) of a first value (e.g., "adoption") of the first attribute (adoption pass / fail) from the data 41. In the example of FIG. 5, the first rate P0 is 36% (0.36).

[0051] 5, DI (Disparate Impact) is a fairness evaluation index and an example of a fairness index. DI is the ratio of the proportion of positive examples in a second group having a second value in a second attribute to a third group having a third value in a second attribute. In one embodiment, the fairness evaluation device 1 may perform fairness evaluation using DI.

[0052] The DI for the second group (e.g., males) may be calculated by dividing the positive case rate of the second group by the positive case rate of the third group, and the DI for the third group (e.g., females) may be calculated by dividing the positive case rate of the third group by the positive case rate of the second group.

[0053] In the example of Figure 5, the second proportion P1 (adoption rate) of the first group 41a in which the first attribute is the first value (adopted) is 51% (0.51), and the third proportion P2 of the second group 41b in which the first attribute is the first value is 22% (0.22).

[0054] In the case of Figure 5, based on the overall statistics (see reference numeral 41c) combining males and females, the parameters of the criteria described in Figure 3 are C0 = 0.36, C1 = 0.36 + (0.51 - 0.22) / 2 = 0.51, and C2 = 0.36 - (0.51 - 0.22) / 2 = 0.22. Therefore, in the example of Figure 5, subgroups with a positive case rate > 0.51 or a positive case rate < 0.22 are targets for fairness evaluation.

[0055] Also, symbol B2 in Fig. 5 indicates subgroups 42 to 45 obtained by dividing the first group 41a and the second group 41b by a third attribute (e.g., "employment type" and "marital status"). Subgroup 42 has a value of "part-time" (indicated as "part-time" in Fig. 5) for the third attribute (e.g., "employment type"), and subgroup 43 has a value of "full-time" (indicated as "regular" in Fig. 5) for the third attribute (e.g., "employment type"). Subgroup 44 has a value of "single" for the third attribute (e.g., "marital status"), and subgroup 45 has a value of "married" for the third attribute (e.g., "marital status").

[0056] Furthermore, reference numeral B3 in FIG. 5 indicates subgroups 46 to 51 which are classified based on a “composite attribute” that combines a plurality of attributes of “employment form” and “marital status” into subgroups 42 to 45. Subgroups 46 and 48 have a “composite attribute” value of “part-time and single”, and subgroups 47 and 50 have a “composite attribute” value of “part-time and married”. Subgroup 49 has a “composite attribute” value of “single and regular”, and subgroup 51 has a “composite attribute” value of “married and regular”. In FIG. 5, “and” is represented by the multiplication symbol “×”.

[0057] In FIG. 5, each of the “employment form”, “marital status” indicated by reference numerals B2 and B3, and the “composite attribute” combining the employment form and the marital status is an example of the third attribute.

[0058] Subgroups 42 to 51 may include a subgroup in which the value of the second attribute is “male” and a subgroup in which the value of the second attribute is “female”. Among subgroups 42 to 51, a subgroup in which the value of the second attribute is the second value (for example, “male”) is denoted as subgroups 42a to 51a, and a subgroup in which the value of the second attribute is the third value (for example, “female”) is denoted as subgroups 42b to 51b. Subgroups 42a to 51a are an example of the third group.

[0059] Consider the case where the fairness evaluation device 1 in the example of FIG. 5 identifies the execution target of the fairness evaluation. The fairness evaluation device 1 determines for each subgroup whether the positive example ratio (adoption rate) of the subgroup is > C1 (0.51) or < C2 (0.22). In the example of FIG. 5, the fairness evaluation device 1 identifies subgroups 42 to 46, 47a, 48, 49, 50a, and 51 whose positive example ratios are outside the range of C1 to C2 (numerical range C) as the execution targets of the fairness evaluation. On the other hand, the fairness evaluation device 1 decides to suppress the execution of the fairness evaluation for subgroups 47b and 50b whose positive example ratios are within the range of C1 to C2 (numerical range C).

[0060] In the example shown by reference symbol B2 in FIG. 5, there is a large difference in DI between the first group 41a and the second group 41b. However, in subgroups 42 to 45, the DI values ​​are similar between males and females. In this case, the difference in the first attribute according to the second attribute in subgroups 42 to 45 (e.g., the difference in the number of successful applicants) may be evaluated as explainable discrimination (segregation) using conventional methods. If this is evaluated as explainable discrimination (segregation), there is a possibility that fairness evaluation will not be performed on the lower-level subgroups 46 to 51 (see reference symbol B3).

[0061] However, in the example shown by reference symbol B3 in Figure 5, it can be seen that there is an unfair bias in subgroups 46 to 51 in the lower hierarchical levels because there is a difference in DI between men and women. This is an example of "Simpson's paradox," which states that even if a hypothesis holds for a certain group, it does not necessarily hold for the groups below it. In such cases, conventional methods do not perform fairness assessments or data processing or other corrections for subgroups 46 to 51 containing unfair bias, so there is a possibility that bias remains in data 111b.

[0062] In contrast, according to one embodiment, whether or not to perform fairness evaluation is determined for each subgroup divided by a composite attribute having a combination length longer than the combination length of attributes of subgroups 42 to 45 that can be evaluated as an explainable discrimination (distinction). Note that the "combination length" may refer to the number of attributes to be combined. As a result, in the example shown in FIG. 5, the fairness evaluation device 1 can perform fairness evaluation on subgroups 46, 47a, 48, 49, 50a, and 51 that meet the criteria for performing evaluation.

[0063] [B] Functional configuration example 6 is a block diagram showing an example of the functional configuration of a fairness evaluation device 1 according to an embodiment. The fairness evaluation device 1 is an example of an information processing device or computer that executes a fairness evaluation process that evaluates the fairness of data. For example, the fairness evaluation device 1 may execute the fairness evaluation process and expand the training data used to train a machine learning model by processing the training data based on the execution result. This makes it possible to suppress the output of unfair inference results by the machine learning model.

[0064] 6, the fairness evaluation device 1 may include, for example, a memory unit 11, an acquisition unit 12, a protection attribute determination unit 13, a bias calculation unit 14, a subgroup distribution estimation unit 15, an execution standard calculation unit 16, a subgroup search unit 17, a positive example ratio calculation unit 18, a determination unit 19, a fairness index calculation unit 20, and a data processing unit 21. The fairness evaluation device 1 may also include a machine learning unit 22, and may further include an inference processing unit 23. These blocks 12 to 23 are examples of a control unit 24.

[0065] The memory unit 11 is an example of a storage area, and stores various data used by the fairness evaluation device 1. The memory unit 11 may be realized, for example, by a storage area included in one or both of the memory 10b and the storage unit 10c shown in FIG. 8, which will be described later.

[0066] 6, the memory unit 11 may be capable of storing, for example, data 111, protected attribute information 112, and training data 113 (processed data). Furthermore, when the fairness evaluation device 1 includes a machine learning unit 22, the memory unit 11 may be capable of storing a machine learning model 114. Furthermore, when the fairness evaluation device 1 includes an inference processing unit 23, the memory unit 11 may be capable of storing an inference result 115.

[0067] The information stored in the memory unit 11 may be in a table format or in another format. In one example, at least one of the pieces of information stored in the memory unit 11 may be in various formats such as a DB or an array.

[0068] The acquiring unit 12 acquires various information used in the fairness evaluation device 1. For example, the acquiring unit 12 may acquire data 111 and protection attribute information 112 from a device (not shown) that provides the data, and store them in the memory unit 11.

[0069] The data 111 is data including a plurality of attributes and is an example of training data. Each of the plurality of attributes may be a protected attribute or an unprotected attribute. For example, the data 111 may be the data 111a shown in FIG. 2 or the data 111b shown in FIG. 4.

[0070] The protection attribute information 112 is information for identifying (for example, specifying) a protection attribute from among a plurality of attributes included in the data 111.

[0071] The protection attribute determination unit 13 determines a protection attribute from among a plurality of attributes included in the data 111 based on the protection attribute information 112 .

[0072] The bias calculation unit 14 calculates the bias in the proportion of positive cases between a plurality of values ​​(for example, between male and female) in the determined protected attribute.

[0073] For example, the bias calculation unit 14 identifies the second ratio P1 (Y=1|A=a1) and the third ratio P2 (Y=1|A=a2), and calculates the bias occurring in the attribute of interest (in one example, the protected attribute) based on the second ratio P1 and the third ratio P2. The bias is an example of the numerical range C calculated based on the above formula (4).

[0074] In one example, the bias calculation unit 14 calculates a statistical parity difference. The statistical parity difference is expressed as the difference in the proportion of positive examples between groups having different values ​​in an attribute of interest. The statistical parity difference is an example of a fairness index.

[0075] The subgroup distribution estimation unit 15 identifies a first proportion P0(Y=1) of data in which a first attribute of the multiple attributes of the data 111 has a first value. For example, the subgroup distribution estimation unit 15 may identify the overall proportion of positive examples in the first attribute (class corresponding to the objective variable) as the first proportion P0(Y=1).

[0076] For example, the subgroup distribution estimation unit 15 may identify the overall proportion of positive examples by estimating the distribution of the proportion of positive examples for each of the multiple subgroups included in the data 111. In one example, the subgroup distribution estimation unit 15 may estimate the proportion of positive examples that indicates a peak in the distribution of the proportion of positive examples for each of the multiple subgroups.

[0077] The execution criterion calculation unit 16 calculates a criterion for selecting a subgroup for which fairness evaluation is to be performed, based on the first rate P0, the second rate P1, and the third rate P2.

[0078] For example, the performance criterion calculation unit 16 may calculate the parameters C0, C1, and C2 based on the above formulas (1) to (3) or (1) and (4) as the criteria for performing the fairness evaluation.

[0079] The subgroup search unit 17 searches for a plurality of subgroups of the data 111. For example, the subgroup search unit 17 may identify a plurality of subgroups each including at least one protected attribute while combining a plurality of attributes (explanatory variables) included in the data 111. The upper limit of the combination length, which is the number of attributes to be combined, may be determined in advance based on the expected amount of calculation depending on the content of the data 111, for example. Note that the subgroup search method may be a breadth search or a depth search.

[0080] The positive example proportion calculation unit 18 obtains the positive example proportion for each of the searched subgroups. For example, the positive example proportion calculation unit 18 may calculate the positive example proportion for each of the subgroups by value of the protection attribute. The positive example proportion in each subgroup is an example of the fourth proportion of data in the third group in which the first attribute has the first value.

[0081] The determination unit 19 determines whether the positive example ratio (the fourth ratio) calculated for each of the plurality of subgroups satisfies the criterion. In one example, the determination unit 19 determines that the criterion is satisfied when the calculated positive example ratio is positive example ratio > C1 or positive example ratio < C2.

[0082] The fairness index calculation unit 20 performs an evaluation of fairness for the subgroup determined by the determination unit 19 to satisfy the criterion. The subgroup determined to perform the evaluation of fairness is an example of a third group in which the second attribute is the second value and the third attribute of the plurality of attributes is the fourth value.

[0083] Note that the fairness index calculation unit 20 may specify the fifth ratio of the data in which the first attribute is the first value among the fourth group in which the second attribute is the third value (in one example, the gender is female) and the third attribute of the plurality of attributes is the fourth value. In other words, the fairness index calculation unit 20 may specify the fifth ratio regarding the fourth group related to the third group determined to perform the fairness evaluation, and the value of the second attribute, which is the protected attribute, is different from that of the third group.

[0084] Then, the fairness index calculation unit 20 may perform an evaluation of fairness based on the fourth ratio (in one example, the positive example ratio in males) and the fifth ratio (in one example, the positive example ratio in females).

[0085] The fairness index may be, for example, a statistical parity difference or a disparate impact (DI). The statistical parity difference may be based on the difference between the fourth ratio and the fifth ratio. The DI may be based on the ratio between the fourth ratio and the fifth ratio. However, the fairness index is not limited to these cases, and various indices may be used.

[0086] The data processing unit 21 processes a portion of the plurality of data 111 based on the fairness evaluation result to generate training data 113 for machine learning. The processing method may be any method that suppresses unfairness (discrimination) in the data 111 based on the fairness evaluation result, and may be realized by various known methods. In one example, the values ​​of unprotected attributes other than protected attributes may be rewritten in the data 111 to reduce the correlation between the protected attributes and the unprotected attributes.

[0087] As described above, the fairness evaluation device 1 may include the machine learning unit 22, and may further include the inference processing unit .

[0088] In the machine learning phase, the machine learning unit 22 executes a machine learning process to train the machine learning model 114 using the processed data, which has been processed by the data processing unit 21 so as to reduce discrimination, as training data 113. The machine learning model 114 may be a neural network (NN) model including trained parameters. The machine learning process may be realized by various known methods.

[0089] In the inference phase, the inference processing unit 23 performs inference processing using the machine learning model 114 trained based on the processed training data 113. For example, the inference processing unit 23 inputs target data (not shown) for the inference processing to the machine learning model 114, and stores the inference result 115 output from the machine learning model 114 in the memory unit 11.

[0090] [C] Example of operation Next, an example of the operation of the fairness evaluation device 1 according to an embodiment will be described. Fig. 7 is a flowchart illustrating an example of the operation of the fairness evaluation device 1 according to an embodiment.

[0091] As illustrated in FIG. 7, the acquisition unit 12 of the fairness evaluation device 1 acquires the data 111 and the protected attribute information 112 (step S1), and stores them in the memory unit 11.

[0092] The subgroup distribution estimation unit 15 identifies a first proportion P0 (Y=1) (step S2). The first proportion is the proportion of data in which the first attribute Y of interest is a first value (Y=1). The first proportion may be a positive case proportion, which is the proportion of positive cases in which the first attribute is positive.

[0093] The bias calculation unit 14 identifies a second proportion P1(Y=1|A=a1) (step S3). The second proportion P1(Y=1|A=a1) is the proportion of data in which the first attribute Y is the first value (Y=1) among the first group in which the second attribute A of the multiple attributes is the second value (A=a1).

[0094] The bias calculation unit 14 identifies a third proportion P2 (Y=1|A=a2) (step S4). The third proportion P2 (Y=1|A=a2) is the proportion of data in which the first attribute Y is the first value (Y=1) among the second group in which the second attribute A of the multiple attributes is the third value (A=a2).

[0095] The processing order of steps S1 to S3 is not limited to that shown in Fig. 7. For example, the processing of steps S1 to S3 may be performed in parallel.

[0096] The execution criteria calculation unit 16 calculates criteria C0, C1, and C2 for selecting a subgroup for performing fairness evaluation based on the first ratio P0 (Y=1), the second ratio P1 (Y=1|A=a1), and the third ratio P2 (Y=1|A=a2) (step S5).

[0097] In one embodiment, it is assumed that the distribution of the positive example proportions of the multiple subgroups is a normal distribution, but this is not limiting, and the values ​​of C0, C1, and C2 may be adjusted depending on the estimated distribution of the positive example proportions of the multiple subgroups. For example, C0 may be adjusted depending on the peak position of the distribution of the positive example proportions of the multiple subgroups, or C1 and C2 may be adjusted so that the numerical range between C1 and C2 becomes wider as the half-width of the distribution of the positive example proportions of the multiple subgroups becomes wider.

[0098] The subgroup search unit 17 searches for one subgroup included in the first group (step S6). In other words, the subgroup search unit 17 searches for a third group in which the second attribute is the second value (included in the first group) and the third attribute of the multiple attributes is the fourth value.

[0099] The positive example proportion calculation unit 18 identifies a fourth proportion (step S7). The fourth proportion is the proportion of data in the searched subgroup (third group) in which the first attribute Y is the first value (Y=1) (positive examples).

[0100] The determination unit 19 determines whether the fourth ratio determined in step S7 satisfies the criteria calculated in step S5 (step S8). If the fourth ratio does not satisfy the criteria (NO in step S8), the process proceeds to step S11.

[0101] If the fourth ratio satisfies the criterion (YES in step S8), the fairness index calculation unit 20 performs a fairness evaluation for the subgroups determined by the determination unit 19 to satisfy the criterion (step S9).

[0102] The data processing unit 21 processes a part of the plurality of data 111 based on the fairness evaluation result to generate training data 113 for machine learning (step S10). The processing method may be any method that suppresses unfairness (discrimination) in the data 111 based on the fairness evaluation result, and may be realized by various known methods.

[0103] The subgroup search unit 17 determines whether or not the search for the subgroups is completed (step S11). Note that the subgroup search unit 17 may determine whether or not the search is completed based on the upper limit of the combination length and the subgroups that have already been searched.

[0104] If the search for the subgroup has not been completed (NO in step S11), the processes of steps S6 to S11 are repeated.

[0105] When the search is completed (YES in step S11), the machine learning unit 22 uses the generated training data 113 to train the machine learning model 114 (step S12), and the process ends.

[0106] [D] Effects of one embodiment According to a method according to an embodiment, the fairness evaluation device 1 acquires a plurality of data 111 and identifies a first proportion P0 (Y=1) of data in which a first attribute of a plurality of attributes of the plurality of data 111 has a first value. The fairness evaluation device 1 also identifies a second proportion P1 (Y=1|A=a1) of data in which a second attribute of the plurality of attributes has a second value a1 and the first attribute has the first value. The fairness evaluation device 1 also identifies a third proportion P2 (Y=1|A=a2) of data in which the first attribute has the first value and the second attribute has the third value. The fairness evaluation device 1 also performs a fairness evaluation on a third group in which the second attribute has the second value and the third attribute of the plurality of attributes has a fourth value, if a fourth proportion of data in which the first attribute has the first value and the third attribute has the second value satisfies a criterion. The criterion is based on the first rate P0 (Y=1), the second rate P1 (Y=1|A=a1), and the third rate P2 (Y=1|A=a2).

[0107] This reduces the computational load for evaluating the fairness of data used to train a machine learning model, for example, by suppressing the execution of fairness evaluation for third groups that do not satisfy the criteria. For example, since the number of third groups increases as the number of attributes included in the data 111 increases, the computational load can be significantly reduced compared to when fairness evaluation is executed for all third groups.

[0108] An example of a reduction in the computational load when the fairness evaluation process of one embodiment is applied to open data related to people's thermal sensations will be described. The objective variable (Y) is whether or not the person felt cold. A positive example (Y=1) is when the person felt cold. The explanatory variables (attributes) are five attributes including the protected attribute "gender." The number of dimensions of the open data after binarization is 20.

[0109] For the above-described open data, if the fairness index is calculated for all combinations, the fairness index will be calculated for 21,699 subgroups. On the other hand, if the method of one embodiment is applied to the above-described open data, the fairness index will be calculated for 6,301 subgroups. Therefore, in the above-described example, the calculation of the fairness index for 15,398 subgroups can be omitted, thereby reducing the calculation processing load.

[0110] Furthermore, for example, when the above-mentioned "Simpson's Paradox" occurs, potential risks to fairness in the plurality of data 111 can be detected.

[0111] Furthermore, since a standard is set based on the first proportion P0, the second proportion P1, and the third proportion P2, it is possible to determine the subgroups that suppress the fairness evaluation by taking into account the distribution of the positive case proportions of the subgroups and the bias statistics.

[0112] The process of evaluating fairness also includes a process of calculating a numerical range C, which has a numerical range obtained based on the difference between the second ratio P1 and the third ratio P2 and includes the first ratio P0. If the fourth ratio is outside the numerical range C, the computer 10 determines that the criterion is met.

[0113] This allows us to omit the calculation of fairness evaluation for subgroups where the distribution of the positive example rate is concentrated near the overall positive example rate and the bias is relatively small. process The load reduction effect can be enhanced.

[0114] Furthermore, the process of performing the fairness evaluation identifies a fifth proportion of data in a fourth group in which the second attribute is a third value and the third attribute of the multiple attributes is a fourth value, in which the first attribute is the first value, and performs the fairness evaluation based on the fourth proportion and the fifth proportion.

[0115] This allows us to appropriately evaluate fairness by utilizing the fourth ratio used to determine the standard.

[0116] Furthermore, the fairness evaluation device 1 further processes a part of the plurality of data based on the fairness evaluation result to generate training data for machine learning.

[0117] This mitigates potential fairness risks in machine learning training data and prevents discriminatory inferences from machine learning models trained based on unfair biases.

[0118] The above method can be applied to fairness evaluation of training data for machine learning models used in education, medicine, communications, business, and other fields, and can be used in the design and auditing of AI.

[0119] [E] Hardware configuration example The fairness evaluation device 1 (see FIG. 6) according to one embodiment may be a virtual server (VM; Virtual Machine) or a physical server. The functions of the fairness evaluation device 1 may be realized by one computer or by two or more computers. Furthermore, at least some of the functions of the fairness evaluation device 1 may be realized using HW (Hardware) resources and NW (Network) resources provided by a cloud environment.

[0120] 8 is a block diagram showing an example of a hardware (HW) configuration of a computer 10 that realizes the functions of the fairness evaluation device 1 according to an embodiment. When multiple computers are used as HW resources that realize the functions of the fairness evaluation device 1, each computer may have the HW configuration exemplified in FIG.

[0121] As shown in FIG. 8, the computer 10 may include, as a HW configuration, a processor 10a, a memory 10b, a storage unit 10c, an IF (Interface) unit 10d, an IO (Input / Output) unit 10e, and a reading unit 10f, for example.

[0122] The processor 10a is an example of a processing unit that performs various controls and calculations. The processor 10a may be connected to each block in the computer 10 via a bus 10i so that they can communicate with each other. The processor 10a may be a multiprocessor including multiple processors, a multi-core processor having multiple processor cores, or a configuration having multiple multi-core processors.

[0123] The processor 10a may be, for example, an integrated circuit (IC) such as a CPU, MPU, GPU, APU, DSP, ASIC, or FPGA. Note that the processor 10a may be a combination of two or more of these integrated circuits. CPU is an abbreviation for Central Processing Unit, MPU is an abbreviation for Micro Processing Unit, GPU is an abbreviation for Graphics Processing Unit, and APU is an abbreviation for Accelerated Processing Unit. DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0124] For example, when the fairness evaluation device 1 executes a machine learning process or an inference process in addition to the fairness evaluation process according to an embodiment, the processor 10a may be a combination of a processing device such as a CPU that executes the fairness evaluation process and an accelerator that executes the machine learning process or the inference process. Examples of the accelerator include the above-mentioned GPU, APU, DSP, ASIC, and FPGA.

[0125] The memory 10b is an example of HW that stores various types of data, programs, and other information. Examples of the memory 10b include one or both of a volatile memory such as a dynamic random access memory (DRAM) and a non-volatile memory such as a persistent memory (PM).

[0126] The storage unit 10c is an example of HW that stores various types of data, programs, and other information. Examples of the storage unit 10c include various storage devices such as a magnetic disk device such as a hard disk drive (HDD), a semiconductor drive device such as a solid state drive (SSD), and a nonvolatile memory. Examples of nonvolatile memory include a flash memory, a storage class memory (SCM), and a read-only memory (ROM).

[0127] The storage unit 10c may store a program 10g (fairness evaluation program) that realizes all or part of the various functions of the computer 10.

[0128] For example, the processor 10a of the fairness evaluation device 1 can realize the function of the fairness evaluation device 1 (the control unit 24 illustrated in FIG. 6) by expanding the program 10g stored in the storage unit 10c into the memory 10b and executing it.

[0129] The IF unit 10d is an example of a communication IF that controls connections and communications between various networks including a network between the fairness evaluation device 1 and devices not shown. Examples of the devices include a computer such as a user terminal or a server that provides data to the fairness evaluation device 1, and a computer such as a server that performs machine learning processing based on data output from the fairness evaluation device 1.

[0130] For example, the IF unit 10d may include an adapter that complies with a LAN (Local Area Network) such as Ethernet (registered trademark) or optical communication such as FC (Fibre Channel). The adapter may be compatible with either or both of wireless and wired communication methods.

[0131] The program 10g may be downloaded to the computer 10 from a network via the communication IF and stored in the storage unit 10c.

[0132] The IO unit 10e may include one or both of an input device and an output device. Examples of input devices include a keyboard, a mouse, and a touch panel. Examples of output devices include a monitor, a projector, and a printer. The IO unit 10e may also include a touch panel that combines an input device and a display device.

[0133] The reading unit 10f is an example of a reader that reads data and program information recorded on the recording medium 10h. The reading unit 10f may include a connection terminal or device to which the recording medium 10h can be connected or inserted. Examples of the reading unit 10f include an adapter that complies with USB (Universal Serial Bus) or the like, a drive device that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. Note that the recording medium 10h may store the program 10g, and the reading unit 10f may read the program 10g from the recording medium 10h and store it in the memory unit 10c.

[0134] Examples of the recording medium 10h include non-transitory computer-readable recording media such as magnetic / optical disks and flash memories. Examples of magnetic / optical disks include flexible disks, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs, and HVDs (Holographic Versatile Discs). Examples of flash memories include semiconductor memories such as USB memories and SD cards.

[0135] The above-described HW configuration of the computer 10 is an example. Therefore, the HW in the computer 10 may be increased or decreased (for example, adding or deleting any block), divided, integrated in any combination, or the HW may be added or deleted as needed.

[0136] [F] Other The technology according to the embodiment described above can be implemented with the following modifications and variations.

[0137] For example, the blocks 12 to 21 (as well as the machine learning unit 22 and the inference processing unit 23) included in the fairness evaluation device 1 shown in FIG. 6 may be merged in any combination, or may be divided into individual blocks.

[0138] 6 may be a configuration (system) in which a plurality of devices cooperate with each other via a network to realize each processing function. As an example, the memory unit 11 may be a DB server, the acquisition unit 12 may be a Web server or an application server, and the blocks 13 to 23 may be application servers, etc. In this case, the DB server, the application server, and the Web server may cooperate with each other via a network to realize each processing function of the fairness evaluation device 1. [Explanation of symbols]

[0139] 1. Fairness evaluation device 10. Computers 10a processor 10b memory 10c storage section 10d IF section 10e IO section 10f Reading unit 10h storage medium 10g Programs 11 Memory section 12 Acquisition Department 13 Protection attribute determination section 14 Bias calculation section 15 Subgroup distribution estimation section 16 10 Kuo Standard Calculation Department 17 Subgroup Search Section 18 Positive example rate calculation section 19 Judgment section 20 Fairness index calculation section 21 Data Processing Department 22 Machine Learning Department 23 Inference processing unit 24 Control Unit

Claims

1. Get multiple data Identifying a first ratio of data among the plurality of data items, in which a first attribute of a plurality of attributes of the plurality of data items has a first value; identifying a second ratio of data in a first group in which a second attribute of the plurality of attributes is a second value, the first attribute of which has the first value, and a third ratio of data in a second group in which the second attribute is a third value, the first attribute of which has the first value; performing a fairness evaluation on a third group when a fourth proportion of data in a third group, in which the second attribute is the second value and a third attribute of the plurality of attributes is a fourth value, in which the first attribute is the first value satisfies a criterion based on the first proportion, the second proportion, and the third proportion; A fairness evaluation program that runs the process on a computer.

2. The process of performing the fairness evaluation includes: calculating a numerical range having a numerical range obtained based on the difference between the second ratio and the third ratio and including the first ratio; determining that the criterion is satisfied when the fourth ratio is within the numerical range; The fairness evaluation program according to claim 1 .

3. The process of performing the fairness evaluation includes: identifying a fifth percentage of data in a fourth group in which the second attribute is the third value and a third attribute of the plurality of attributes is a fourth value, the fifth percentage being in which the first attribute is the first value; performing the fairness assessment based on the fourth ratio and the fifth ratio. The fairness evaluation program according to claim 1 .

4. generating training data for machine learning by processing a part of the plurality of data based on the fairness evaluation result; causing the computer to execute a process; The fairness evaluation program according to any one of claims 1 to 3.

5. Get multiple data Identifying a first ratio of data among the plurality of data items, in which a first attribute of a plurality of attributes of the plurality of data items has a first value; identifying a second ratio of data in a first group in which a second attribute of the plurality of attributes is a second value, the first attribute of which has the first value, and a third ratio of data in a second group in which the second attribute is a third value, the first attribute of which has the first value; performing a fairness evaluation on a third group when a fourth proportion of data in a third group, in which the second attribute is the second value and a third attribute of the plurality of attributes is a fourth value, in which the first attribute is the first value satisfies a criterion based on the first proportion, the second proportion, and the third proportion; A fairness evaluation method in which processing is performed by a computer.

6. Acquire multiple pieces of data, Identifying a first ratio of data among the plurality of data items, in which a first attribute of a plurality of attributes of the plurality of data items has a first value; identifying a second ratio of data in a first group in which a second attribute of the plurality of attributes is a second value, the first attribute of which has the first value, and a third ratio of data in a second group in which the second attribute is a third value, the first attribute of which has the first value; performing a fairness evaluation on a third group when a fourth proportion of data in a third group, in which the second attribute is the second value and a third attribute of the plurality of attributes is a fourth value, in which the first attribute is the first value satisfies a criterion based on the first proportion, the second proportion, and the third proportion; An information processing device comprising a control unit.

Images