Fairness evaluation program, fairness evaluation method, and information processing device

Abstract

This fairness evaluation program causes a computer to execute processes of: acquiring a plurality of pieces of data; identifying a first ratio of data, from the plurality of pieces of data, for which a first attribute of a plurality of attributes of the plurality of pieces of data has a first value; identifying a second ratio of data for which the first attribute has the first value from a first group for which a second attribute of the plurality of attributes has a second value, and a third ratio of data for which the first attribute has the first value from a second group for which the second attribute has a third value; and evaluating fairness relating to a third group for which the second attribute has the second value and a third attribute of the plurality of attributes has a fourth value, if a fourth ratio of data for which the first attribute has the first value from the third group satisfies a criterion based on the first ratio, the second ratio, and the third ratio.

Description

Fairness evaluation program, fairness evaluation method, and information processing device

[0001] The present invention relates to a fairness evaluation program, a fairness evaluation method, and an information processing device.

[0002] Machine learning models trained using biased data may produce unfair inferences, such as discriminatory results. Bias is, for example, a bias in protected attributes such as gender. Protected attributes include gender, race, skin color, nationality, and age.

[0003] To evaluate the fairness of data, the fairness may be evaluated in subgroups contained in the data, where the subgroups are defined by a combination of the protection attribute and other attributes.

[0004] Pedreschi, D., Ruggieri, S. and Turini, F.: Discrimination-aware data mining, Proc. 14th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp.560-568, 2008KAMIRAN, Faisal; ZLIOBAITE, Indre; CALDERS, Toon. Quantifying explainable discrimination and illegal removing discrimination in automated decision making. Knowledge and information systems, 2013.

[0005] However, the number of subgroups increases as the number of attributes to be combined increases. When fairness is evaluated for all subgroups, the processing load of the calculation process also increases as the number of subgroups increases.

[0006] On the other hand, a method can be considered in which the fairness evaluation is omitted for subgroups that are lower in the subgroup that can be explained as fair, and the process is terminated. However, there is a risk that the lower subgroups that are lower in the subgroup that can be explained as fair may contain potential fairness risks.

[0007] As such, the above-mentioned methods may make it difficult to reduce the computational load required to evaluate the fairness of data used for training in machine learning.

[0008] In one aspect, an object of the present invention is to reduce the computational load required to evaluate the fairness of data used for training in machine learning.

[0009] In one aspect, the fairness evaluation program may cause a computer to perform the following processes. The processes may include acquiring a plurality of data. The processes may include identifying a first proportion of data, among the plurality of data, in which a first attribute of a plurality of attributes of the plurality of data is a first value. The processes may include identifying a second proportion of data, among a first group in which a second attribute of the plurality of attributes is a second value, in which the first attribute is the first value, and a third proportion of data, among a second group in which the second attribute is a third value, in which the first attribute is the first value. The processes may include performing a fairness evaluation on a third group in which the second attribute is the second value and a third attribute of the plurality of attributes is a fourth value, if a fourth proportion of data, among the third group in which the second attribute is the second value and a third attribute of the plurality of attributes is a fourth value, in which the first attribute is the first value satisfies a criterion based on the first proportion, the second proportion, and the third proportion.

[0010] In one aspect, the computational load for evaluating the fairness of data used for training in machine learning can be reduced.

[0011] 4 is a diagram for explaining the fairness of the inference results of a machine learning model. FIG. 5 is a diagram illustrating an example of data to be processed by the fairness evaluation device. FIG. 6 is a diagram illustrating an example of criteria for whether or not to perform fairness evaluation by the fairness evaluation device. FIG. 7 is a diagram illustrating another example of data to be processed by the fairness evaluation device. FIG. 8 is a diagram illustrating an example of a plurality of subgroups in the data shown in FIG. 4. FIG. 9 is a block diagram illustrating an example of the functional configuration of a fairness evaluation device according to an embodiment. FIG. 10 is a flowchart illustrating an example of the operation of a fairness evaluation device according to an embodiment. FIG. 11 is a block diagram illustrating an example of the hardware (HW) configuration of a computer that realizes the functions of a fairness evaluation device according to an embodiment.

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude various modifications or application of techniques not explicitly described below. For example, the present embodiment can be implemented with various modifications within the scope of the spirit thereof. In the drawings used in the following description, parts with the same reference numerals represent the same or similar parts unless otherwise specified.

[0013] [A] Description of Fairness Evaluation Process According to One Embodiment Hereinafter, a method for evaluating the fairness of training data will be described using a fairness evaluation device 1 according to one embodiment (see FIG. 6) as an example.

[0014] 1 is a diagram illustrating the fairness of inference results of a machine learning model. When a machine learning model is trained using a dataset containing an unfair bias as training data, the trained machine learning model may output unfair inference results, such as inference results that cause unfair discrimination. The training data is an example of multiple data to be processed by the fairness evaluation device 1. Discrimination includes discrimination regarding protected attributes.

[0015] Protected attributes may be referred to as sensitive attributes. Examples of protected attributes include at least one of race, color, national origin, religion, sex, familial status, and disability status, as defined by the U.S. Fair Housing Act (FHA). Other examples of protected attributes include at least one of marital status, public assistance status, and age, as defined by the U.S. Credit Card Elimination Act (ECIA).

[0016] In order to prevent the machine learning model from outputting unfair inference results, the fairness evaluation device 1 performs a fairness evaluation on whether or not bias regarding specific protected attributes such as gender is included in the training data.

[0017] The fairness evaluation device 1 may generate training data for machine learning from which unfair bias has been removed by processing a portion of the training data based on the fairness evaluation result. This allows the original training data containing bias to be corrected, for example, expanded, into appropriate training data that does not contain bias. The fairness evaluation device 1 may train a machine learning model using the training data generated in this manner.

[0018] 2 is a diagram showing an example of data 111a to be processed by the fairness evaluation device 1. The fairness evaluation device 1 acquires the data 111a and protection attribute information from a device (not shown) that provides the data.

[0019] The data 111a is an example of a plurality of data (data sets) acquired by the fairness evaluation device 1. The data 111a may be processed to generate training data used for training a machine learning model. The data 111a includes a plurality of attributes. Each of the plurality of attributes may be a protected attribute or an unprotected attribute.

[0020] The protected attribute information is information indicating attributes that are protected attributes among a plurality of attributes. For example, the protected attribute information may be predetermined based on external knowledge such as laws. The non-protected attribute means an attribute other than the protected attribute among a plurality of attributes.

[0021] The fairness evaluation device 1 determines a first attribute (first attribute) from among a plurality of attributes included in the data 111a, and also determines a second attribute (second attribute) based on the protected attribute information.

[0022] The first attribute is, for example, a dependent variable. In the example of FIG. 2, the first attribute is an attribute indicating whether an applicant to a certain university passed or failed. The second attribute is, for example, one of the protected attributes. In FIG. 2, the second attribute is "gender." Note that attributes other than the dependent variable, such as the second attribute, may also be called explanatory variables.

[0023] In one embodiment, among the values ​​of the first attribute (e.g., pass / fail), a value indicating "pass" may be positive, i.e., a positive example, and is an example of the first value. The fairness evaluation device 1 may, for example, identify the ratio of the first value to the multiple values ​​of the first attribute included in the data 111a (first ratio, positive example ratio). Note that the ratio of the first value may be included in the data 111a. As indicated by reference symbol A1 in FIG. 2 , the first ratio is "pass rate": 30% (0.3).

[0024] In one embodiment, as shown by symbol A2 in Fig. 2, among the values ​​of the second attribute (e.g., gender), a value indicating "male" is an example of a second value, and a value indicating "female" is an example of a third value. The data 111a may include a first group 31a in which the second attribute has a second value, and a second group 31b in which the second attribute has a third value. The first group 31a is an example of a first group, and the second group 31b is an example of a second group.

[0025] The fairness evaluation device 1 identifies the proportion (positive example proportion) of the first group 31a in which the first attribute has the first value ("pass" in FIG. 2) as the second proportion. The fairness evaluation device 1 also identifies the proportion (positive example proportion) of the second group 31b in which the first attribute has the first value as the third proportion. In the example of FIG. 2, the second proportion is 36% (0.36), and the third proportion is 24% (0.24).

[0026] In Figure 2, when focusing on the pass rate for all departments, there is a difference between the proportion of positive cases (pass rate) in the objective variable between men and women. Therefore, the data 111a appears to contain a bias in the protected attribute of gender. However, when the data 111a is divided into subgroups as shown below, it is clear that the data 111a does not contain a bias in the protected attribute.

[0027] 2 shows subgroups 32 and 33 obtained by grouping the first group 31a and the second group 31b by a third attribute (e.g., "faculty"). The value of the third attribute (e.g., "faculty") of subgroup 32 is "pharmacy," and the value of the third attribute (e.g., "faculty") of subgroup 33 is "computer science."

[0028] The subgroup 32 may include a subgroup 32a whose second attribute value is "male" and a subgroup 32b whose second attribute value is "female." The subgroup 33 may include a subgroup 33a whose second attribute value is "male" and a subgroup 33b whose second attribute value is "female."

[0029] Each of the subgroups 32a and 33a is an example of a third group (third group) in which the second attribute (gender) is a second value (e.g., male) and the third attribute (e.g., "faculty") is a fourth value. Note that the second value of the second attribute ("gender") may be "female" and the third value may be "male." In this case, each of the subgroups 32b and 33b is an example of a third group in which the second attribute (gender) is a second value (e.g., female) and the third attribute (e.g., "faculty") is a fourth value.

[0030] The fairness evaluation device 1 may identify the proportion of the plurality of third groups in which the first attribute has the first value ("Pass" in FIG. 2) as the fourth proportion. In FIG. 2, the fourth proportion in the subgroup 32a is 20% (0.2), and the fourth proportion in the subgroup 33a is 40% (0.4). The fourth proportion in the subgroup 32b is 20% (0.2), and the fourth proportion in the subgroup 33b is 40% (0.4).

[0031] When the data 111a is divided into subgroups 32 and 33 in this way, there is no difference between the genders in the proportion of positive cases in each of the subgroups 32 and 33, and it can be said that the data 111a does not contain bias in the protection attributes.

[0032] Here, the fairness evaluation device 1 according to one embodiment determines whether to perform a fairness evaluation of the third group based on whether a fourth proportion of data among the multiple third groups whose first attribute is a positive example satisfies a criterion based on the first proportion, the second proportion, and the third proportion.

[0033] For example, the fairness evaluation device 1 performs a fairness evaluation for a subgroup among the plurality of subgroups whose fourth ratio satisfies the criterion. On the other hand, the fairness evaluation device 1 suppresses, for example, omits (does not perform) the fairness evaluation for a subgroup among the plurality of subgroups whose fourth ratio does not satisfy the criterion.

[0034] Fig. 3 is a diagram showing an example of a criterion for determining whether or not to perform a fairness evaluation by the fairness evaluation device 1. The vertical axis in Fig. 3 corresponds to the positive example rate.

[0035] In FIG. 3, the parameter C ０ , C １ , C ２ may be calculated based on the following formulas (1) to (3): ０ =P ０ (1) C １ =P ０ + (| P １ -P ２ |) / 2 (2) C ２ =P ０ - (| P １ -P ２ |) / 2(3)

[0036] In the above formulas (1) to (3), P ０ is the first rate P ０ (Y=1) where Y is a response variable, and Y=1 means that the first attribute indicates a first value.

[0037] In the above formulas (2) and (3), P １ is the second rate P １ (Y=1 | A=a1), and P ２ is the third rate P ２ (Y=1|A=a2). A=a1 means that the second attribute indicates a second value. A=a2 means that the second attribute indicates a third value (second value ≠ third value). Furthermore, Y=1|A=a1 means that Y=1 and A=a1, and Y=1|A=a2 means that Y=1 and A=a2.

[0038] In the example shown in FIG. 2, the first ratio P ０ (Y=1)=0.3, the second ratio P １ (Y=1|A=a1)=0.36, the third ratio P ２ (Y=1|A=a2)=0.24. Therefore, C ０ = 0.3, C １ =0.3+(0.36-0.24) / 2=0.36, C ２ = 0.3 - (0.36 - 0.24) / 2 = 0.24.

[0039] The parameter C based on the above formulas (1) to (3) ０ ~C ２ The calculation of the value of the numerical range C may be considered to be performed based on the calculation of the numerical range C according to the following formula (4): １ -P ２ | (4)

[0040] For example, the fairness evaluation device 1 calculates the numerical range C based on the above formula (4), and sets the parameter C of the above formula (1) within the numerical range C. ０ The upper limit C of the numerical range C is set to include (for example, the median value). １ and lower limit C ２ may be calculated.

[0041] In one embodiment, the fairness evaluation device 1 determines whether the fourth ratio > C １ Or the fourth ratio < C ２ In other words, if the fourth ratio is outside the numerical range C, it may be determined that the fourth ratio satisfies the criterion.

[0042] For example, in each of the plurality of subgroups (third groups), a fourth ratio (positive example ratio) which is the ratio of data in which the first attribute has the first value is “fourth ratio>C １ or the fourth ratio < C ２ On the other hand, for each of the subgroups (third group), if the fourth ratio (positive example ratio) is "C １ ≧fourth ratio≧C ２ For groups where ", fairness evaluation is suppressed.

[0043] Here, the first ratio P ０ (Y=1) is an example of an index related to the estimated distribution of the positive example proportion in the subgroup. In this example, it is assumed that the estimated distribution of the positive example proportion in the subgroup is a normal distribution, and the center of the normal distribution is the first proportion P ０ Assume that (Y=1).

[0044] According to the above assumption, the proportion of positive examples in each subgroup is expected to be densely distributed near the center of the normal distribution, while the proportion of positive examples in a subgroup that contains an unfair bias is expected to be distributed away from the center of the normal distribution. Thus, for subgroups whose estimated distribution of the proportion of positive examples for each subgroup is near the center, it is assumed that they do not contain an unfair bias, and therefore it is permissible to omit the execution of fairness evaluation. Therefore, by suppressing the execution of fairness evaluation for such subgroups, the computational load for evaluating the fairness of the data can be reduced.

[0045] As shown in the above formula (4), the numerical range C (parameter C １ and C ２ The second ratio P １ and the third ratio P ２ is an example of an index that indicates the statistical amount of bias occurring between values ​​of the protected attribute of interest (for example, between males and females). １ (Y=1 | A=a1) and the third ratio P ２The larger the difference from (Y=1|A=a2), the larger the numerical range C, and therefore the fewer subgroups for which fairness evaluation is performed. １ and the third ratio P ２ The greater the difference between them, the greater the effect of reducing the calculation processing load.

[0046] 2 shows an example in which the third group is a group in which the second attribute has a second value (in one example, gender is male) and another attribute (third attribute, for example, the department taken) has a certain value (fourth value, for example, the department of pharmacy). However, the other attribute (third attribute) is not limited to a single attribute, and may be a combination of multiple attributes.

[0047] 4 is a diagram showing another example of data 111b to be processed by the fairness evaluation device 1. The data 111b is another example of a plurality of data (data sets) to be acquired. The data 111b is an example of training data used for training a machine learning model.

[0048] 4 illustrates an example of data 111b, which includes, as a first attribute, an attribute indicating whether an applicant will be hired or not at a certain company. The data 111b also includes, as a second attribute (protected attribute), an attribute of gender. Furthermore, the data 111b may include, as attributes (explanatory variables), marital status (single or married) and the type of employment applied for (full-time or part-time).

[0049] Fig. 5 is a diagram showing an example of a plurality of subgroups in the data 111b shown in Fig. 4. As indicated by reference symbol B1 in Fig. 5, the data 41 (all data 111b) indicating the first attribute (employment success / failure) is divided into a first group 41a in which the second attribute (e.g., "gender") has a second value (male) and a second group 41b in which the second attribute has a third value (female).

[0050] Also, from the data 41, a first ratio P of a first value (for example, “hired”) of a first attribute (hiring success / failure) is calculated. ０ In the example of FIG. 5, the first rate P ０ is 36% (0.36).

[0051] 5, DI (Disparate Impact) is a fairness evaluation index and an example of a fairness index. DI is the ratio of the proportion of positive examples in a second group having a second value in the second attribute to the proportion of positive examples in a third group having a third value in the second attribute. In one embodiment, the fairness evaluation device 1 may perform fairness evaluation using DI.

[0052] The DI for the second group (e.g., males) may be calculated by dividing the positive case rate of the second group by the positive case rate of the third group, and the DI for the third group (e.g., females) may be calculated by dividing the positive case rate of the third group by the positive case rate of the second group.

[0053] In the example of FIG. 5, a second proportion P １ The (adoption rate) is 51% (0.51), and the third rate P ２ is 22% (0.22).

[0054] In the case of FIG. 5, from the overall statistics (see reference numeral 41c) for both men and women, the standard parameters explained in FIG. 3 are C ０ = 0.36, C １ =0.36+(0.51-0.22) / 2=0.51, C ２ =0.36-(0.51-0.22) / 2=0.22 Therefore, in the example of Fig. 5, subgroups with a positive example rate >0.51 or a positive example rate <0.22 are targets for fairness evaluation.

[0055] Also, symbol B2 in Fig. 5 indicates subgroups 42 to 45 obtained by dividing the first group 41a and the second group 41b by a third attribute (e.g., "employment type" and "marital status"). Subgroup 42 has a value of "part-time" (represented as "part-time" in Fig. 5) for the third attribute (e.g., "employment type"), and subgroup 43 has a value of "full-time" (represented as "regular" in Fig. 5) for the third attribute (e.g., "employment type"). Subgroup 44 has a value of "single" for the third attribute (e.g., "marital status"), and subgroup 45 has a value of "married" for the third attribute (e.g., "marital status").

[0056] Furthermore, symbol B3 in Figure 5 shows subgroups 46 to 51, which are subgroups 42 to 45 divided based on a "composite attribute" that combines the multiple attributes of "employment type" and "marital status." Subgroups 46 and 48 have a "composite attribute" value of "part-time and single," while subgroups 47 and 50 have a "composite attribute" value of "part-time and married." Subgroup 49 has a "composite attribute" value of "single and full-time," while subgroup 51 has a "composite attribute" value of "married and full-time." Note that in Figure 5, "and" is represented by the multiplication symbol "x."

[0057] In FIG. 5, "employment status", "marital status", and a "composite attribute" that combines employment status and marital status, denoted by reference characters B2 and B3, are examples of the third attribute.

[0058] The subgroups 42 to 51 may include a subgroup whose second attribute value is "male" and a subgroup whose second attribute value is "female." Of the subgroups 42 to 51, the subgroups whose second attribute value is a second value (e.g., "male") will be referred to as subgroups 42a to 51a, and the subgroups whose second attribute value is a third value (e.g., "female") will be referred to as subgroups 42b to 51b. The subgroups 42a to 51a are an example of a third group.

[0059] 5, consider a case where the fairness evaluation device 1 specifies a target for fairness evaluation. The fairness evaluation device 1 determines whether the positive example rate (acceptance rate) of the subgroup is greater than C １ (0.51) or the rate of positive cases (acceptance rate) < C ２ In the example of FIG. 5, the fairness evaluation device 1 determines whether the positive example rate is C １ ~C ２ The subgroups 42 to 46, 47a, 48, 49, 50a, and 51 that are outside the range of (numerical range C) are identified as targets for fairness evaluation. １ ~C ２ It is determined that the execution of fairness evaluation is to be suppressed for the subgroups 47b and 50b that fall within the range of (numerical range C).

[0060] In the example shown by reference symbol B2 in FIG. 5 , there is a large difference in DI between the first group 41a and the second group 41b. However, in subgroups 42 to 45, the DI values ​​are similar between men and women. In this case, the difference in the first attribute according to the second attribute in subgroups 42 to 45 (e.g., the difference in the number of successful applicants) may be evaluated as explainable discrimination (segregation) using conventional methods. If this is evaluated as explainable discrimination (segregation), there is a possibility that a fairness evaluation will not be performed on the lower-level subgroups 46 to 51 (see reference symbol B3).

[0061] However, in the example shown by reference symbol B3 in Figure 5, it can be seen that there is an unfair bias in subgroups 46 to 51 in the lower layer because there is a difference in DI between men and women. This is an example of "Simpson's paradox," which states that even if a hypothesis holds for a certain group, it does not necessarily hold for the groups below it. In such cases, conventional methods do not perform fairness evaluations or data processing or other corrections for subgroups 46 to 51 that contain unfair bias, so there is a possibility that bias remains in data 111b.

[0062] In contrast, according to one embodiment, whether or not to perform a fairness evaluation is determined for each subgroup divided by a composite attribute whose combination length is longer than the combination length of attributes of subgroups 42 to 45 that can be evaluated as explainable discrimination (distinction). Note that the "combination length" may refer to the number of attributes combined. As a result, in the example shown in FIG. 5, the fairness evaluation device 1 can perform a fairness evaluation on subgroups 46, 47a, 48, 49, 50a, and 51 that meet the criteria for performing the evaluation.

[0063] [B] Functional Configuration Example FIG. 6 is a block diagram showing an example of the functional configuration of a fairness evaluation device 1 according to an embodiment. The fairness evaluation device 1 is an example of an information processing device or computer that executes a fairness evaluation process that evaluates the fairness of data. For example, the fairness evaluation device 1 may execute the fairness evaluation process and expand the training data used to train a machine learning model by processing the training data based on the execution results. This makes it possible to suppress the output of unfair inference results by the machine learning model.

[0064] 6 , the fairness evaluation device 1 may illustratively include a memory unit 11, an acquisition unit 12, a protection attribute determination unit 13, a bias calculation unit 14, a subgroup distribution estimation unit 15, an execution standard calculation unit 16, a subgroup search unit 17, a positive example ratio calculation unit 18, a determination unit 19, a fairness index calculation unit 20, and a data processing unit 21. The fairness evaluation device 1 may also include a machine learning unit 22 and may further include an inference processing unit 23. These blocks 12 to 23 are examples of a control unit 24.

[0065] The memory unit 11 is an example of a storage area, and stores various data used by the fairness evaluation device 1. The memory unit 11 may be realized, for example, by a storage area included in one or both of the memory 10b and the storage unit 10c shown in FIG. 8 (described later).

[0066] 6 , the memory unit 11 may be capable of storing, for example, data 111, protected attribute information 112, and training data 113 (processed data). Furthermore, when the fairness evaluation device 1 includes a machine learning unit 22, the memory unit 11 may be capable of storing a machine learning model 114. Furthermore, when the fairness evaluation device 1 includes an inference processing unit 23, the memory unit 11 may be capable of storing an inference result 115.

[0067] The information stored in the memory unit 11 may be in a table format or in another format. In one example, at least one of the pieces of information stored in the memory unit 11 may be in various formats such as a DB or an array.

[0068] The acquisition unit 12 acquires various pieces of information used in the fairness evaluation device 1. For example, the acquisition unit 12 may acquire data 111 and protection attribute information 112 from a device (not shown) that provides the data, and store them in the memory unit 11.

[0069] The data 111 is data including multiple attributes and is an example of training data. Each of the multiple attributes may be a protected attribute or an unprotected attribute. For example, the data 111 may be the data 111a shown in FIG. 2 or the data 111b shown in FIG. 4.

[0070] The protection attribute information 112 is information for identifying (for example, specifying) a protection attribute from among a plurality of attributes included in the data 111 .

[0071] The protection attribute determination unit 13 determines a protection attribute from among a plurality of attributes included in the data 111 based on the protection attribute information 112 .

[0072] The bias calculation unit 14 calculates the bias in the proportion of positive cases between a plurality of values ​​of the determined protected attribute (for example, between male and female).

[0073] For example, the bias calculation unit 14 calculates the second ratio P １ (Y=1|A=a1) and the third ratio P ２ (Y=1|A=a2) is identified, and the second ratio P １ and the third ratio P ２ The bias occurring in the attribute of interest (in one example, the protection attribute) is calculated based on the above formula (4). The bias is an example of the numerical range C calculated based on the above formula (4).

[0074] In one example, the bias calculation unit 14 calculates a statistical parity difference. The statistical parity difference is expressed as the difference in the proportion of positive examples between groups having different values ​​in an attribute of interest. The statistical parity difference is an example of a fairness index.

[0075] The subgroup distribution estimation unit 15 estimates a first proportion P of data in which a first attribute of the plurality of attributes of the data 111 has a first value. ０For example, the subgroup distribution estimation unit 15 determines the overall positive example ratio for the first attribute (class corresponding to the objective variable) as the first ratio P ０ (Y=1).

[0076] For example, the subgroup distribution estimation unit 15 may specify the overall proportion of positive examples by estimating the distribution of the proportion of positive examples for each of the multiple subgroups included in the data 111. In one example, the subgroup distribution estimation unit 15 may estimate the proportion of positive examples that indicates a peak in the distribution of the proportion of positive examples for each of the multiple subgroups.

[0077] The execution standard calculation unit 16 calculates the first ratio P ０ , the second proportion P １ , and the third ratio P ２ Based on this, a criterion for selecting the subgroups on which to perform the fairness assessment is calculated.

[0078] For example, the execution standard calculation unit 16 calculates the parameter C based on the above formulas (1) to (3) or (1) and (4) as the standard for executing the fairness evaluation. ０ , C １ and C ２ may be calculated.

[0079] The subgroup search unit 17 searches for multiple subgroups of the data 111. For example, the subgroup search unit 17 may identify multiple subgroups, each of which includes at least one protected attribute, by combining multiple attributes (explanatory variables) included in the data 111. The upper limit of the combination length, which is the number of attributes to be combined, may be determined in advance based on the expected amount of calculation depending on the content of the data 111, for example. Note that the subgroup search method may be a breadth search or a depth search.

[0080] The positive example proportion calculation unit 18 obtains the positive example proportion for each of the searched subgroups. For example, the positive example proportion calculation unit 18 may calculate the positive example proportion for each of the subgroups by value of the protection attribute. The positive example proportion for each subgroup is an example of the fourth proportion of data in the third group in which the first attribute has the first value.

[0081] The determination unit 19 determines whether the positive example ratio (fourth ratio) calculated for each of the plurality of subgroups satisfies a criterion. In one example, the determination unit 19 determines that the calculated positive example ratio satisfies the criterion when the positive example ratio is greater than C1 or less than C2.

[0082] The fairness index calculation unit 20 performs a fairness evaluation on the subgroups determined by the determination unit 19 to satisfy the criteria. The subgroups determined to undergo a fairness evaluation are an example of a third group in which the second attribute is a second value and the third attribute of the multiple attributes is a fourth value.

[0083] The fairness index calculation unit 20 may specify a fifth percentage of data in which the first attribute is the first value among a fourth group in which the second attribute is a third value (in one example, the gender is female) and the third attribute of the multiple attributes is a fourth value. In other words, the fairness index calculation unit 20 may specify a fifth percentage of data in which the first attribute is the first value among a fourth group related to the third group determined to be subjected to fairness evaluation, and in which the value of the second attribute, which is a protected attribute, differs from that of the third group.

[0084] Then, the fairness index calculation unit 20 may perform an evaluation of fairness based on the fourth ratio (in one example, the ratio of positive cases among males) and the fifth ratio (in one example, the ratio of positive cases among females).

[0085] As an example, the fairness index may be a statistical parity difference or a disparate impact (DI). The statistical parity difference may be based on the difference between the fourth ratio and the fifth ratio. The disparate impact (DI) may be based on the ratio between the fourth ratio and the fifth ratio. However, the fairness index is not limited to these cases, and various indexes may be used.

[0086] The data processing unit 21 processes a portion of the plurality of data 111 based on the fairness evaluation result to generate training data 113 for machine learning. The processing method may be any method that suppresses unfairness (discrimination) in the data 111 based on the fairness evaluation result, and may be realized by various known methods. In one example, the values ​​of unprotected attributes other than protected attributes may be rewritten in the data 111 to reduce the correlation between the protected attributes and the unprotected attributes.

[0087] As described above, the fairness evaluation device 1 may include the machine learning unit 22 and may further include the inference processing unit 23 .

[0088] In the machine learning phase, the machine learning unit 22 executes a machine learning process to train the machine learning model 114 using, as training data 113, the processed data that has been processed by the data processing unit 21 so as to reduce discrimination. The machine learning model 114 may be a neural network (NN) model including trained parameters. The machine learning process may be realized by various known techniques.

[0089] In the inference phase, the inference processing unit 23 performs inference processing using the machine learning model 114 trained based on the processed training data 113. For example, the inference processing unit 23 inputs target data (not shown) for the inference processing to the machine learning model 114, and stores the inference result 115 output from the machine learning model 114 in the memory unit 11.

[0090] [C] Operation Example Next, an operation example of the fairness evaluation device 1 according to an embodiment will be described. Fig. 7 is a flowchart illustrating an operation example of the fairness evaluation device 1 according to an embodiment.

[0091] As illustrated in FIG. 7, the acquisition unit 12 of the fairness evaluation device 1 acquires the data 111 and the protected attribute information 112 (step S1), and stores them in the memory unit 11.

[0092] The subgroup distribution estimation unit 15 estimates the first proportion P ０(Y=1) is identified (step S2). The first proportion is the proportion of data in which the first attribute Y of interest is the first value (Y=1). The first proportion may be a positive case proportion, which is the proportion of positive cases in which the first attribute is positive.

[0093] The bias calculation unit 14 calculates the second ratio P １ (Y=1|A=a1) is identified (step S3). １ (Y=1|A=a1) is the proportion of data in the first group in which the second attribute A of the multiple attributes is the second value (A=a1) that has the first attribute Y being the first value (Y=1).

[0094] The bias calculation unit 14 calculates the third ratio P ２ (Y=1|A=a2) is identified (step S4). ２ (Y=1|A=a2) is the proportion of data in the second group in which the second attribute A of the multiple attributes is the third value (A=a2) and in which the first attribute Y is the first value (Y=1).

[0095] The processing order of steps S1 to S3 is not limited to that shown in Fig. 7. For example, the processing of steps S1 to S3 may be performed in parallel.

[0096] The execution standard calculation unit 16 calculates the first ratio P ０ (Y=1), the second ratio P １ (Y=1|A=a1), and the third ratio P ２ Criterion C for selecting a subgroup to perform fairness evaluation based on (Y=1 | A=a2) ０ , C １ , C ２ is calculated (step S5).

[0097] In one embodiment, it is assumed that the distribution of the positive example ratios of the plurality of subgroups is a normal distribution. However, this is not limiting. The distribution of the positive example ratios of the plurality of subgroups may be calculated by ０ , C １ , C ２ For example, the value of C may be adjusted depending on the peak position of the distribution of the positive case ratios of multiple subgroups. ０may be adjusted, and as the half-width of the distribution of the positive example proportions of multiple subgroups becomes wider, C １ and C ２ C so that the numerical range between １ and C ２ may be adjusted.

[0098] The subgroup search unit 17 searches for one subgroup included in the first group (step S6). In other words, the subgroup search unit 17 searches for a third group in which the second attribute has the second value (included in the first group) and the third attribute of the multiple attributes has the fourth value.

[0099] The positive example proportion calculation unit 18 identifies a fourth proportion (step S7). The fourth proportion is the proportion of data in the searched subgroup (third group) in which the first attribute Y is the first value (Y=1) (positive examples).

[0100] The determination unit 19 determines whether the fourth ratio determined in step S7 satisfies the criteria calculated in step S5 (step S8). If the fourth ratio does not satisfy the criteria (NO in step S8), the process proceeds to step S11.

[0101] If the fourth ratio satisfies the criterion (YES in step S8), the fairness index calculation unit 20 performs a fairness evaluation for the subgroup determined by the determination unit 19 to satisfy the criterion (step S9).

[0102] Based on the fairness evaluation result, the data processing unit 21 processes part of the plurality of data 111 to generate training data 113 for machine learning (step S10). The processing method may be any method that suppresses unfairness (discrimination) in the data 111 based on the fairness evaluation result, and may be realized by various known methods.

[0103] The subgroup search unit 17 determines whether the search for the subgroups is completed (step S11). Note that the subgroup search unit 17 may determine whether the search is completed based on the upper limit of the combination length and the subgroups that have already been searched.

[0104] If the search for the subgroup is not completed (NO in step S11), the processes in steps S6 to S11 are repeated.

[0105] If the search is completed (YES in step S11), the machine learning unit 22 trains the machine learning model 114 using the generated training data 113 (step S12), and the process ends.

[0106] [D] Effects of an embodiment According to the method of an embodiment, the fairness evaluation device 1 acquires a plurality of data 111, and calculates a first ratio P of data in which a first attribute of a plurality of attributes of the plurality of data 111 is a first value. ０ Furthermore, the fairness evaluation device 1 specifies a second ratio P of data in which the first attribute of the plurality of attributes is the first value among the first groups in which the second attribute of the plurality of attributes is the second value a1. １ Furthermore, the fairness evaluation device 1 specifies a third proportion P of data in which the first attribute is the first value among the second group in which the second attribute is the third value. ２ Furthermore, the fairness evaluation device 1 performs a fairness evaluation for a third group in which the second attribute is the second value and the third attribute of the plurality of attributes is a fourth value, when a fourth ratio of data in which the first attribute is the first value satisfies a criterion. The criterion is the first ratio P ０ (Y=1) and the second ratio P １ (Y=1 | A=a1) and the third ratio P ２ (Y=1|A=a2).

[0107] This reduces the computational load for evaluating the fairness of data used to train a machine learning model, for example, by suppressing the execution of fairness evaluation for third groups that do not satisfy the criteria. For example, since the number of third groups increases as the number of attributes included in the data 111 increases, the computational load can be significantly reduced compared to when fairness evaluation is executed for all third groups.

[0108] An example of a reduction in the computational load when the fairness evaluation process of one embodiment is applied to open data related to people's thermal sensations will be described. The objective variable (Y) is whether or not the person felt cold. A positive example (Y=1) is when the person felt cold. The explanatory variables (attributes) are five attributes including the protected attribute "gender." The number of dimensions of the open data after binarization is 20.

[0109] For the above-described open data, if the fairness index is calculated for all combinations, the fairness index will be calculated for 21,699 subgroups. On the other hand, if the method of one embodiment is applied to the above-described open data, the fairness index will be calculated for 6,301 subgroups. Therefore, in the above-described example, the calculation of the fairness index for 15,398 subgroups can be omitted, thereby reducing the calculation processing load.

[0110] Furthermore, for example, when the above-mentioned "Simpson's Paradox" occurs, potential risks to fairness in the plurality of data 111 can be detected.

[0111] Furthermore, the first ratio P ０ , the second ratio P １ , the third ratio P ２ Since the criteria are set based on the above, it is possible to determine the subgroups that suppress the fairness evaluation by taking into consideration the distribution of the positive example ratios of the subgroups and the bias statistics.

[0112] The process of evaluating the fairness is also １ and the third ratio P ２ The first ratio P ０ The computer 10 determines that the criterion is satisfied when the fourth ratio is outside the numerical range C.

[0113] This allows us to omit the calculation of fairness evaluation for subgroups whose positive example proportions are concentrated near the overall positive example proportion and have a relatively small bias in the distribution of the positive example proportions of the subgroups, thereby further reducing the calculation load.

[0114] Furthermore, the process of performing the fairness evaluation identifies a fifth proportion of data in a fourth group in which the second attribute is a third value and the third attribute of the multiple attributes is a fourth value, in which the first attribute is the first value, and performs the fairness evaluation based on the fourth proportion and the fifth proportion.

[0115] This allows fairness to be appropriately evaluated by utilizing the fourth ratio used to determine the standard.

[0116] Furthermore, the fairness evaluation device 1 further processes a part of the plurality of data based on the fairness evaluation result to generate training data for machine learning.

[0117] This mitigates potential fairness risks in machine learning training data and prevents discriminatory inferences from machine learning models trained based on unfair biases.

[0118] The above method can be applied to fairness evaluation of training data for machine learning models used in education, medicine, communications, business, and other fields, and can be used in AI design and auditing.

[0119] [E] Hardware Configuration Example The fairness evaluation device 1 (see FIG. 6 ) according to one embodiment may be a virtual server (VM) or a physical server. Furthermore, the functions of the fairness evaluation device 1 may be implemented by one computer or two or more computers. Furthermore, at least some of the functions of the fairness evaluation device 1 may be implemented using hardware (HW) resources and network (NW) resources provided by a cloud environment.

[0120] 8 is a block diagram showing an example of a hardware (HW) configuration of a computer 10 that realizes the functions of the fairness evaluation device 1 according to an embodiment. When multiple computers are used as HW resources that realize the functions of the fairness evaluation device 1, each computer may have the HW configuration shown in FIG.

[0121] As shown in FIG. 8, the computer 10 may include, as its HW configuration, a processor 10a, a memory 10b, a storage unit 10c, an IF (Interface) unit 10d, an IO (Input / Output) unit 10e, and a reading unit 10f, for example.

[0122] The processor 10a is an example of a processing unit that performs various controls and calculations. The processor 10a may be connected to each block in the computer 10 via a bus 10i so that they can communicate with each other. The processor 10a may be a multiprocessor including multiple processors, a multi-core processor having multiple processor cores, or a configuration having multiple multi-core processors.

[0123] Examples of the processor 10a include integrated circuits (ICs) such as a CPU, MPU, GPU, APU, DSP, ASIC, and FPGA. Note that the processor 10a may be a combination of two or more of these integrated circuits. CPU is an abbreviation for Central Processing Unit, MPU is an abbreviation for Micro Processing Unit, GPU is an abbreviation for Graphics Processing Unit, and APU is an abbreviation for Accelerated Processing Unit. DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0124] For example, when the fairness evaluation device 1 executes a machine learning process or an inference process in addition to the fairness evaluation process according to an embodiment, the processor 10 a may be a combination of a processing device such as a CPU that executes the fairness evaluation process and an accelerator that executes the machine learning process or the inference process. Examples of the accelerator include the above-mentioned GPU, APU, DSP, ASIC, and FPGA.

[0125] The memory 10b is an example of HW that stores various types of data, programs, and other information. Examples of the memory 10b include a volatile memory such as a dynamic random access memory (DRAM) and / or a non-volatile memory such as a persistent memory (PM).

[0126] The storage unit 10c is an example of HW that stores various types of data, programs, and other information. Examples of the storage unit 10c include various storage devices such as a magnetic disk device such as a hard disk drive (HDD), a semiconductor drive device such as a solid state drive (SSD), and a nonvolatile memory. Examples of nonvolatile memory include a flash memory, a storage class memory (SCM), and a read-only memory (ROM).

[0127] The storage unit 10c may store a program 10g (fairness evaluation program) that realizes all or part of the various functions of the computer 10.

[0128] For example, the processor 10a of the fairness evaluation device 1 can realize the function of the fairness evaluation device 1 (the control unit 24 illustrated in Figure 6) by expanding the program 10g stored in the storage unit 10c into the memory 10b and executing it.

[0129] The IF unit 10d is an example of a communication IF that controls connections and communications between various networks, including a network between the fairness evaluation device 1 and devices not shown. Examples of devices include a computer such as a user terminal or server that provides data to the fairness evaluation device 1, and a computer such as a server that performs machine learning processing based on data output from the fairness evaluation device 1.

[0130] For example, the IF unit 10d may include an adapter that complies with a LAN (Local Area Network) such as Ethernet (registered trademark) or optical communication such as FC (Fibre Channel). The adapter may be compatible with either or both of wireless and wired communication methods.

[0131] The program 10g may be downloaded to the computer 10 from a network via the communication IF and stored in the storage unit 10c.

[0132] The IO unit 10e may include one or both of an input device and an output device. Examples of input devices include a keyboard, a mouse, and a touch panel. Examples of output devices include a monitor, a projector, and a printer. The IO unit 10e may also include a touch panel that combines an input device and a display device.

[0133] The reading unit 10f is an example of a reader that reads data and program information recorded on the recording medium 10h. The reading unit 10f may include a connection terminal or device to which the recording medium 10h can be connected or inserted. Examples of the reading unit 10f include an adapter compliant with USB (Universal Serial Bus) or the like, a drive device that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. Note that the recording medium 10h may store a program 10g, and the reading unit 10f may read the program 10g from the recording medium 10h and store it in the memory unit 10c.

[0134] Examples of the recording medium 10h include non-transitory computer-readable recording media such as magnetic / optical disks and flash memories. Examples of magnetic / optical disks include flexible disks, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs, and HVDs (Holographic Versatile Discs). Examples of flash memories include semiconductor memories such as USB memories and SD cards.

[0135] The above-described HW configuration of the computer 10 is an example. Therefore, the HW in the computer 10 may be increased or decreased (for example, adding or deleting any block), divided, integrated in any combination, or the HW may be added or deleted as needed.

[0136] [F] Others The technology according to the embodiment described above can be implemented with the following modifications and changes.

[0137] For example, the blocks 12 to 21 (as well as the machine learning unit 22 and the inference processing unit 23) included in the fairness evaluation device 1 shown in FIG. 6 may be merged in any combination, or may be divided into individual blocks.

[0138] 6 may be a configuration (system) in which a plurality of devices cooperate with each other via a network to realize each processing function. As an example, the memory unit 11 may be a DB server, the acquisition unit 12 may be a Web server or an application server, and the blocks 13 to 23 may be application servers, etc. In this case, the DB server, application server, and Web server may cooperate with each other via a network to realize each processing function of the fairness evaluation device 1.

[0139] 1 Fairness evaluation device 10 Computer 10a Processor 10b Memory 10c Storage unit 10d IF unit 10e IO unit 10f Reading unit 10h Storage medium 10g Program 11 Memory unit 12 Acquisition unit 13 Protection attribute determination unit 14 Bias calculation unit 15 Subgroup distribution estimation unit 16 10-Kuo criterion calculation unit 17 Subgroup search unit 18 Positive example ratio calculation unit 19 Judgment unit 20 Fairness index calculation unit 21 Data processing unit 22 Machine learning unit 23 Inference processing unit 24 Control unit

Claims

1. Get multiple data, identifying a first ratio of data among the plurality of data items, the first ratio being a first value among a plurality of attributes of the plurality of data items; identifying a second ratio of data in a first group in which a second attribute of the plurality of attributes is a second value, the first attribute of which is the first value, and a third ratio of data in a second group in which the second attribute of the plurality of attributes is a third value, the first attribute of which is the first value; performing a fairness assessment with respect to a third group, where the second attribute is the second value and a third attribute of the plurality of attributes is a fourth value, if a fourth proportion of data in which the first attribute is the first value satisfies a criterion based on the first proportion, the second proportion, and the third proportion. A fairness evaluation program that causes a computer to execute the process.

2. The process of performing the fairness evaluation includes: calculating a numerical range having a numerical range obtained based on a difference between the second ratio and the third ratio and including the first ratio within the range; determining that the criterion is satisfied when the fourth ratio is within the numerical range; The fairness evaluation program according to claim 1 .

3. The process of performing the fairness evaluation includes: identifying a fifth percentage of data in a fourth group in which the second attribute is the third value and a third attribute of the plurality of attributes is a fourth value, the fifth percentage being the first percentage of data in which the first attribute is the first value; performing the fairness assessment based on the fourth ratio and the fifth ratio. The fairness evaluation program according to claim 1 .

4. generating training data for machine learning by processing a part of the plurality of data based on the fairness evaluation result; causing the computer to execute a process; The fairness evaluation program according to any one of claims 1 to 3.

5. Get multiple data, identifying a first ratio of data among the plurality of data items, the first ratio being a first value among a plurality of attributes of the plurality of data items; identifying a second ratio of data in a first group in which a second attribute of the plurality of attributes is a second value, the first attribute of which is the first value, and a third ratio of data in a second group in which the second attribute of the plurality of attributes is a third value, the first attribute of which is the first value; performing a fairness assessment with respect to a third group, where the second attribute is the second value and a third attribute of the plurality of attributes is a fourth value, if a fourth proportion of data in which the first attribute is the first value satisfies a criterion based on the first proportion, the second proportion, and the third proportion. A fairness evaluation method in which processing is performed by a computer.

6. Acquire a plurality of data, identifying a first ratio of data among the plurality of data items, the first ratio being a first value among a plurality of attributes of the plurality of data items; identifying a second ratio of data in a first group in which a second attribute of the plurality of attributes is a second value, the first attribute of which is the first value, and a third ratio of data in a second group in which the second attribute of the plurality of attributes is a third value, the first attribute of which is the first value; performing a fairness assessment with respect to a third group, where the second attribute is the second value and a third attribute of the plurality of attributes is a fourth value, if a fourth proportion of data in which the first attribute is the first value satisfies a criterion based on the first proportion, the second proportion, and the third proportion. An information processing device comprising a control unit.