Evaluation device, evaluation method, and evaluation program

Abstract

This evaluation device calculates a privacy risk of each piece of data included in a data set used for training a machine-learning model. In addition, the evaluation device calculates a gain that a user has obtained by providing the data to the data set. For example, the evaluation device calculates how much the accuracy of the machine-learning model is improved by using the data for training. In addition, the evaluation device calculates, as an inequality degree, the difference between the gain estimated from the privacy risk of the data and a user gain obtained by providing the data, and evaluates the inequality degree of the machine-learning model by using the calculated inequality of each of the users.

Description

[Technical Field]

[0001] This invention relates to an evaluation device, an evaluation method, and an evaluation program for evaluating fairness regarding privacy risks. [Background technology]

[0002] Machine learning techniques, such as Deep Neural Networks (DNNs), have been criticized for posing privacy risks due to their ability to easily remember training data. Specifically, it has been shown that it is possible to estimate whether or not certain data was included in the training data from the output of a trained model. Therefore, when handling data that users do not want others to know, such as medical data or web browsing history, consideration for privacy risks is necessary.

[0003] Furthermore, in machine learning, when considering data used as training data, it is necessary to consider the balance between the user's privacy risks and the benefits the user gains from providing the data (for example, how much the provided data contributes to improving the accuracy of the trained model). For example, if a user's privacy risks are balanced with the benefits the user gains from providing the data, it can be considered fair. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Dwork, C. et al.: Fairness Through Awareness, Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp. 214‐226 (2012). [Non-Patent Document 2] John, PG, Vijaykeerthy, D. and Saha, D.: Verifying Individual Fairness in Machine Learning Models, Proceedings of the 2020 Conference on Uncertainty in Artificial Intelligence, PMLR, pp. 749‐758 (2020). [Overview of the project] [Problems that the invention aims to solve]

[0005] While technologies exist in the field of machine learning for evaluating fairness between individuals in classification problems (see Non-Patent Documents 1 and 2), no technology exists for evaluating fairness regarding privacy risks as described above. Therefore, the present invention aims to evaluate fairness regarding privacy risks. [Means for solving the problem]

[0006] To solve the aforementioned problems, the present invention is characterized by comprising: a privacy risk calculation unit that calculates the privacy risk of each data item included in a dataset used to train a machine learning model; a gain calculation unit that calculates the gain that a user who provided data to the dataset receives from providing the data; an unfairness calculation unit that calculates the difference between the privacy risk of the data and the gain the user receives from providing the data as the degree of unfairness for the user, and evaluates the degree of unfairness of the machine learning model based on the calculated degree of unfairness for each user; and an output processing unit that outputs the evaluation result of the degree of unfairness of the machine learning model. [Effects of the Invention]

[0007] According to the present invention, it is possible to evaluate fairness regarding privacy risks. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a diagram illustrating the overview of the evaluation device. [Figure 2] Figure 2 shows an example of the configuration of the evaluation device. [Figure 3] Figure 3 is a flowchart showing an example of the processing procedure performed by the evaluation device. [Figure 4] Figure 4 is a flowchart illustrating an example of the application of the evaluation device. [Figure 5] Figure 5 shows a computer running the evaluation program. [Modes for carrying out the invention]

[0009] The following describes embodiments for carrying out the present invention with reference to the drawings. The present invention is not limited to these embodiments.

[0010] [overview] The evaluation device of this embodiment evaluates whether the users who provided data to the dataset used to build the machine learning model are receiving benefits that justify the privacy risks (fairness).

[0011] For example, as shown in Figure 1, the evaluation device extracts data from the dataset used to build the machine learning model and calculates the privacy risk of that data. The evaluation device also calculates the gains that a user can obtain by providing that data to the dataset.

[0012] Then, based on the difference between the gain assumed from the user's privacy risk and the gain actually obtained by the user by providing the data, the evaluation device calculates the unfairness of each user. Here, the evaluation device calculates a higher unfairness as the difference between the gain assumed from the user's privacy risk and the gain actually obtained by the user by providing the data is larger. Thereby, the evaluation device calculates whether each user who provided data to the dataset has obtained a gain commensurate with the privacy risk (unfairness). Then, the evaluation device evaluates the unfairness of the machine learning model based on the calculated unfairness of each user.

[0013] [Configuration Example] Next, a configuration example of the evaluation device 10 will be described using FIG. 2. The evaluation device 10 includes, for example, an input / output unit 11, a storage unit 12, and a control unit 13.

[0014] The input / output unit 11 is an interface that controls the input / output of various data. The input / output unit 11 receives, for example, the input of a dataset used for constructing a machine learning model. The input dataset is stored in the storage unit 12.

[0015] The storage unit 12 stores data, programs, etc. that are referred to when the control unit 13 executes various processes. The storage unit 12 is realized by a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 12 stores the dataset received by the input / output unit 11. Also, for example, the storage unit 12 may store information indicating which user provided each data in the dataset.

[0016] The control unit 13 is responsible for controlling the entire evaluation device 10. The functions of the control unit 13 are realized, for example, by the CPU (Central Processing Unit) executing a program stored in the memory unit 12. The control unit 13 includes, for example, a privacy risk calculation unit 131, a gain calculation unit 132, an unfairness calculation unit 133, and an output processing unit 134.

[0017] The privacy risk calculation unit 131 calculates the privacy risk of each data point included in the dataset. For example, the privacy risk calculation unit 131 calculates the privacy risk by calculating the lower bound (LB) of the differential privacy parameter ε based on the following equation (1).

[0018]

number

[0019] Note that the upper bound of the false positive rate (FPR) in equation (1) UB ) and the upper bound of the missed rate (FNR) UB The false positive rate (FPR) and false negative rate (FNR) are calculated using the false positive rate (FPR) and false negative rate (FNR) obtained when a game is played many times (for example, around 1000 times) in which the player guesses whether or not the data being evaluated was used to train the model (see Equation (2)). Furthermore, the upper bounds for both the false positive rate (FPR) and the false negative rate (FNR) can be calculated using the Clopper-Pearson method.

[0020]

number

[0021] Furthermore, the privacy risk calculation unit 131 may calculate the privacy risk based, for example, on the success rate when membership estimation is performed many times.

[0022] The gain calculation unit 132 calculates the gain obtained by the user providing data to the dataset. The profit obtained by the user providing data to the dataset is, for example, the degree to which the accuracy of the machine learning model improves when the data is used for learning the machine learning model, etc.

[0023] For example, the gain calculation unit 132 calculates the degree to which the accuracy of the machine learning model improves when the data provided by the user is used for learning as follows. First, the gain calculation unit 132 constructs n shadow models (first shadow models) that use the data provided by the user for learning and n shadow models (second shadow models) that do not use the data for learning, respectively.

[0024] Next, the gain calculation unit 132 sets the number of first shadow models that output correct data among the n first shadow models as c in And the value obtained by dividing c in by n (c in / n) is taken as the accuracy of the first shadow model. Also, the gain calculation unit 132 sets the number of second shadow models that output correct data among the n second shadow models as c out And the value obtained by dividing c out by n (c out / n) is taken as the accuracy of the second shadow model.

[0025] Then, the gain calculation unit 132 calculates the difference between the accuracy of the first shadow model (c in / n) and the accuracy of the second shadow model (c out / n) as the gain (g) obtained by the user providing the data (see Equation (3)).

[0026]

Equation

[0027] In addition, the payoff calculation unit 132 may calculate the payoff based on how much other data held by the user improves the accuracy of the shadow model, rather than by the method described above. Furthermore, the user's payoff may also be in the form of services or monetary rewards provided in exchange for the data provided by the user.

[0028] The unfairness calculation unit 133 calculates the unfairness of each user who provided data to the dataset. The unfairness calculation unit 133 then evaluates the unfairness of the machine learning model based on the unfairness of each user that it has calculated. For example, the unfairness calculation unit 133 calculates the unfairness of a user as the difference between the user's gain expected from the privacy risk of the data calculated by the privacy risk calculation unit 131 and the user's gain obtained by providing the data calculated by the gain calculation unit 132. The unfairness calculation unit 133 then takes the maximum of the calculated unfairness of each user as the unfairness of the machine learning model.

[0029] For example, all users U={u i} i= n Let's set the total risk for all users to R = {r i} i= n 1. The gain is G = {g i} i= n Let the mean be 1. Also, let R' be the risk normalized to mean=0 and variance=1, and G' be the gain. The unfairness calculation unit 133 then calculates the difference between the normalized risk r' and the normalized gain g' for each user, and the maximum value of this difference is taken as the unfairness of the machine learning model (δ^) (see equation (4)).

[0030]

number

[0031] Furthermore, if outliers are included in the unfairness scores of each user, the unfairness calculation unit 133 may use the maximum value among the unfairness scores after excluding those outliers as the unfairness score of the machine learning model.

[0032] The output processing unit 134 outputs the processing results from the control unit 13. For example, the output processing unit 134 outputs the evaluation result of the unfairness of the machine learning model by the unfairness calculation unit 133.

[0033] Such an evaluation device 10 can be used to evaluate the degree of unfairness in machine learning models.

[0034] [Example of processing procedure] Next, an example of the processing procedure performed by the evaluation device 10 will be explained using Figure 3. First, the privacy risk calculation unit 131 of the evaluation device 10 calculates the privacy risk of each data included in the dataset (S1).

[0035] Next, the gain calculation unit 132 calculates the gain obtained by the user providing data to the dataset (S2). For example, the gain calculation unit 132 calculates how much the accuracy of the shadow model improves when the data provided by the user is used for training.

[0036] Next, the unfairness calculation unit 133 evaluates the unfairness of the machine learning model (S3). For example, the unfairness calculation unit 133 calculates the unfairness of a user as the difference between the gain expected from the data privacy risk calculated in S1 and the actual gain obtained by the user from providing the data calculated in S3. The unfairness calculation unit 133 then takes the maximum value of the unfairness of each user calculated by the unfairness calculation unit 133 as the unfairness of the machine learning model. Finally, the output processing unit 134 outputs the evaluation result of the unfairness of the machine learning model obtained in S3 (S4).

[0037] The evaluation device 10 performs the above processing to evaluate the degree of unfairness in the machine learning model.

[0038] [Examples of application] Next, we will explain an example of the application of the evaluation device 10 using Figure 4. For example, the administrator of the evaluation device 10 designs a neural network (NN) to be used for calculating privacy risk (S11). For example, they train an NN that satisfies differential privacy using DP-SGD (Differentially Private Stochastic Gradient Descent).

[0039] Next, the administrator selects the dataset to be used to evaluate the degree of unfairness among users and the users to be evaluated for unfairness (S12). For example, the administrator selects about 100 users, taking into account user diversity.

[0040] Subsequently, the evaluation device 10 uses the dataset selected in S12 to calculate the degree of unfairness for each user selected in S12, and evaluates the degree of unfairness of the machine learning model (NN) based on the calculated degree of unfairness for each user (S13). For example, the evaluation device 10 calculates the privacy risk of each data in the dataset for the NN designed in S11. The evaluation device 10 also calculates the gain for the users selected in S12 (users who provided data to the dataset). Then, the evaluation device 10 calculates the degree of unfairness for each user based on the calculated privacy risk of each data and the gain for each user who provided the data. Subsequently, the evaluation device 10 takes the maximum value of the calculated degrees of unfairness for each user as the degree of unfairness of the machine learning model (NN).

[0041] [System configuration, etc.] Furthermore, the components of each part shown in the diagram are functional concepts and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown in the diagram, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. Moreover, all or any part of the processing functions performed by each device can be realized by a CPU and the program executed on that CPU, or by hardware using wired logic.

[0042] Furthermore, among the processes described in the embodiments described above, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[0043] [program] The aforementioned evaluation device 10 can be implemented by installing a program (evaluation program) as packaged software or online software on a desired computer. For example, by having the above program run on an information processing device, the information processing device can function as the evaluation device 10. The information processing device referred to here includes mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handyphone System), as well as terminals such as PDA (Personal Digital Assistant).

[0044] Figure 5 shows an example of a computer running an evaluation program. Computer 1000 has, for example, memory 1010 and a CPU 1020. Computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0045] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM (Random Access Memory) 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0046] The hard disk drive 1090 stores, for example, the OS 1091, application program 1092, program module 1093, and program data 1094. That is, the program that defines each process executed by the evaluation device 10 is implemented as a program module 1093 in which executable code for a computer is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing a process similar to the functional configuration of the evaluation device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0047] Furthermore, the data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes them.

[0048] Furthermore, the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090; for example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via a network interface 1070. [Explanation of Symbols]

[0049] 10 Evaluation device 11 Input / output section 12 Storage section 13 Control Unit 131 Privacy Risk Calculation Unit 132 Gain Calculation Unit 133 Unfairness Calculation Department 134 Output Processing Unit

Claims

1. A privacy risk calculation unit that calculates the privacy risk of each data point included in the dataset used to train a machine learning model, A gain calculation unit that calculates the gain that a user who has provided data to the aforementioned dataset will obtain by providing the aforementioned data, An unfairness calculation unit calculates the difference between the privacy risk of the data and the user gain obtained from providing the data as the degree of unfairness for the user, and evaluates the degree of unfairness of the machine learning model based on the calculated degree of unfairness for each user. An output processing unit that outputs the evaluation result of the unfairness of the machine learning model, An evaluation device characterized by comprising the following features.

2. The aforementioned unfairness calculation unit, The privacy risk of the data and the user's gain are both normalized, and the difference between the normalized privacy risk of the data and the user's gain is defined as the degree of unfairness to the user. The evaluation apparatus according to feature 1.

3. The aforementioned unfairness calculation unit, The maximum value among the calculated unfairness scores for each user is used as the evaluation result of the machine learning model. The evaluation apparatus according to feature 1.

4. The gain obtained by providing the aforementioned data is This is the degree to which the accuracy of the machine learning model improves when the aforementioned data is used to train the machine learning model. The evaluation apparatus according to feature 1.

5. The aforementioned gain calculation unit, By constructing a shadow model that uses the aforementioned data for training and a shadow model that does not use the aforementioned data, and calculating the difference between the accuracy of the shadow model that uses the aforementioned data and the accuracy of the shadow model that does not use the aforementioned data, the degree to which the accuracy of the machine learning model improves when the aforementioned data is used for training is calculated. The evaluation apparatus according to feature 4.

6. The gain obtained by providing the aforementioned data is Services or monetary compensation provided to the user in exchange for providing the aforementioned data. The evaluation apparatus according to feature 1.

7. An evaluation method performed by an evaluation device, The process of calculating the privacy risk of each data point included in the dataset used to train the machine learning model, A step of calculating the gain that a user who has provided data to the aforementioned dataset will obtain by providing the aforementioned data, A step of calculating the difference between the privacy risk of the data and the user gain obtained by providing the data as the degree of unfairness for the user, and evaluating the degree of unfairness of the machine learning model based on the calculated degree of unfairness for each user, A step of outputting the evaluation result of the unfairness of the machine learning model. An evaluation method characterized by including the following.

8. The process of calculating the privacy risk of each data point included in the dataset used to train the machine learning model, A step of calculating the gain that a user who has provided data to the aforementioned dataset will obtain by providing the aforementioned data, A step of calculating the difference between the privacy risk of the data and the user gain obtained by providing the data as the degree of unfairness for the user, and evaluating the degree of unfairness of the machine learning model based on the calculated degree of unfairness for each user, A step of outputting the evaluation results of the machine learning model An evaluation program to get a computer to run it.

Images