Calculation device, calculation method, and calculation program

Abstract

In a calculation device (10), an acquisition unit (15a) acquires a dataset and target samples including a plurality of samples. A learning unit (15b) uses the dataset and at least one sample of the target samples to cause a model (f) to learn. A determination unit (15c) uses output at the time when each sample of the target samples is input to the model (f) to determine whether or not the sample was used in the learning of the model (f). A calculation unit (15d) uses a success rate of the determination to calculate a privacy risk that represents a risk of data leakage in the model (f).

Description

[Technical Field]

[0001] The present invention relates to a calculation device, a calculation method, and a calculation program. [Background technology]

[0002] Machine learning techniques, such as Deep Neural Networks (DNNs), have been criticized for posing privacy risks due to their ability to easily remember training data. Specifically, it has been pointed out that it is possible to estimate whether certain data was included in the training data from the output of a trained model. Particular consideration of privacy risks is necessary when dealing with data that users do not want others to know, such as medical data or web browsing history.

[0003] In contrast, there are known techniques for calculating privacy risk based on the success rate of attacks that attempt to identify whether certain data is included in a dataset (see Non-Patent Documents 1-3). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Jagielski, M. et al., “Auditing Differentially Private Machine Learning: How Private is Private SGD?”, arXiv preprint, arXiv:2006.07709v1, 2020 [Non-Patent Document 2] Nasr, M. et al., “Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning”, arXiv preprint, arXiv:2101.04535v1, 2021. [Non-Patent Document 3] Malek Esmaeili, M. et al., “Antipods of Label Differential Privacy: PATE and ALIBI”, Proceedings of the 35th Advances in Neural Information Processing Systems, 2021, pp.6934-6945 [Overview of the project] [Problems that the invention aims to solve]

[0005] However, conventional technologies have the problem of high computational costs when calculating privacy risk. For example, calculating privacy risk using conventional technology requires repeating a game with one target sample about 1000 times. Since learning is required once for each game, about 1000 learning iterations are necessary to calculate privacy risk. Therefore, the computational cost of calculating privacy risk is high in neural networks (NNs).

[0006] This invention has been made in view of the above, and aims to calculate privacy risk while reducing computational costs. [Means for solving the problem]

[0007] To solve the above-mentioned problems and achieve the objective, the calculation device according to the present invention is characterized by comprising: an acquisition unit that acquires a dataset and a target sample including a plurality of samples; a learning unit that learns a model using the dataset and at least one of the target samples; a determination unit that determines whether or not a sample was used to learn the model using the output when each of the target samples is input to the model; and a calculation unit that calculates a privacy risk representing the risk of data leakage in the model using the success rate of the determination. [Effects of the Invention]

[0008] According to the present invention, it is possible to reduce the calculation cost and calculate the privacy risk.

Brief Description of the Drawings

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating a schematic configuration of the calculation device of the present embodiment. [Figure 2] FIG. 2 is a diagram for explaining the calculation process. [Figure 3] FIG. 3 is a flowchart showing the calculation process procedure. [Figure 4] FIG. 4 is a diagram showing an example of a computer that executes the calculation program.

Modes for Carrying Out the Invention

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited by this embodiment. In the description of the drawings, the same parts are denoted by the same reference numerals.

[0011] [Configuration of the Calculation Device] The calculation device according to the present embodiment calculates the privacy risk while suppressing the calculation cost by performing one game using a plurality of target samples.

[0012] FIG. 1 is a schematic diagram illustrating a schematic configuration of the calculation device of the present embodiment. Further, FIG. 2 is a diagram for explaining the calculation process. First, as illustrated in FIG. 1, the calculation device 10 of the present embodiment is realized by a general-purpose computer such as a personal computer, and includes an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.

[0013] The input unit 11 is implemented using an input device such as a keyboard or mouse, and in response to input operations by the operator, it inputs various instruction information, such as the start of processing, to the control unit 15. The output unit 12 is implemented using a display device such as a liquid crystal display, a printing device such as a printer, etc. For example, the output unit 12 displays the results of the calculation process described later.

[0014] The communication control unit 13 is implemented using a NIC (Network Interface Card) or the like, and controls communication between the control unit 15 and external devices via telecommunication lines such as a LAN (Local Area Network) or the Internet. For example, the communication control unit 13 controls communication between the control unit 15 and a management device that manages various information related to such devices.

[0015] The memory unit 14 is implemented using semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or storage devices such as hard disks or optical discs. The memory unit 14 pre-stores processing programs for operating the computing device 10, as well as data used during the execution of the processing programs, or temporarily stores them each time processing is performed. The memory unit 14 may also be configured to communicate with the control unit 15 via the communication control unit 13.

[0016] In this embodiment, the memory unit 14 stores the model f generated by learning in the calculation process described later.

[0017] The control unit 15 is implemented using a CPU (Central Processing Unit) or the like, and executes a processing program stored in memory. As a result, the control unit 15 functions as an acquisition unit 15a, a learning unit 15b, a determination unit 15c, and a calculation unit 15d, as illustrated in Figure 1, and performs calculation processing. Note that each or part of these functional units may be implemented on different hardware. For example, the calculation unit 15d and the other functional units may be implemented on different hardware. Furthermore, the control unit 15 may also include other functional units.

[0018] The acquisition unit 15a acquires a data set and a target sample including a plurality of samples. For example, as illustrated in FIG. 2(1), the acquisition unit 15a acquires a data set D0 and a plurality (i = 1, …, n) of samples (x , j , , n , ,

[0019] , i , , j , , i , ,

[0023] ,

[0021] , i , j , i=1 , ,

[0020] , , ,

[0022] , , i , , j , , , , y i ) included in the target sample {(x i , y i )}. i=1 n The acquisition unit 15a may acquire the data set and the target sample in advance and store them in the storage unit 14 prior to the calculation process described later.

[0019] <0​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​In this case, the acquisition unit 15a acquires target samples that include samples in which the loss l of the model f trained using the dataset is less than a predetermined threshold. This is because samples with large losses tend to interact with each other, and by selecting samples with moderate losses, interactions can be suppressed. It has been empirically proven that by reducing the interactions between samples in this way, it becomes possible to calculate privacy risk with high accuracy in the calculation process described later.

[0024] In this embodiment, the acquisition unit 15a acquires 100 samples as target samples, which are obtained by removing samples with high loss from randomly selected samples.

[0025] The learning unit 15b trains the model f using the dataset and at least one of the target samples. Specifically, as illustrated in Figure 2(2), the learning unit 15b first randomly decides whether or not to use each sample of the target sample for training. Next, the learning unit 15b uses the dataset D0 and the sample (b) that has been decided to be used for training. i The model f is trained using (=1).

[0026] In this process, the learning unit 15b is trained using the same learning algorithm as the model used in the privacy risk calculation process described later.

[0027] Furthermore, as shown in Figure 2(2), the learning unit 15b uses multiple (i=1,…,n) samples (x i ,y i For this, model f is trained once.

[0028] The determination unit 15c uses the output when each sample of the target sample is input to the model f to determine whether or not the sample was used to train the model f.

[0029] Specifically, the determination unit 15c determines whether or not a sample was used to train model f by using either the output loss or likelihood ratio when each sample is input to model f.

[0030] For example, the determination unit 15c, as illustrated in Figure 2(3), determines the sample (x i ,y i The output f(x) when ) is input to a trained model f. i Using (x i ,y i Loss of ) l(f(x i ),y i If this sample (x) is smaller than a predetermined threshold, i ,y i ) is determined to have been used to train model f.

[0031] Alternatively, the determination unit 15c determines the sample (x i ,y i The output f(x) when ) is input to a trained model f. i ) Instead of that, use the output of other samples or sample (x i ,y i You may also use the output of a sample to which noise has been added.

[0032] Alternatively, the determination unit 15c determines the sample (x i ,y i Instead of the loss l of ), the sample (x) is calculated using a method based on the likelihood ratio. i ,y i You may also determine whether or not ) was used to train model f.

[0033] Furthermore, the determination unit 15c uses multiple samples to determine one sample (x i ,y i When making a determination of the sample (x), the output of each of the multiple samples input into model f is integrated, and the sample (x) is determined. i ,y iThe determination unit 15c determines whether or not the sample was used to train model f. For example, the determination unit 15c makes the determination by majority vote of the determination results for each sample. Alternatively, the determination unit 15c determines D' if all samples have the same determination result D'. Alternatively, the determination unit 15c may determine D' if one or more samples have been determined to be D'.

[0034] The calculation unit 15d uses the success rate of the judgment to calculate the privacy risk, which represents the risk of data leakage in model 14a. Specifically, the learning unit 15b and the judgment unit 15c repeatedly execute the game, which is the process shown in Figures 2(2) and 2(3). The calculation unit 15d then derives the success rate of the judgment and calculates the privacy risk based on the success rate. In this embodiment, the game is repeated 100 times to calculate the privacy risk.

[0035] For example, the calculation unit 15d calculates the privacy risk using the false positive rate and the missed detection rate as the success rate. Specifically, the privacy risk ε^ can be expressed by the following equation (2) using the false positive rate FPR and the missed detection rate FNR.

[0036]

number

[0037] Here, δ is 10 -5 It is a constant of a small magnitude. The calculation unit 15d may also calculate the privacy risk considering the confidence interval. The calculation device 10 determines that the privacy risk is high if the calculated privacy risk is greater than a predetermined threshold.

[0038] [Calculation Process] Next, with reference to Figure 3, the calculation process by the calculation device 10 according to this embodiment will be described. Figure 3 is a flowchart of the calculation process procedure. The flowchart in Figure 3 starts, for example, when the user inputs an operation to instruct the process to begin.

[0039] First, the acquisition unit 15a acquires the dataset and the target sample containing multiple samples (step S1). For example, the acquisition unit 15a acquires the dataset D o Then, a target sample consisting of 100 samples is obtained by removing samples with high loss from randomly selected samples.

[0040] Next, the learning unit 15b trains model f using the dataset and at least one of the target samples (step S2). For example, the learning unit 15b uses the dataset D0 and the samples decided to be used for training to generate model f, which is a neural network (NN) to which DP-SGD is applied as the learning algorithm.

[0041] Next, the determination unit 15c determines whether a sample was used to train model f by using the output of inputting each sample of the target sample into model f (step S3). For example, the determination unit 15c determines whether a sample was used to train model f by using either the loss or likelihood ratio of the output of inputting each sample into model f. For example, the learning unit 15b and the determination unit 15c repeat this game 100 times.

[0042] Then, the calculation unit 15d calculates the privacy risk, which represents the risk of data leakage in model 14a, using the success rate of the judgment (step S4). For example, the calculation unit 15d calculates the privacy risk based on the false positive rate (FPR) and the missed positive rate (FNR) of judgments in a game repeated 100 times.

[0043] The calculation device 10 determines that the privacy risk is high if the calculated privacy risk is greater than a predetermined threshold. This completes the series of calculation processes.

[0044] [effect] As described above, in the calculation device 10 of this embodiment, the acquisition unit 15a acquires a dataset and a target sample containing multiple samples. The learning unit 15b learns a model f using the dataset and at least one of the target samples. The determination unit 15c determines whether a sample was used to learn the model f using the output when each sample of the target sample is input to the model f. The calculation unit 15d calculates a privacy risk representing the risk of data leakage in the model f using the success rate of the determination.

[0045] Specifically, the determination unit 15c determines whether or not a sample was used to train model f by using either the output loss or likelihood ratio when each sample is input to model f.

[0046] Furthermore, the determination unit 15c uses multiple samples to determine one sample (x i ,y i When making a determination of the sample (x), the output of each of the multiple samples input into model f is integrated, and the sample (x) is determined. i ,y i Determine whether ) was used to train model f.

[0047] Furthermore, the calculation unit 15d calculates the privacy risk using the false detection rate and the missed detection rate as the success rate.

[0048] In this way, by using multiple samples in a single training run, the number of times we have to play the game of determining whether or not each sample was used to train model f can be reduced. Therefore, it becomes possible to reduce the computational cost of privacy risk.

[0049] Furthermore, the acquisition unit 15a acquires target samples in which the interaction between each sample is smaller than a predetermined threshold. In this case, the acquisition unit 15a acquires target samples that include samples in which the loss of the model f trained using the dataset is smaller than a predetermined threshold. This makes it possible to calculate privacy risk with high accuracy.

[0050] [program] It is also possible to create a program that describes the processing performed by the calculation device 10 according to the above embodiment in a language that can be executed by a computer. In one embodiment, the calculation device 10 can be implemented by installing a calculation program that performs the above calculation processing as packaged software or online software on a desired computer. For example, by having the above calculation program run on an information processing device, the information processing device can be made to function as the calculation device 10. The information processing device referred to here includes desktop or notebook personal computers. In addition, the category of information processing device also includes mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handyphone System), and slate terminals such as PDA (Personal Digital Assistant). Furthermore, the functions of the calculation device 10 may be implemented on a cloud server.

[0051] Figure 4 shows an example of a computer running a calculation program. Computer 1000 includes, for example, memory 1010, CPU 1020, hard disk drive interface 1030, disk drive interface 1040, serial port interface 1050, video adapter 1060, and network interface 1070. These components are connected by bus 1080.

[0052] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1031. The disk drive interface 1040 is connected to the disk drive 1041. A removable storage medium, such as a magnetic disk or optical disk, is inserted into the disk drive 1041. A serial port interface 1050 is connected to, for example, a mouse 1051 and a keyboard 1052. A video adapter 1060 is connected to, for example, a display 1061.

[0053] Here, the hard disk drive 1031 stores, for example, the OS 1091, the application program 1092, the program module 1093, and the program data 1094. The information described in the above embodiment is stored, for example, in the hard disk drive 1031 or the memory 1010.

[0054] Furthermore, the calculation program is stored in the hard disk drive 1031 as a program module 1093 containing instructions to be executed by the computer 1000, for example. Specifically, the program module 1093 containing instructions for each process executed by the calculation device 10 described in the above embodiment is stored in the hard disk drive 1031.

[0055] Furthermore, the data used for information processing by the calculation program is stored as program data 1094, for example, in the hard disk drive 1031. The CPU 1020 then reads the program module 1093 and program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as needed and executes the procedures described above.

[0056] Furthermore, the program module 1093 and program data 1094 related to the calculation program are not limited to being stored on the hard disk drive 1031; for example, they may be stored on a removable storage medium and read by the CPU 1020 via a disk drive 1041 or the like. Alternatively, the program module 1093 and program data 1094 related to the calculation program may be stored on another computer connected via a network such as a LAN or WAN (Wide Area Network) and read by the CPU 1020 via a network interface 1070.

[0057] Although embodiments applying the invention made by the present inventors have been described above, the present invention is not limited by the descriptions and drawings that constitute part of the disclosure of the present invention in this embodiment. That is, all other embodiments, examples, and operational techniques made by those skilled in the art based on this embodiment are included in the scope of the present invention. [Explanation of Symbols]

[0058] 10 Calculation device 11 Input section 12 Output section 13 Communication Control Unit 14 Storage section 15 Control Unit 15a Acquisition part 15b Learning Department 15c Judgment section 15d Calculation Unit f model

Claims

1. An acquisition unit that acquires a dataset and a target sample containing multiple samples, A learning unit that trains a model using the aforementioned dataset and at least one of the target samples, A determination unit that determines whether or not a sample was used to train the model, using the output when each sample of the target sample is input to the model, A calculation unit that uses the success rate of the above determination to calculate the privacy risk representing the risk of data leakage in the model, A calculation device characterized by having the following features.

2. The calculation device according to claim 1, characterized in that the determination unit determines whether or not the sample was used for training the model using either the loss or likelihood ratio of the output.

3. The calculation device according to claim 1, characterized in that, when the determination unit performs a determination on one sample using multiple samples, it integrates the outputs when each of the multiple samples is input to the model to determine whether or not the sample was used for training the model.

4. The calculation device according to claim 1, characterized in that the calculation unit calculates the privacy risk using the false detection rate and the missed detection rate as the success rate.

5. The calculation apparatus according to claim 1, characterized in that the acquisition unit acquires the target sample in which the interaction between each sample is less than a predetermined threshold.

6. The calculation apparatus according to claim 5, characterized in that the acquisition unit acquires the target sample which includes a sample in which the loss of the model trained using the dataset is less than a predetermined threshold.

7. A calculation method performed by a calculation device, The acquisition process involves obtaining a dataset and target samples containing multiple samples, A learning process in which a model is trained using the dataset and at least one of the target samples, A determination step of determining whether or not a sample was used to train the model, using the output when each sample of the target sample is input to the model, A calculation step of calculating the privacy risk, which represents the risk of data leakage in the model, using the success rate of the above determination, A calculation method characterized by including the following.

8. Acquisition steps include obtaining a dataset and target samples containing multiple samples, A learning step in which a model is trained using the dataset and at least one of the target samples, A determination step of determining whether or not a sample was used to train the model, using the output when each sample of the target sample is input to the model, A calculation step of calculating the privacy risk, which represents the risk of data leakage in the model, using the success rate of the above determination, A calculation program to be executed by a computer.

Images