Secret computation client device and hyperparameter determination method

Abstract

This secret computation client device includes a secret computation client unit and an AutoML unit. The secret computation client unit connects to a secret computation system formed by a plurality of secret computation servers, transmits a share of instruction information related to machine learning of an AI model to the secret computation system, and receives a share of an evaluation value related to machine learning of the AI model. The AutoML unit selects hyperparameters for the AI model for instruction to the secret computation client unit, and cost-evaluates the AI model in plain text on the basis of the evaluation value. The evaluation value is a statistic for calculating the loss of the AI model. The cost evaluation is an evaluation taking the computation time and memory usage into consideration in addition to the loss. The AutoML unit repeats selection of the hyperparameters and cost evaluation, and determines the hyperparameters.

Description

Secure computation client device and hyperparameter selection method

[0001] The present invention relates to a secure computation client device connected to a secure computation system consisting of a plurality of secure computation servers, and a hyperparameter selection method using the secure computation client device.

[0002] While the momentum for data utilization using AI and other technologies is growing, demand for privacy protection is also increasing, and there is a demand for an environment where highly confidential data can be analyzed using proven AI algorithms in fields such as medical care. Secure computing AI is a technology that brings together data accumulated by multiple companies and safely utilizes it. It enables the creation of models and inference (data operation in which only the calculation results are visible to anyone) using extremely secure and diverse AI algorithms while keeping the data confidential (secret distribution) and never restoring it. Furthermore, AutoML (Automated Machine Learning), as shown in Non-Patent Documents 1 and 2, is known as a technology for automatically tuning AI models.

[0003] Yao, Quanming & Wang, Mengshuo & Escalante, Hugo Jair & Isabelle, Guyon & Hu, Yi-Qi & Yu-Feng, Li & Tu, Wei-Wei & Qiang, Yang & Yang, Yu. (2018), "Taking Human out of Learning Applications: A Survey on Automated Machine Learning", 31, October, 2018, [Retrieved June 11, 2020], Internet<https: / / ar5iv.labs.arxiv.org / html / 1810.13306> Xin He, Kaiyong Zhao, Xiaowen Chu, "AutoML: A survey of the state-of-the-art", Knowledge-Based Systems, Volume 212, 2021, 106622, ISSN 0950-7051, [Retrieved June 11, 2024], Internet<https: / / doi.org / 10.1016 / j.knosys.2020.106622> .

[0004] However, secure computation of machine learning in actual analysis and services requires appropriate determination of hyperparameters, such as the learning type and learning method to be used. However, this is less efficient than plaintext, and manual implementation is unrealistic. AutoML (Non-Patent Documents 1 and 2) is known as a technology for automatically tuning AI models, but because it automatically tunes by repeating various machine learning procedures multiple times, the computational cost is too high to directly implement it in secure computation. Abstracting all aspects, such as model type, parameter depth, and structural information, requires long calculation times, impairing usability. Therefore, there is a need for technology that can automatically search for learning models within a realistic computation time. The present invention aims to provide a method that can shorten the time required to select hyperparameters for secure computation AI.

[0005] The secure computation client device of the present invention comprises a secure computation client unit and an AutoML unit. The secure computation client unit is connected to a secure computation system consisting of one or more secure computation servers, and transmits shares of instruction information related to machine learning of an AI model to the secure computation system, and receives shares of evaluation values ​​related to the machine learning of the AI ​​model. The AutoML unit selects hyperparameters for the AI ​​model and instructs the secure computation client unit, and performs cost evaluation of the AI ​​model in plain text based on the evaluation values. The evaluation values ​​are statistics for calculating the loss of the AI ​​model. The cost evaluation takes into account not only loss but also calculation time and memory usage. The AutoML unit repeats hyperparameter selection and cost evaluation to select hyperparameters.

[0006] According to the secure computation client device of the present invention, statistics for calculating loss are calculated using secure computation, and evaluation in the AutoML section, which takes into account computation time and memory usage, is performed using plain text, thereby shortening the time required to select hyperparameters for secure computation AI.

[0007] A secure computation system including a plurality of secure computation servers connected via a network, and a secure computation client device. A secure computation system including a plurality of secure computation servers connected via a network, and a secure computation client device. A secure computation client device ...

[0008] Prior to describing the embodiments, the basic technical concepts used in the present disclosure will be described.

[0009] [Secret Sharing Technology] Secret sharing is a technology that converts data into multiple shares, allowing the original data to be restored using a certain number of shares or more, but making it impossible to restore the original data using shares less than the certain number. (k,n)-secret sharing, a type of secret sharing, divides an input plaintext into n shares, distributes the shares to n computing entities, and allows the plaintext to be restored using any k shares, but no information about the plaintext can be obtained using shares less than k. Here, n and k are integers greater than or equal to 1, and n≧k. A typical example of (k,n)-secret sharing is Shamir secret sharing, described in "A. Shamir, "How to share a secret," Communications of the ACM, Volume 22, Issue 11, pp. 612-613, 1979 (Reference 1)." The secret sharing used in this disclosure may be any method that can utilize secure computation, as described below.

[0010] [Secure computation technology] Secure computation is a technology in which data to be computed is secretly shared and stored among multiple computing entities, and the computing entities cooperate with each other to compute the shared values ​​of a function of the original data without restoring the original data.Secure computation uses secret sharing as an elemental technology.

[0011] The secure computation used in the first embodiment of the present disclosure may be one that allows various computations required for desired data processing to be performed on shared values ​​obtained by a specific secret sharing method. A secure computation technique that performs basic computations such as addition and multiplication on secret shared values ​​is described, for example, in "Koji Senda, Hiroki Hamada, Dai Igarashi, and Katsumi Takahashi, 'Rethinking Lightly Verifiable Three-Party Secure Function Computation,' Computer Security Symposium 2010, 2010 (Reference 2)." A secret matching technique that searches for information from secret shared values ​​of a data string while keeping it secret is described, for example, in "Koji Senda, Masayuki Terada, Takayasu Yamaguchi, Dai Igarashi, Hiroki Hamada, and Katsumi Takahashi, 'Secure Matching Protocol Considering Statistical Disclosure Control,' Information Processing Society of Japan Research Report, 2011-CSEC-52(12), 2011 (Reference 3)." A secret sorting technique for sorting a data string while keeping the secret sharing values ​​of the data string secret is described, for example, in "Hiroki Hamada, Dai Igarashi, Koji Senda, Katsumi Takahashi, "Linear Time Sorting on Secure Function Computation," Computer Security Symposium 2011, 2011 (Reference 4)."

[0012] Hereinafter, embodiments of the present invention will be described in detail. Components having the same functions are given the same numbers, and duplicated explanations will be omitted. First embodiment

[0013] FIG. 1 shows an example of the configuration of a secure computation system and a secure computation client device, each consisting of multiple secure computation servers connected via a network. FIG. 2 shows the processing flow of a hyperparameter selection method in a secure computation client device. The secure computation system 10 is composed of X secure computation servers 100-1, ..., X. X may be, for example, 3 or greater, but may be any number greater than or equal to 1 as long as the method uses secure computation, as described below. Existing technologies can be used for the secure computation system 10 (such as the technology disclosed in WO2012 / 046692 (family U.S. Patent US8,989,391B2) (Reference 5)). For example, when X = 3, one piece of data is converted into three shares, and the three secure computation servers 100-1, 100-2, and 100-3 each record one share. Two shares are required to restore the data. In the secure computation system 10, calculations can be performed while maintaining the data in a secretly shared state (anonymous state), and the results are also recorded in anonymized form as shares of the results in each secure computation server 100-x. If the results are statistics, the statistics can be known by obtaining and restoring two or more shares, but the confidentiality of the original data can be maintained.

[0014] The secure computation client device 200 includes a secure computation client unit 210, an AutoML unit 220, a communication unit 280, and a recording unit 290. The communication unit 280 may be connected to a user device 300. In this case, encryption technology may be used for communication between the communication unit 280 and the user device 300. The recording unit 290 records records used for training the AI ​​model. Each record is composed of data of multiple attributes. The attributes include, for example, age, gender, basic medical information (such as the values ​​of blood components), and disease name.

[0015] The secure computation client unit 210 is connected to a secure computation system consisting of multiple secure computation servers 100-1, ..., X, and includes a secret sharing unit 211 and a restoration unit 212. The secure computation client unit 210 transmits shares of instruction information related to machine learning of the AI ​​model to the secure computation system 10, and receives shares of evaluation values ​​of the machine learning of the AI ​​model from the secure computation system 10. Here, "transmitting shares to the secure computation system 10" means transmitting shares corresponding to each secure computation server 100-x, and "receiving shares from the secure computation system 10" means receiving shares recorded by each secure computation server 100-x. In other words, the secret sharing unit 211 converts records, instruction information, etc. into multiple shares and transmits shares corresponding to each secure computation server 100-x. The instruction information is information such as which attribute is used to perform machine learning of the AI ​​model. The restoration unit 212 receives shares of calculation results and shares of evaluation values ​​in the secure computation system 10, and acquires plaintext calculation results and evaluation values.

[0016] The AutoML unit 220 selects hyperparameters for the AI ​​model and instructs the secure computation client unit 210 (S221). The secure computation client unit 210 instructs the secure computation system 10 to perform machine learning of the AI ​​model using secure computation (S211). The secure computation client unit 210 receives a share of the statistics required to calculate the loss from the secure computation system 10 as a share of the evaluation value (S212). The AutoML unit 220 performs cost evaluation of the AI ​​model in plain text based on the evaluation value (S222). The evaluation value is a statistic for calculating the loss of the AI ​​model. The cost evaluation takes into account not only the loss but also the calculation time and memory usage. The secure computation client device 200 checks whether the evaluation has ended (S201). If it has not ended, the AutoML unit 220 repeats the selection of hyperparameters and the cost evaluation. If it has ended, the AutoML unit 220 selects hyperparameters (S223). The calculation time and memory usage may be predicted values, or the actually measured time and memory usage may be used.

[0017] The loss may be calculated using the conventional techniques disclosed in Non-Patent Documents 1 and 2. The "statistics required to calculate the loss" may be the loss itself or a value used to calculate the loss. In other words, the loss itself may be calculated as a statistic in the secure computation system 10, and the secure computation client unit 210 may receive and restore the share of the loss as an evaluation value. Alternatively, the statistics used to calculate the loss may be calculated in the secure computation system 10, and the secure computation client unit 210 may receive and restore the share of the evaluation value, and the loss may be calculated in plaintext. If the data exchanged between the secure computation system 10 and the secure computation client unit 210 is a statistic, the individual data included in the record can be kept confidential.

[0018] There are several possible methods for cost evaluation. For example, memory usage and calculation time can be treated as pure penalties, and the calculation can be done as follows: Cost = Loss + κ × Memory Usage + λ × Calculation Time, where κ and λ are predetermined weighting coefficients.

[0019] When the calculation time is weighted more heavily and the longer the calculation time, the worse the evaluation, the cost evaluation may be calculated as follows, for example: Cost = (loss + κ × memory usage) × λ × calculation time, where κ and λ are predetermined weighting coefficients.

[0020] To avoid over-evaluating models with extremely short times, the cost evaluation may be calculated as follows, for example: Cost = (loss + κ · memory usage) × F (maximum value (λ · calculation time - θ, 0)) where κ and λ are predetermined weighting coefficients, θ is a predetermined constant, and F is a predetermined monotonically non-decreasing function.

[0021] AutoML tunes various machine learning algorithms multiple times, so the computational cost is too high to directly implement it on secure computation. However, with the secure computation client device 200, statistics for calculating loss are calculated using secure computation, and evaluation in the AutoML unit, which takes into account computation time and memory usage, is performed using plain text, so the time required to select hyperparameters for the secure computation AI can be shortened.

[0022] Machine learning is a highly data-dependent technology, with optimal approaches varying depending on the characteristics of the data. From a practical standpoint, it is incompatible with secure computing, which is performed as a general process without looking at the data. However, the AutoML unit 220 automatically tunes, allowing repeated optimal searches even without looking at the data. Depending on the learning model, adopting this method can reduce calculation time by half or one-third. The optimal machine learning model can be automatically searched for while maintaining data confidentiality. Users can select the optimal machine learning model with minimal effort, without worrying about individual learning results. Furthermore, the contents of the learning data are never leaked, except for the number of records, number of attributes, and learning model results. The implementation of the AutoML unit 220 allows data accumulated by multiple companies to be pooled and safely utilized, expanding the applicability and ease of implementation of secure computing AI and broadening the range of users.

[0023] In other words, based on the public information sent to the client (hyperparameters such as model type, accuracy, and likelihood), the AutoML unit 220 selects hyperparameters randomly or strategically through an algorithm. Furthermore, by adding the computation time and memory usage as penalties to the loss function used to evaluate the analysis cost, it can be treated as a comprehensive cost function and optimized. The secure computation server 100-x learns based on the information received from the client and returns the learning results. The AutoML unit determines the quality of the specified model based on the magnitude of the evaluation cost, and can more quickly determine whether to include or exclude it as a candidate for subsequent hyperparameters, or the priority of its selection.

[0024] [Processor, Program, Recording Medium] The functions performed by the components described herein may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to perform the described functions. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes a program stored in a memory.

[0025] In this specification, a circuitry, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0026] If the hardware is a processor considered to be a type of circuitry, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.

[0027] The various processes described above can be implemented by loading a program that executes each step of the above method into the recording unit 2020 of the computer 2000 shown in Figure 3, and operating the control unit 2010, input unit 2030, output unit 2040, display unit 2050, etc.

[0028] The program describing the processing contents can be recorded on a computer-readable recording medium, which may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other suitable recording medium.

[0029] The program may be distributed by, for example, selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to other computers via a network, thereby distributing the program.

[0030] A computer that executes such a program may first temporarily store the program recorded on a portable recording medium or transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored on its own recording medium and executes the process in accordance with the read program. Alternatively, the computer may read the program directly from a portable recording medium and execute the process in accordance with the program. Furthermore, the computer may execute the process in accordance with the program each time a program is transferred from a server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may instead execute the process through a so-called ASP (Application Service Provider) service, which realizes the processing function by issuing an execution instruction and obtaining the results. Furthermore, the server computer may execute the process at the terminal using a so-called SaaS (Software as a Service) service, which allows users to use part of a server computer along with the program. In this embodiment, the program includes information used for processing by an electronic computer that is equivalent to a program (such as data that is not a direct instruction to a computer but has properties that dictate computer processing).

[0031] Furthermore, in this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

[0032] 10 Secure computation system 100 Secure computation server 200 Secure computation client device 210 Secure computation client unit 211 Secret sharing unit 212 Restoration unit 220 AutoML unit 280 Communication unit 290 Recording unit 300 User device

Claims

1. A secure computation client device comprising: a secure computation client unit connected to a secure computation system consisting of one or more secure computation servers, and which transmits shares of instruction information regarding machine learning of an AI model to the secure computation system and receives shares of evaluation values ​​regarding the machine learning of the AI ​​model; and an AutoML unit which selects hyperparameters of the AI ​​model and instructs the secure computation client unit to do so, and evaluates the cost of the AI ​​model in plain text based on the evaluation values, wherein the evaluation values ​​are statistics for calculating the loss of the AI ​​model, and the cost evaluation is an evaluation that takes into account not only the loss but also calculation time and memory usage, and the AutoML unit selects hyperparameters by repeating hyperparameter selection and cost evaluation.

2. A secure computation client device according to claim 1, wherein the cost evaluation is performed based on a cost calculated as follows: Cost = Loss + κ·Memory usage + λ·Computation time, where κ and λ are predetermined weighting coefficients.

3. A secure computation client device according to claim 1, characterized in that the cost evaluation is performed based on a cost calculated as follows: Cost = (loss + κ memory usage) × λ computation time or Cost = (loss + κ memory usage) × F (maximum value (λ computation time - θ, 0)), where κ and λ are predetermined weighting coefficients, θ is a predetermined constant, and F is a predetermined monotonically non-decreasing function.

4. A hyperparameter selection method using a secure computation client device connected to a secure computation system consisting of one or more secure computation servers, comprising the steps of: selecting hyperparameters for an AI model; instructing the secure computation system to perform machine learning of the AI ​​model using secure computation; receiving a share of an evaluation value required to calculate loss from the secure computation system; and cost-evaluating the AI ​​model in plain text based on the evaluation value, wherein the evaluation value is a statistic for calculating the loss of the AI ​​model; the cost evaluation is an evaluation that takes into account calculation time and memory usage in addition to loss; and selecting hyperparameters by repeating processes from hyperparameter selection to cost evaluation.

Images