Estimation device, estimation method, and estimation program

Abstract

To provide a technique that enables estimation of Lasso solutions in the privacy-preserving sparse modeling.SOLUTION: An estimation device 20 is provided, comprising a communication unit 201 for receiving encrypted data encrypted with a random unitary matrix, and an estimation unit 202 configured to derive an estimated value of a regression coefficient of a linear regression model regarding the encrypted data by Lasso regression using ISTA or FISTA for the linear regression model.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an estimation device, an estimation method, and an estimation program.

Background Art

[0002] Cloud / edge computing has been rapidly spreading as a computing resource for big data analysis. However, data that may lead to the identification of an individual is restricted from being used in cloud / edge computing from the perspective of privacy protection.

[0003] Therefore, in cloud / edge computing, the importance of data analysis techniques for analyzing encrypted data has been increasing. For example, in Non-Patent Document 1, in the technique of encrypted sparse modeling, a technique for estimating an estimated value (Lasso solution) of a regression coefficient of a linear regression model for encrypted data by Lasso regression of the linear regression model has been proposed.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, Non-Patent Document 1 only discloses a method for estimating the Lasso solution by the CDA (Coordinate Descent Algorithm), and does not show a method for solving the Lasso solution other than the CDA.

[0006] This disclosure is made in view of the above circumstances and aims to provide a technique for estimating Lasso solutions in concealed sparse modeling techniques. [Means for solving the problem]

[0007] An estimation device according to one aspect of the present disclosure comprises a communication unit that receives encrypted data encrypted with a random unitary matrix, and an estimation unit that calculates estimated values ​​of the regression coefficients of a linear regression model relating to the encrypted data using Lasso regression with respect to ISTA (Iterative Shrinkage Soft-thresholding Algorithm) or FISTA (Fast Iterative Shrinkage Soft-thresholding Algorithm).

[0008] An estimation method according to one aspect of the present disclosure is an estimation method performed by an estimation device, in which encrypted data encrypted with a random unitary matrix is ​​received, and for a linear regression model relating to the encrypted data, an estimated value of the regression coefficient of the linear regression model by Lasso regression is calculated using ISTA (Iterative Shrinkage Soft-thresholding Algorithm) or FISTA (Fast Iterative Shrinkage Soft-thresholding Algorithm).

[0009] An estimation program according to one aspect of this disclosure causes a computer to function as the estimation device. [Effects of the Invention]

[0010] According to this disclosure, a technique for estimating Lasso solutions can be provided in the art of concealed sparse modeling. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows the configuration of the estimation system. [Figure 2]Figure 2 shows the processing flow of the estimation system. [Figure 3] Figure 3 shows the estimation process flow of Lasso solutions by ISTA. [Figure 4] Figure 4 shows the estimation process flow for Lasso solutions using FISTA. [Figure 5] Figure 5 shows the estimated values ​​of the Lasso solution obtained through simulation. [Figure 6] Figure 6 shows the hardware configuration of the estimation device. [Modes for carrying out the invention]

[0012] Embodiments of this disclosure will be described below with reference to the drawings.

[0013] [Configuration of the estimation system] Figure 1 shows the configuration of the estimation system according to this embodiment. The estimation system 1 comprises a local device 10 and an estimation device 20. The local device 10 and the estimation device 20 are connected to each other via a wired or wireless communication network 30.

[0014] Local device 10 is, for example, a sensor that performs observations, or a computer connected to the sensor. As shown in Figure 1, local device 10 comprises a processing unit 101, a communication unit 102, and a storage unit 103. Hereafter, an observation sensor will be used as an example.

[0015] The processing unit 101 has the function of acquiring raw data (observation signals and dictionary data, which is a set of feature vectors for representing the observation signals) of the observation data observed by the local device 10, and encrypting the acquired raw data with a random unitary matrix.

[0016] The communication unit 102 has the function of transmitting encrypted raw data, encrypted with a random unitary matrix, to the estimation device 20 via the communication network 30. The communication unit 102 also has the function of receiving the estimation result of the Lasso solution from the estimation device 20 via the communication network 30.

[0017] The memory unit 103 has the function of storing various types of data handled by the local device 10, such as raw observation data, encrypted raw data, and estimated results of Lasso solutions.

[0018] Next, we will explain the function of the estimation device 20.

[0019] The estimation device 20 is, for example, a cloud / edge computing device equipped with computing resources for big data analysis. As shown in Figure 1, the estimation device 20 comprises a communication unit 201, an estimation unit 202, and a storage unit 203.

[0020] The communication unit 201 has the function of receiving encrypted raw data of observation data from the local device 10 via the communication network 30. The communication unit 201 also has the function of transmitting the estimation result of the Lasso solution to the local device 10 via the communication network 30.

[0021] The estimation unit 202 has the function of estimating the Lasso solution of the regression coefficients of a linear regression model relating to encrypted data using ISTA (Iterative Shrinkage Soft-thresholding Algorithm). In this embodiment, this estimation method is called Secure ISTA (Secure ISTA; Secure Proximity Gradient Method).

[0022] Furthermore, the estimation unit 202 is equipped with the function of estimating and calculating the above Lasso solution using FISTA (Fast Iterative Shrinkage Soft-thresholding Algorithm), which is an accelerated algorithm of ISTA. In this embodiment, this estimation method is called Secure FISTA (Secure FISTA; Secure Accelerated Proximity Gradient Method).

[0023] The memory unit 203 has the function of storing various types of data handled by the estimation device 20, such as encrypted data and estimated results of Lasso solutions.

[0024] [How the estimation system works] Figure 2 shows the processing flow of the estimation system.

[0025] First, the processing unit 101 of the local device 10 acquires the raw data of the observation data observed by the local device 10 (observation signal y, dictionary data A) (step S1).

[0026] Here, we will provide a supplementary explanation of the linear regression model. In a linear regression model, the observed signal y is described by a linear combination of n feature vectors and regression coefficients x. The set of these feature vectors is represented as dictionary data A. Dictionary data A is a set of features that represent the observed signal y, and is represented in the form of a matrix.

[0027] If the observed signal y is a row vector of length p, then the feature vector is also a row vector of length p. The dictionary data A, which is a set of feature vectors, is represented as a p x n matrix. The regression coefficients are row vectors of length n.

[0028] Next, the processing unit 101 of the local device 10 applies a random unitary matrix Q to the acquired raw data. p By multiplying by the obtained raw data (y, A), the encrypted data (y ^ , A ^ ) is generated (step S2). At this time, a random unitary matrix Q is generated. p This is a p x p unitary matrix.

[0029]

number

[0030]

number

[0031] Equation (1) shows the observed signal y after encryption. ^ ( ^is above y). Equation (2) is the encrypted dictionary data A ^ ( ^ is above A

[0032] Next, the communication unit 102 of the local device 10 transmits the encrypted data (y p encrypted with the random unitary matrix Q ^ and A ^ ) to the estimation device 20 via the communication network 30 (step S3).

[0033] At this time, since the data transmitted and held to the estimation device 20 is encrypted, the data content is not known on the cloud / edge side. Therefore, the local user can utilize the computing resources for big data analysis provided by the estimation device 20 while protecting the privacy regarding the data.

[0034] Next, the estimation unit 202 of the estimation device 20 estimates the estimated value (Lasso solution) x p of the regression coefficient x that forms the linear regression model for the linear regression model (y ^ and A ^ ) encrypted with the random unitary matrix Qby Lasso regression of the linear regression model (y ^ = A ^ x) using ISTA or FISTA (step S4). *

[0035] Specifically, first, the estimation unit 202 formulates a linear regression model shown in Equation (3) for the encrypted data (y ^ and A ^ ) transmitted from the estimation device 20 (step S4-1).

[0036]

Equation

[0037] y ^ is the observed signal encrypted with the random unitary matrix Q p and A ^This is a random unitary matrix Q p This is encrypted dictionary data. x is the regression coefficient to be estimated, and is also called a sparse coefficient as it represents a sparse solution.

[0038] Subsequently, the estimation unit 202 calculates the estimated value of the regression coefficient x (Lasso solution) x using Lasso regression. * This is calculated using ISTA or FISTA (Step S4-2). Lasso regression refers to solving the optimization problem shown in equation (4) for the regression coefficient x.

[0039]

number

[0040] λ is any real number greater than '0'.

[0041] Finally, the communication unit 201 of the estimation device 20 transmits the Lasso solution x estimated using ISTA or FISTA via the communication network 30. * Send this to local device 10 (step S5).

[0042] The Lasso solution x* estimated using ISTA or FISTA yields the same solution whether estimated from encrypted data or unencrypted data (raw data). Furthermore, this implies that the regression coefficients are not encrypted, but it is difficult to infer personal information from the observed signals and the data contained in the dictionary data based solely on the regression coefficients.

[0043] In other words, local users are freed from the effort of calculating regression coefficients from raw observation data. Without having to perform the time-consuming calculation of regression coefficients themselves, they can have the high-performance estimation device 20 calculate the same regression coefficients for the same solution as if they were calculated from the raw data.

[0044] Furthermore, the data transmitted to the estimation device 20 for the calculation of regression coefficients is encrypted, and only the regression coefficients are returned from the estimation device 20, thus protecting data privacy from third parties on the cloud / edge or communication network.

[0045] [Estimation of Lasso solution] Next, we will explain that the Lasso solution estimated using ISTA or FISTA is the same whether it is estimated from encrypted data or from unencrypted data (raw data).

[0046] [Estimation of Lasso solutions from unencrypted data by ISTA] First, we will explain "ISTA," which uses ISTA to estimate the Lasso solution from unencrypted data (y, A). Figure 3 shows the processing flow for estimating the Lasso solution using ISTA. Here, we consider solving the optimization problem shown in equation (5) for the regression coefficient x.

[0047]

number

[0048] First, the local device 10 initializes the regression coefficient x[t] to be estimated by setting the counter variable t to '0' (step S101).

[0049]

number

[0050] A T This is the transpose of matrix A.

[0051] Next, the local device 10 sets the first term on the right-hand side of equation (5) as g(x) and finds the vertex v[t] of the quadratic approximation of g(x) by completing the square (step S102).

[0052]

number

[0053]

number

[0054] λ is any real number greater than '0'. L is the Lipschitz coefficient.

[0055] In ISTA, equation (9) is solved sequentially, but equation (9) can be expressed as equation (10) using v[t] obtained by equation (8).

[0056]

number

[0057]

number

[0058] Local device 10 applies the soft decision threshold function S to the calculation result of v[t]. 1 / L Apply the formula and define the result as the regression coefficient x[t+1] (Step S103).

[0059] Finally, the local device 10 determines whether the regression coefficient x[t+1] satisfies a certain termination criterion (step S104).

[0060] If the regression coefficient x[t+1] satisfies the termination criteria, the local device 10 terminates the process, using the regression coefficient x[t+1] as the Lasso solution x*. If the regression coefficient x[t+1] does not satisfy the termination criteria, steps S102 to S104 are repeated until the termination criteria are met.

[0061] [Estimation of Lasso solutions from encrypted data by ISTA] Next, referring to Figure 3 again, we use ISTA to encrypt the data (y ^ , A ^We will explain "Secure ISTA," which estimates the Lasso solution from ). Here, we consider solving the optimization problem shown in equation (4) for the regression coefficient x.

[0062] First, the communication unit 201 of the estimation device 20 initializes the regression coefficient x[t] to be estimated by setting the counter variable t to '0' (step S201).

[0063]

number

[0064] A ^T is, A ^ This is the transpose matrix. The initial value x[0] of the regression coefficient x[t] is the random unitary matrix Q, as shown in equation (12). p The property of Q p T Q p Since =I (where I is the identity matrix), the initial value x[0] of the regression coefficient x[t] before encryption, as shown in equation (6), remains unchanged.

[0065]

number

[0066] Next, the communication unit 201 of the estimation device 20 takes the first term on the right-hand side of equation (4) as g(x) and completes the square to find the vertex v of the quadratic function approximation of g(x). ^ [t]( ^ We find the value of v (step S202).

[0067]

number

[0068]

number

[0069] λ is any real number greater than '0'.^ ( ^ The value above L is the Lipschitz coefficient. The Lipschitz coefficient L for encrypted data. ^ As shown in equation (15), the random unitary matrix Q p Due to its properties, the data remains unchanged before and after encryption.

[0070]

number

[0071] The v shown in equation (14) ^ For [t], as shown in equation (16), the random unitary matrix Q p Due to its properties, the data remains unchanged before and after encryption.

[0072]

number

[0073] Next, the communication unit 201 of the estimation device 20, v ^ The calculation result of [t] is given by the soft decision threshold function S 1 / L^ Apply the formula and define the result as the regression coefficient x[t+1] (Step S203).

[0074]

number

[0075] Evaluation function v ^ Since [t] does not change before and after data encryption, the soft decision threshold function S 1 / L^ It is clear that the regression coefficient x[t+1] after applying the formula does not change before and after data encryption.

[0076] Finally, the communication unit 201 of the estimation device 20 determines whether the regression coefficient x[t+1] satisfies a certain termination criterion (step S204).

[0077] The communication unit 201 of the estimation device 20 terminates processing if the regression coefficient x[t+1] satisfies the termination criteria, using the regression coefficient x[t+1] as the Lasso solution x*. If the regression coefficient x[t+1] does not satisfy the termination criteria, steps S202 to S204 are repeated until the termination criteria are met.

[0078] Thus, a random unitary matrix Q p The property of Q p T Q p Because we are using =I, the Lasso solution estimated using ISTA will be the same whether it is estimated from encrypted data or unencrypted data.

[0079] [Estimation of Lasso solution from encrypted data using FISTA] Next, use FISTA to encrypt the data (y ^ , A ^ We will now explain "Secure FISTA," which estimates the Lasso solution from ). Figure 4 shows the processing flow for estimating the Lasso solution using FISTA. FISTA is an algorithm that can quickly find the Lasso solution by reducing the number of iterations required for ISTA to converge. Here, we consider solving the optimization problem shown in equation (4) for the regression coefficient x.

[0080] First, the communication unit 201 of the estimation device 20 initializes the regression coefficient x[t] to be estimated by setting the counter variable t to '0', and initializes the first acceleration variable β[t] and the second acceleration variable w[t] by setting the counter variable t to '1' (step S301).

[0081]

number

[0082] A ^T is, A ^ This is the transpose matrix. The initial value x[0] of the regression coefficient x[t] is the random unitary matrix Q, as shown in equation (12). p The property of Qp T Q p Since =I, the initial value x[0] of the regression coefficient x[t] before encryption shown in equation (6) remains unchanged.

[0083] Next, the communication unit 201 of the estimation device 20 takes the first term on the right-hand side of equation (4) as g(x) and approximates g(x) as a quadratic function v by completing the square. ^ Find the vertex of [t] (step S302).

[0084]

number

[0085] λ is any real number greater than '0'. ^ L is the Lipschitz coefficient. The Lipschitz coefficient L for encrypted data. ^ As shown in equation (15), the random unitary matrix Q p Due to its properties, it does not change before and after data encryption. Also, as shown in equation (19), v ^ For [t], similar to equation (16), the random unitary matrix Q p Due to its properties, the data remains unchanged before and after encryption.

[0086] Next, the communication unit 201 of the estimation device 20, v ^ The calculation result of [t] is given by the soft decision threshold function S 1 / L^ Apply the formula and define the result as the regression coefficient x[t+1] (step S303).

[0087]

number

[0088] Evaluation function v ^ Since [t] does not change before and after data encryption, the soft decision threshold function S 1 / L^ It is clear that the regression coefficient x[t+1] after applying the formula does not change before and after data encryption.

[0089] In other words, in both step S302 and step S303, the same value is output regardless of whether the data is encrypted or not, based on the same reasoning as ISTA.

[0090] Next, the communication unit 201 of the estimation device 20 updates the first speed-up variable β[t] (step S304).

[0091]

number

[0092] The first optimization variable β[t] is irrelevant to data encryption.

[0093] Next, the communication unit 201 of the estimation device 20 updates the second speed-up variable w[t] (step S305).

[0094]

number

[0095] The first optimization variable β[t] is irrelevant to data encryption, and the regression coefficient x[t+1] does not change before and after data encryption; therefore, the second optimization variable w[t] is also irrelevant to data encryption.

[0096] Finally, the communication unit 201 of the estimation device 20 determines whether the regression coefficient x[t+1] satisfies a certain termination criterion (step S306).

[0097] The communication unit 201 of the estimation device 20 terminates processing if the regression coefficient x[t+1] satisfies the termination criteria, using the regression coefficient x[t+1] as the Lasso solution x*. If the regression coefficient x[t+1] does not satisfy the termination criteria, steps S302 to S306 are repeated until the termination criteria are met.

[0098] Thus, similar to ISTA, the Lasso solution estimated using FISTA yields the same solution whether it is estimated from encrypted data or unencrypted data. FISTA is described in "A. Beck, et al., 'Fast iterative shrinkage-thresholding algorithm for linear inverse problems,' SIAM J. Imaging Sciences, 2009, Vol.2, No.1, pp.183-p.202."

[0099] [Simulation results of the Lasso solution] We simulated whether "ISTA" using unencrypted data and "Secure ISTA" using encrypted data would generate the same Lasso solution. Figure 5 shows the estimated value of the Lasso solution from the simulation. The simulation results using "CDA" were also calculated. The horizontal axis represents the element number of the Lasso solution, and the vertical axis represents the estimated value of the Lasso solution. From Figure 5, it was confirmed that the same value was calculated in all cases: "ISTA," "Secure ISTA," and "CDA."

[0100] [effect] According to this embodiment, the estimation device 20 includes a communication unit 201 that receives encrypted data encrypted with a random unitary matrix, and an estimation unit 202 that calculates estimated values ​​of the regression coefficients of a linear regression model relating to the encrypted data using Lasso regression, using ISTA or FISTA. Therefore, it is possible to provide a technology that can estimate Lasso solutions in secure sparse modeling technology. That is, ISTA and its high-speed algorithm FISTA can be executed in a secure domain, and a variety of solution-finding methods can be applied in secure sparse modeling technology.

[0101] [others] This disclosure is not limited to the embodiments described above. This disclosure can be modified in numerous ways within the scope of its essence. For example, it can be applied to predetermined data other than observational data, or to encrypted data of such predetermined data.

[0102] The estimation device 20 of this embodiment described above can be implemented using a general-purpose computer system, for example, as shown in Figure 6, which includes a CPU 901, memory 902, storage 903, communication device 904, input device 905, and output device 906.

[0103] Memory 902 and storage 903 are storage devices. In this computer system, the CPU 901 executes a predetermined program loaded onto memory 902, thereby realizing each function of the estimation device 20.

[0104] The estimation device 20 may be implemented on a single computer. The estimation device 20 may be implemented on multiple computers. The estimation device 20 may be a virtual machine implemented on a computer.

[0105] The program for the estimation device 20 can be stored on a computer-readable recording medium such as an HDD, SSD, USB memory, CD, or DVD. A computer-readable recording medium is, for example, a non-transitory recording medium. The program for the estimation device 20 can also be distributed via a communication network.

[0106] The local device 10 has the same hardware configuration as the estimation device 20. [Explanation of symbols]

[0107] 1: Estimation System 10: Local devices 20: Estimation device 30: Communication Networks 101: Processing Unit 102: Communications Department 103: Storage section 201: Communications Department 202: Estimation section 203: Storage section 901:CPU 902: Memory 903: Storage 904: Communication device 905: Input device 906: Output device

Claims

1. A communication unit that receives encrypted data encrypted with a random unitary matrix, An estimation unit calculates the estimated values ​​of the regression coefficients of the linear regression model relating to the encrypted data using Lasso regression, using ISTA (Iterative Shrinkage Soft-thresholding Algorithm) or FISTA (Fast Iterative Shrinkage Soft-thresholding Algorithm), An estimation device equipped with the following features.

2. The encrypted data is, The estimation device according to claim 1, comprising encrypted data of predetermined data and encrypted data of dictionary data which is a set of feature vectors for representing the predetermined data.

3. In estimation methods performed by estimation devices, Receive encrypted data encrypted with a random unitary matrix, For the linear regression model relating to the encrypted data, the estimated values ​​of the regression coefficients of the linear regression model are calculated using Lasso regression with ISTA (Iterative Shrinkage Soft-thresholding Algorithm) or FISTA (Fast Iterative Shrinkage Soft-thresholding Algorithm). Estimation method.

4. An estimation program that causes a computer to function as the estimation device described in claim 1.

Images