Model intermediate result processing method, medium, device and program product

Abstract

The invention discloses a model intermediate result processing method and device, equipment and a program product. A data owner includes a first partial model and owns data containing private content. And the data owner obtains a statistical value of the parameter corresponding to the intermediate result of the first partial model by executing a loop. A sample is obtained in each round of circulation, the sample is input into the first part model to obtain a first intermediate result, and the statistical value of the parameter is updated according to the first intermediate result until the statistical value of the parameter converges. And after obtaining a statistical value of the parameter, converting a second intermediate result of the first partial model relative to the input data.

Description

Technical Field

[0001] The embodiments of this specification belong to the field of computer application technology, and in particular, to a method, device, equipment and program product for processing intermediate results of a model. Background Art

[0002] In some scenarios, there is a need to obtain the estimated value of a parameter corresponding to an intermediate result of a neural network model. For example, in a privacy computing scenario, multiple parties will collaborate to use large language models, tree models, and small neural network models. In this scenario, the data is sensitive and certain measures are often required to protect the data. For example, the model can be split horizontally or vertically to protect data privacy.

[0003] In the horizontal split scenario, the data owner owns the data and the computing party owns the model. The data owner will use the part of the model owned by the computing party to obtain the intermediate results corresponding to the input data, and send the intermediate results to the computing party so that the computing party can perform subsequent calculations of the model to obtain the output. However, even if the data owner only passes the intermediate results to the computing party, the computing party can launch a reconstruction attack based on the intermediate results, resulting in the leakage of input data. In order to ensure data privacy, a common method is to use differential privacy methods to perturb the intermediate results. However, differential privacy methods require the determination of the sensitivity of the intermediate results.

[0004] However, since the intermediate results of the model may be infinite, it is impossible to directly determine the estimated values ​​of the sensitivity of the intermediate results. In the prior art, methods such as clipping, traversal or mathematical analysis are often used to analyze the estimated values ​​of the parameters corresponding to the intermediate results.

[0005] Among them, clipping is a method in the differential privacy mechanism to scale the data according to a pre-designed threshold to ensure that its L1 or L2 sensitivity is not higher than the threshold. In this method, the setting of the clipping threshold is very sophisticated. If the clipping threshold is set too small, a lot of information will be lost in the clipping process. If the clipping threshold is set too large, it will cause excessive noise to be added.

[0006] Traversal refers to the method of traversing all situations and calculating statistical values. However, the traversal method has too high a computational overhead, and in some cases, traversal cannot be performed because the input space is infinite.

[0007] The mathematical analysis method is to obtain the estimated value of the parameter of the intermediate result by assuming some conditions and scaling the inequality. The upper and lower bounds obtained by this method are often more relaxed.

[0008] It can be seen that in the prior art methods, it is difficult to obtain accurate estimated values ​​of parameters corresponding to the intermediate results. Summary of the invention

[0009] The purpose of this specification is to provide a method, device, equipment and program product for processing intermediate results of a model.

[0010] In a first aspect, the present specification provides a method for processing an intermediate result of a model, the method being executed by a data owner, the data owner including at least a first part of the model, the data owner owning data including private content, the method comprising:

[0011] S1 to S3 are executed repeatedly until the statistical values ​​of the parameters corresponding to the intermediate results of the first part of the model converge:

[0012] S1 obtains samples;

[0013] S2 inputs the sample into the first part model to obtain a first intermediate result;

[0014] S3: updating the statistical value of the parameter according to the first intermediate result;

[0015] Based on the statistical value of the parameter, a second intermediate result of the first partial model output relative to the data is transformed.

[0016] The second aspect of this specification provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the intermediate result processing method of the model described in the first aspect.

[0017] The third aspect of this specification provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the intermediate result processing method of the model described in the first aspect is implemented.

[0018] A fourth aspect of the present specification provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, performs the steps of the intermediate result processing method of the model described in the first aspect.

[0019] In the intermediate result processing method provided in the embodiment of this specification, the data owner includes the first part of the model and has data containing privacy content. The data owner updates the statistical value of the parameter corresponding to the intermediate result of the first part of the model by executing a loop until the statistical value of the parameter converges. Compared with the clipping method, the above method can give a more accurate estimate of the parameter, ensuring that the clipping does not lose too much information and does not add too strong noise. Compared with the traversal method, the method provided in this specification does not need to traverse all possible situations. The method of iterating by random sampling can obtain the statistical value of the converged parameter only through part of the data, so that the method can complete the estimation of the statistical value of the parameter with high efficiency. In addition, compared with the mathematical analysis method, the method of this specification can obtain more accurate statistical values ​​of the parameters, and can also give confidence intervals of the statistical values. Therefore, the scheme in the embodiment of this specification can obtain more accurate statistical values ​​of the parameters with high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 It is a schematic diagram of an application scenario;

[0022] Figure 2 It is a schematic diagram of another application scenario;

[0023] Figure 3 Flowchart of the intermediate result processing method of the model;

[0024] Figure 4 is a flow chart of a method for processing intermediate results of a model in another embodiment;

[0025] Figure 5 It is a block diagram of an intermediate result processing device of a model in one embodiment. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0027] In view of the problems existing in the prior art, the present application proposes a method for obtaining the statistical value of the parameter corresponding to the intermediate result of the model. The parameter can be the intermediate result itself, the maximum value, the minimum value of the intermediate result, the norm of the difference between the intermediate results corresponding to two similar input data sets, etc.

[0028] Specifically, the data owner includes the first part of the model and owns the data containing privacy content. The data owner obtains the statistical value of the parameter corresponding to the intermediate result of the first part of the model by executing a loop. In each round of the loop, a sample is obtained, the sample is input into the first part of the model to obtain the first intermediate result, and the statistical value of the parameter is updated according to the first intermediate result until the statistical value of the parameter converges. After obtaining the statistical value of the parameter, the second intermediate result of the first part of the model relative to the input data is converted.

[0029] The above method can efficiently obtain more accurate statistical values ​​of parameters. When this method is applied to the privacy computing scenario, the second intermediate result is converted based on the statistical values ​​of the parameters, thereby avoiding the leakage of input data and protecting data security.

[0030] Compared with the clipping method, the above method can give a more accurate estimate of the parameters, ensuring that clipping does not lose too much information or add too much noise.

[0031] Compared with the traversal method, the method provided in this specification does not need to traverse all possible situations. Through the iterative method of random sampling, the statistical value of the converged parameter can be obtained only through part of the data, so that the method can complete the estimation of the statistical value of the parameter more efficiently.

[0032] In addition, compared with the mathematical analysis method, the method of the present specification can obtain more accurate statistical values ​​of parameters and can also provide confidence intervals for the statistical values.

[0033] Next, a method for processing intermediate results of a model proposed in this specification will be described.

[0034] The method is executed by a data owner, the data owner includes at least a first part of a model, and the data owner owns data including private content.

[0035] The first part of the model can be a model obtained by splitting the model horizontally or vertically, and this specification does not limit the method for obtaining the first part of the model. Next, two examples will be used to illustrate the application scenarios of this specification. It should be noted that the following two examples do not limit this specification.

[0036] First, this specification can be applied to the horizontal split scenario. The horizontal split scenario is to split the model horizontally. Figure 1 As shown, it can be Figure 1 The model shown is split into the first three layers and the last two layers. Figure 1 The dotted box in the lower middle section is the first three layers. The dotted box portion may also be the first part of the model, and the first part of the model at least belongs to the data owner. Figure 1 The dotted box in the upper middle is the last two layers, and this part of the model belongs to the computing party. In practical applications, the data owner can input the data into the first part of the model to obtain the intermediate result, and then add disturbance to the intermediate result through the differential privacy method. The disturbed intermediate result is then sent to the computing party, which obtains the output result through the last two layers of the model.

[0037] In the horizontal split scenario, the computing party owns the model, and the data owner owns the data. The computing party shares the first part of the model with the data owner so that the two parties can complete the acquisition of the output data together. For example, the data owner is a hospital, which owns the patient's test data and other data including privacy content. The computing party is a data analysis company, which owns a model that can determine the health status based on the input test data. The data owner can obtain the first part of the model of the computing party, such as the first few layers of the model. The data owner can obtain the estimated value of the intermediate result obtained by inputting the patient data into the first part of the model through the following method, and convert the intermediate result obtained after the patient data is input into the first part of the model through the estimated value to protect the patient's privacy data and prevent the computing party from obtaining the patient's privacy data through data reconstruction methods.

[0038] Secondly, this manual can also be applied to the vertical split scenario. In the vertical split scenario, the model can be split vertically. For example, Figure 2 As shown in the figure, the vertical splitting is to divide the first three layers of the model into two parts. Figure 2 As shown in the dashed box, any dashed box can be the first part of the model mentioned above. These two parts belong to two data owners, the first data owner and the second data owner. Figure 2 As shown, the first data owner owns the partial model corresponding to the dotted box on the left, and the second data owner owns the partial model corresponding to the dotted box on the right. The two data owners input their own data into the two parts respectively to obtain the intermediate results. The intermediate results of the two data owners are then encrypted and aggregated to one participant to calculate the results.

[0039] In the vertical split scenario, the problem that may exist is that the two data owners jointly perform calculations on the first few layers of the model in a secure multi-party computation (MPC) state and aggregate to obtain the final result. In this scenario, it is often necessary to perform nonlinear operations on the ciphertext, such as the Softmax function, and these nonlinear calculations will bring about a large computational overhead. In order to reduce the computational overhead, it is necessary to convert the nonlinear calculation into an approximate linear calculation. For example, for the Softmax function, when it is found that the input data is in the range of 0-10, the calculation result is similar to the result of a linear calculation, then the nonlinear Softmax function can be converted into the above linear calculation. In order to convert to a linear calculation, it is necessary to know the upper and lower bounds of the intermediate results. Similar to the horizontal split scenario, the parameters corresponding to the intermediate results also need to be obtained in the vertical split scenario.

[0040] For example, in a vertical split scenario, both data owners also have data containing private content, and the two parties jointly perform calculations to obtain output results. For example, the first data owner wants to open a coffee shop at a certain address. In order to prevent poor sales after the store is opened, the first data owner needs to obtain coffee shops opened in the area near the address. The second data owner has the opening status of coffee shops in each area. The first data owner can obtain the opening status of coffee shops near the address through the data of the second data owner. For the first data owner, opening a coffee shop at the address is private data and cannot be obtained by the second data owner, otherwise the second data owner may leak the data that the first data owner wants to open a coffee shop at the address to other competing stores, and then other competing stores may also open coffee shops at the address, affecting the store opened by the first data owner.

[0041] In the above scenario, the first data owner can perform fuzzy address matching with the second data owner, so that when the second data owner cannot obtain the address input by the first data owner, the first data owner can query the opening of coffee shops near the address. In the above operation scenario, nonlinear operations between ciphertext and ciphertext or ciphertext and plaintext are involved, such as the Softmax function in neural networks. In the field of cryptography, this nonlinear operation will consume a lot of computing resources. In order to reduce the computing overhead, it is necessary to convert the above nonlinear operation into a linear operation with an approximate result. For nonlinear operations, the approximate linear operations converted are different under different value ranges. Therefore, in order to more accurately convert nonlinear operations into linear operations, it is necessary to obtain the maximum and / or minimum values ​​of the intermediate results. Therefore, the statistical values ​​of the maximum and / or minimum values ​​of the intermediate results can be obtained by the following method to convert the nonlinear operation of the intermediate results into an approximate linear operation.

[0042] Next, we will combine Figure 3 The steps included in the intermediate result processing method of the model shown in this specification are described in detail.

[0043] First of all, it should be noted that the data processed by the model in this specification can be text data, for example, the user's questions can be answered based on the input text data. The data processed by the model in this specification can also be image data, for example, the image data can be classified. In addition, the data processed by the model in this specification can also be other types of data, such as transaction data, medical test data, and address data. This specification does not limit this.

[0044] like Figure 3 As shown, Figure 3 is a flowchart of a method for processing intermediate results of a model according to an exemplary embodiment of the present specification, including:

[0045] Step 301, looping through S1 to S3 until the statistical values ​​of the parameters corresponding to the intermediate results of the first part of the model converge: S1 obtains samples;

[0046] S2 inputs the sample into the first partial model to obtain a first intermediate result;

[0047] S3 updates the statistical value of the parameter according to the first intermediate result.

[0048] Step 301 is a method for obtaining the statistical value of the parameter. The method uses the ideas of the Monte Carlo method and the Welford iteration method. The statistical value can be a digital measurement used to describe the basic characteristics of the parameter, which can usually be used to describe the central tendency, dispersion, distribution form and other characteristics of the data. The statistical value can include at least one of the mean, median, mode, standard deviation and variance.

[0049] Among them, the Monte Carlo method is also called the statistical simulation method, a numerical calculation method guided by probability statistics theory, which usually uses random numbers to solve many calculation problems. Generally speaking, the Monte Carlo method generates a large number of random samples or randomly samples a large number of samples, and then calculates the corresponding results based on the calculation results of these samples. The Monte Carlo method is used to obtain samples in S1, and the statistical values ​​of the first intermediate results of multiple samples are used to estimate the statistical values ​​of the parameters.

[0050] The Welford iteration method is an efficient algorithm for online calculation of the mean and variance of a data set. The ordinary mean and variance calculation method is based on all the data, while the Welford method inputs one sample in one cycle and calculates the mean and variance through multiple cycles. Compared with directly using the mean and variance formula for point-by-point calculation, this method has advantages in numerical stability and computational efficiency. Especially when dealing with large data sets or data streams, the Wilford iteration method is very useful because it does not need to store all data points. This method can obtain more accurate statistical values ​​with high efficiency.

[0051] In the process of obtaining samples in step S1, input samples can be randomly generated according to the form of input data. For example, when the input data is a sentence with a length of n, a sentence with a length of n can be randomly generated according to the vocabulary as an input sample. In addition, samples can also be obtained according to existing data, such as obtaining a data from an existing input data set as a sample.

[0052] The above is an explanation of the method of randomly obtaining a sample. In addition, in some cases, Wilford's method can be improved by obtaining multiple samples in S1 and updating the statistical value in S2 and S3 using the data of the multiple samples. The method of obtaining multiple samples will be described below.

[0053] Two application scenarios are introduced above. One is Figure 1 The sensitivity of the intermediate results is determined in the horizontal split scenario shown in the figure. Another method is Figure 2 The maximum and minimum values ​​of the intermediate results are determined in the vertical split scenario shown in the figure. Next, the parameters in different scenarios will be described in combination with specific scenarios, and the method for obtaining multiple samples in S1 will be described in combination with the selection of parameters.

[0054] First, in Figure 1 In the scenario, the first part of the model can be the first few layers of the model, such as Figure 1 The first three layers are shown. The parameters of the intermediate result may include the norm of the difference between the intermediate results output by the first partial model for the two sets of data, for example, it may be ||M(x)-M(x′)||, where Δ(x,x′)=1, that is, only one data in the two sample sets x and x′ is different. The statistical value of the parameter can be used to determine the sensitivity of the intermediate result output by the first partial model. The sensitivity includes L1 sensitivity and L2 sensitivity. When calculating the statistical value, at least one of the L1 sensitivity and the L2 sensitivity can be calculated.

[0055] This parameter was chosen because Figure 1 In the scenario, we need to calculate the maxx,x′；Δ(x,x′)＝1 ||M(x)-M(x′)||. If the statistical value of the maximum value of the norm is directly calculated, even if the currently updated statistical value has converged, a larger norm value may be obtained in the next loop, which may cause the loop to fail to stop. Therefore, here we calculate the statistical value of the norm (such as the mean or variance, etc.), and then determine the maximum value based on the mean and variance according to the distribution obeyed by the norm, and then determine the sensitivity. This can ensure accuracy.

[0056] Correspondingly, in Figure 1 In the scenario, since the constraint condition of Δ(x,x′)=1 needs to be satisfied, the input samples need to include multiple samples, and the difference between any two sample sets is only one minimum changeable unit.

[0057] Accordingly, the process of obtaining samples may include: randomly generating a sample; for a randomly generated sample, changing any minimum modifiable unit in the sample multiple times to obtain multiple samples. For example, for a model whose input is text, if b input samples need to be generated, a sentence may be randomly generated, and a word in the sentence may be randomly replaced to obtain b-1 sentences.

[0058] Correspondingly, the process of obtaining the local statistical value may include: for any two-by-two combinations of the plurality of first intermediate results, calculating the norm of each two-by-two combination; and calculating the statistical value of the norms of the plurality of two-by-two combinations as the local statistical value.

[0059] Among them, the norm of the pairwise combination is the above-mentioned ||M(x)-M(x′)||.

[0060] Second, in Figure 2 In the scenario shown, the first part of the model can be a vertically split model, for example Figure 2 In this scenario, the data owner mentioned above may be the first data owner or the second data owner, and correspondingly, there may be another data owner, the second data owner or the first data owner in this scenario. The two data owners each own part of the data. The parameter may be the intermediate result itself, and correspondingly, the parameter may be used to determine the maximum value of the intermediate result.

[0061] In addition, when the intermediate result itself is used as a parameter, the statistical value of the parameter may include the mean of the intermediate result output by the first part of the model, and the statistical value of the parameter may be used to determine the mean of the intermediate result output by the first part of the model.

[0062] When the intermediate result itself is used as a parameter, and the statistical value of the parameter includes the mean and variance of the intermediate result output by the first part of the model, in addition to determining the mean of the intermediate result, the statistical value of the parameter can also be used to determine the maximum value, minimum value and / or value range of the intermediate result output by the first part of the model. The specific determination method will be described in step 303 below.

[0063] exist Figure 2 In the scenario shown, since there is no restriction on the relationship between samples, multiple samples can all be randomly generated. Multiple samples can also be multiple samples randomly selected from an existing data set, and this specification does not limit the method for obtaining multiple samples. Correspondingly, calculating local statistical values ​​can directly calculate the statistical values ​​of the first intermediate results of multiple samples. The reason for calculating the statistical value of the intermediate result here instead of calculating the statistical value of the maximum or minimum value of the first intermediate result in multiple samples is the same as the first one, and will not be repeated here.

[0064] In addition, in other scenarios, there may be a need to calculate the mean or value range of the intermediate results. In these scenarios, a calculation method similar to the second one can be used, that is, directly calculating the statistical value of the intermediate results. How to use the statistical value to obtain the maximum value, value range, etc. of the intermediate results will be described below in conjunction with step 303, and will not be repeated here.

[0065] In step S2, the first intermediate result corresponding to the input sample is obtained by operating the first part of the model. In step S3, the parameter value is updated by using the Wilford iteration method.

[0066] Taking the statistical values ​​including variance and mean as an example, the specific updating method can refer to the Wilford iteration method, that is, the following formula (1) and formula (2):

[0067]

[0068] Among them, μ k is the mean updated in the kth cycle, μ k-1 is the mean updated in the k-1th cycle, x k is the parameter corresponding to the first intermediate result of the sample input in this cycle, and k is the number of cycles. Correspondingly, σ k is the variance updated in the kth cycle, σ k-1 is the variance updated in the k-1th cycle. In addition, for the first cycle, the variance can be directly set to 0 and the mean to x k .

[0069] In addition, for x kIn terms of the selection of , when the data to be obtained is the maximum value, minimum value, or sensitivity, it is often necessary to obtain the maximum value of the parameter. When calculating the statistical value, since it is impossible to obtain the maximum value through one input data in one cycle. In order to facilitate the calculation, the maximum value can be used as a parameter instead of x. k Itself is used as a parameter. After calculating the statistical values ​​such as the mean and variance of the parameter, the maximum value can be determined by the sum of the mean and several times the variance according to the distribution that the parameter conforms to. The specific method of obtaining the maximum value based on the parameter will be explained below and will not be repeated here.

[0070] Compared with calculating the mean and variance point by point, the above formula (1) and formula (2) update the mean and variance that have been obtained based on a single input. This calculation method is more efficient for computers and can ensure that the stability of the data will not be lost during the floating-point number calculation process due to the limited length of the floating-point number, making the calculation result more accurate.

[0071] In the above example, the parameter statistics are updated by one sample in one cycle. In another optional implementation, the Wilford iteration method can also be improved, for example, multiple samples can be input in one cycle, and the parameter statistics are updated by multiple samples in each cycle. In this way, since more data are involved in the calculation in one cycle, the number of cycles is reduced and the calculation efficiency is improved on the basis of ensuring the calculation accuracy to a certain extent. The number of samples input in one cycle should be less than a certain value, so that a large amount of data does not need to be stored in each cycle, and the advantages of the Wilford method in reducing storage and ensuring the accuracy of statistics can be brought into play.

[0072] Next, we will explain how to update the statistical value through multiple samples in one cycle.

[0073] The samples obtained in S1 may be multiple samples. Correspondingly, the process of obtaining the first intermediate result in S2 may be: inputting multiple samples into the first partial model to obtain multiple first intermediate results. Then the process of updating the statistical value in S3 may be: calculating the local statistical value of the parameter based on the multiple first intermediate results; and updating the statistical value of the parameter based on the local statistical value and the cycle rounds corresponding to the multiple samples.

[0074] The local statistical value is the statistical value of the parameter corresponding to the first intermediate result of multiple samples in this cycle. For example, if the parameter is the first intermediate result itself and the local statistical value is the local mean of the first intermediate result, then the local statistical value is the mean of multiple first intermediate results.

[0075] It should be noted that in both application scenarios described above, the statistical values ​​can be updated in the following ways.

[0076] In an optional implementation, the statistical value includes at least one of the following: mean, variance.

[0077] As for the specific method of updating the statistical value, first of all, for the mean, the local mean can be used as the x in formula (1) k , the mean is updated in a similar way to formula (1).

[0078] Secondly, for the variance, since multiple samples are input in each cycle, in each cycle, in addition to being affected by the degree of dispersion of the local mean of the current cycle compared to the mean updated in the previous cycle, as shown in formula (2), the variance is also affected by the degree of dispersion between the first intermediate results of the samples in the cycle. Therefore, in the variance update, it is necessary to further introduce the degree of dispersion between the data in the cycle on the basis of formula (2).

[0079] Specifically, the method for updating the variance may include: determining a first variance based on the difference between the local mean and the mean of the parameter, and the cycle rounds corresponding to the multiple samples; the first variance is used to characterize the discreteness between the local mean and the mean of the parameter obtained by the previous cycle update; updating the variance of the parameter according to the first variance, the local variance and the cycle rounds; updating the variance of the parameter according to the local mean and the cycle rounds.

[0080] Among them, the effect of the first variance mentioned above and the The functions are similar, and they are used to describe the degree of dispersion between the parameters obtained in this cycle and the parameters obtained in the previous cycle. The local variance is the variance between the data obtained in this cycle. The local variance can be used to describe the degree of dispersion between the first intermediate results of the samples in this cycle.

[0081] In an optional implementation, the variance and mean may be updated by the following formulas (3) and (4):

[0082]

[0083] In formula (3) and formula (4), the same symbols as those in formula (1) and (2) have the same meanings. new is the local mean, σ new is the local variance. Used to characterize the first variance.

[0084] Step 303: Based on the statistical value of the parameter, convert the second intermediate result of the first partial model relative to the data output.

[0085] After obtaining the statistical value of the parameter, the intermediate result can be converted by the statistical value in step 303. Specifically, the estimated value of the parameter can be determined by the statistical value of the parameter, and the second intermediate result can be further converted according to the estimated value of the parameter to meet the needs of different scenarios.

[0086] It should also be noted that steps 301 and 303 may be performed continuously or discontinuously.

[0087] Next, the specific implementation methods of step 303 in different scenarios will be described. The two examples mentioned above will be used for description.

[0088] First, in Figure 1 In the scenario shown, the sensitivity can first be estimated based on the statistical values ​​of the parameters.

[0089] As for sensitivity, the sensitivity of the intermediate result output by the first part of the model is determined according to the distribution of the parameter and the statistical value of the parameter. For example, assuming that the parameter obeys a normal distribution, the sensitivity can be determined according to the sum of the determined parameter mean and variance.

[0090] For example, the statistical value μ+3σ corresponding to the parameter can be used as the sensitivity, where μ is the mean and σ is the variance. According to the normal distribution, it can be determined that the failure probability of a sensitivity of μ+3σ is 2.7×10 -3 In addition, the sensitivity can also be determined according to the requirements for the failure probability of sensitivity. For example, if the sensitivity requirement is lower, the sensitivity can be determined as μ+2σ, and the failure probability of this sensitivity is 4.66×10 -2 .

[0091] In addition, the sensitivity includes L1 sensitivity and L2 sensitivity, which are calculated by setting the parameters to the L1 norm and L2 norm of ||M(x)-M(x′)||, respectively. When calculating the sensitivity, only the statistical value of the parameter corresponding to the L1 sensitivity or the statistical value of the parameter corresponding to the L2 sensitivity can be calculated.

[0092] Alternatively, the L1 norm and L2 norm of ||M(x)-M(x′)|| can be used as parameters, and the statistical values ​​of the two parameters are updated in each cycle, and the L1 sensitivity and L2 sensitivity are determined based on the statistical values ​​of the two parameters. And the minimum value is taken as the final sensitivity value. In other words, the sensitivity is the minimum value of the L1 sensitivity and the L2 sensitivity of the intermediate result output by the first part of the model. Such sensitivity can be selected more appropriately.

[0093] After calculating the sensitivity, the intermediate results can be perturbed according to the sensitivity to prevent the computing party from performing reconstruction attacks based on the intermediate results and protect the security of the data owner's data.

[0094] In other words, the method further involves a computing party, and the computing party has at least a second partial model of the model; step 303 includes: based on the sensitivity, perturbing the second intermediate result using a differential privacy method. After obtaining the perturbed second intermediate result, the perturbed second intermediate result can also be sent to the computing party, so that the computing party obtains the output of the model using the second partial model according to the perturbed second intermediate result.

[0095] The second part model refers to the part of the model other than the first part model. The input of the second part model can be the intermediate result, and the output is the output of the model composed of the first part model and the second part model. After determining the sensitivity, the intermediate result of the output of the first part model can be perturbed. The perturbed intermediate result is sent to the calculation party so that the calculation party can input the intermediate result into the second part model to obtain the output of the model.

[0096] The difference between the first intermediate result and the second intermediate result lies in the difference in input data, and the acquisition methods of the two are the same. The first intermediate result is the intermediate result obtained by inputting the samples obtained in each cycle into the first part of the model, while the second intermediate result is the intermediate result obtained by inputting the data owned by the data owner into the first part of the model.

[0097] Second, in Figure 2 In the scenario shown, similar to the sensitivity determination method, the maximum value and / or minimum value of the intermediate result output by the first part of the model can be determined based on the distribution obeyed by the parameter and the statistical value of the parameter.

[0098] In addition, in other scenarios, if there is a need to determine the mean and / or value range of the intermediate results output by the first part of the model, it can also be determined based on the distribution obeyed by the parameters and the statistical values ​​of the parameters.

[0099] For example, assuming that the parameter follows a normal distribution, the mean μ in the statistical value of the parameter can be used as the mean of the intermediate result output by the first part of the model. The sum of the mean μ of the parameter and several times the variance σ can be used as the maximum value of the intermediate result. For example, μ+3σ can be used as the maximum value of the intermediate result output by the first part of the model. The failure probability of this maximum value is 2.7×10 -3The difference between the mean value of the parameter and several times the variance can be used as the minimum value of the above intermediate results. For example, μ-3σ can be used as the minimum value of the intermediate results output by the first part of the model. The failure probability of this minimum value is also 2.7×10 -3 As for the value range of the intermediate result output by the first part of the model, the value range can also be determined based on the determined minimum value and maximum value.

[0100] In the vertical splitting scenario, after determining the maximum and / or minimum value of the intermediate result, step 303 may include: based on the maximum or minimum value, converting the nonlinear operation in the multi-party secure computation MPC into a linear operation; based on the linear operation, performing an operation on the second intermediate result.

[0101] As mentioned above, in the vertical split scenario, since nonlinear operations consume more computing resources, it is necessary to convert nonlinear operations into linear operations consisting of addition, subtraction, multiplication, and division with similar operation results to complete the calculations required by the MPC scenario. For different nonlinear operations, the linear operations with similar operation results are different under different value ranges. Therefore, by determining the maximum and / or minimum value, it is more convenient to determine which linear operation to convert to.

[0102] The above two examples do not limit the present application. It is easy to understand that in other scenarios, if it is necessary to obtain the parameters of a certain intermediate result, it can also be obtained according to the above method.

[0103] Next, the method shown in this specification will be explained through a specific example of determining L2 sensitivity in differential privacy. The data that needs to be differentially private in this scenario is the intermediate result obtained by inputting the data into the first part of the model in the horizontal split scenario. The model in this scenario can be a model for processing text data.

[0104] like Figure 4 As shown, this scenario includes the following steps.

[0105] Step 401, determine the given model M and the corresponding statistical value G for which the statistical value needs to be calculated. In the sensitivity scenario, G = max x,x′；Δ(x,x′)＝1 ||M(x)-M(x′)||. M is the first part of the model.

[0106] Step 402: Generate b model input samples {x1, x2, ..., x b Specifically, a sentence is randomly generated first, and then each word in the sentence is randomly replaced to obtain the remaining b-1 sentences.

[0107] Step 403, calculate the local statistics of the batch of sentence samples. For the b sentences, combine them in pairs, and calculate the L2 norm of the difference between the two sentences in each combination corresponding to the intermediate results of the model. And calculate the average value μ for the L2 norm of all the differences. new and variance σ new .

[0108] Step 404, determine whether it is the first iteration:

[0109] Step 404a. is the first iteration, and the local mean and local variance of the L2 norm of the current batch of data are used as the overall mean and variance, and jump to step 402.

[0110] Step 404b: If it is not the first iteration, the mean and variance are updated by a recursive formula. The mean and variance update formulas refer to formula (3) and formula (4).

[0111] Step 405, determine whether convergence has occurred. Determine whether the mean and variance of the current round are close to 0 compared to the mean and variance of the previous round. If they are close to 0, convergence has occurred, and the process goes to step 407. If not, the process goes to step 406.

[0112] For example, it can be determined whether the difference between the mean of the current round and the mean of the previous round is less than a first preset threshold, and whether the difference between the variance of the current round and the variance of the previous round is less than a second preset threshold. The first preset threshold and the second preset threshold can be the same or different. When both differences are less than the threshold, it is confirmed that convergence has occurred.

[0113] Step 406, determine whether the upper limit of the number of iterations is reached. If the upper limit of the number of iterations is not reached, jump back to step 402 to continue. If the upper limit of the number of iterations is reached, continue to step 407.

[0114] Step 407, output μ and σ.

[0115] After step 407, the sensitivity G can be calculated according to the method mentioned in step 403 above, and the intermediate result can be processed. The specific processing method will not be described in detail here.

[0116] Through the above method, in the case of large models where it is impossible for humans to accurately calculate the statistical value, an experimental science method is proposed as a supplement, which can give a more accurate estimate of the statistical value and give a confidence level. In addition, the improved batch Welford iteration method calculates the statistics of the cumulative samples without storing all the samples, and supports batch processing to improve the calculation speed, and can achieve better results under limited computing power. An adaptive method for calculating model statistics is proposed: for the model, the statistics are infinitely approximated; for L1 and L2 sensitivities, the norm with less influence is freely selected. The whole method is flexible and lightweight, and is suitable for various scenarios.

[0117] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0118] This specification also provides a model intermediate result processing device, which is applied to a data owner, the data owner at least includes the first part of the model, and the data owner owns data including privacy content. Figure 5 As shown, the device is used to perform Figure 3 or Figure 4 The method comprises the steps of:

[0119] The loop unit 510 is used to execute S1 to S3 in a loop until the statistical value of the parameter corresponding to the intermediate result of the first part of the model converges. S1 obtains a sample; S2 inputs the sample into the first part of the model to obtain a first intermediate result; S3 updates the statistical value of the parameter according to the first intermediate result.

[0120] The conversion unit 520 converts a second intermediate result of the first partial model relative to the data output based on the statistical value of the parameter.

[0121] In an optional embodiment, the loop unit 510 is specifically used to obtain samples; input multiple samples into the first part model to obtain multiple first intermediate results; calculate the local statistical value of the parameter based on the multiple first intermediate results; and update the statistical value of the parameter based on the local statistical value and the loop rounds corresponding to the multiple samples.

[0122] In an optional implementation, the statistical value includes at least one of the following: mean, variance.

[0123] In an optional embodiment, the loop unit 510 is specifically used to obtain samples; input multiple samples into the first part model to obtain multiple first intermediate results; calculate the local statistical value of the parameter based on the multiple first intermediate results; determine the first variance based on the difference between the local mean and the mean of the parameter, and the cycle rounds corresponding to the multiple samples; the first variance is used to characterize the discreteness between the local mean and the mean of the parameter obtained by the previous cycle update; update the variance of the parameter according to the first variance, the local variance and the cycle rounds; update the variance of the parameter according to the local mean and the cycle rounds.

[0124] In an optional implementation, the parameter includes the norm of the difference between the intermediate results output by the first partial model for two sets of data, and the statistical value of the parameter is used to determine the sensitivity of the intermediate result output by the first partial model; the loop unit 510 is specifically used to randomly generate a sample; for a randomly generated sample, change any minimum changeable unit in the sample multiple times to obtain multiple samples; for any two-by-two combinations of the multiple first intermediate results, calculate the norm of each two-by-two combination; calculate the statistical value of the norm of multiple two-by-two combinations as a local statistical value. Based on the local statistical value and the number of loops corresponding to the multiple samples, update the statistical value of the parameter.

[0125] In an optional implementation, the sensitivity is the minimum value of L1 sensitivity and L2 sensitivity of the intermediate result output by the first partial model.

[0126] In an optional embodiment, the method further involves a computing party, the computing party having at least a second partial model of the model; a conversion unit 520, specifically configured to perturb the second intermediate result based on the sensitivity using a differential privacy method. The device also includes a sending unit 330 (not shown in the figure), configured to send the perturbed second intermediate result to the computing party, so that the computing party obtains the output of the model using the second partial model according to the perturbed second intermediate result.

[0127] In an optional embodiment, the statistical value of the parameter includes the mean and variance of the intermediate result output by the first partial model, and the statistical value of the parameter is used to determine the mean, maximum value, minimum value and / or value range of the intermediate result output by the first partial model.

[0128] In an optional implementation, the sensitivity, mean, maximum value, minimum value and / or value range of the intermediate result output by the first partial model is determined according to the distribution obeyed by the parameter and the statistical value of the parameter.

[0129] In an optional embodiment, the statistical value of the parameter is used to determine the maximum or minimum value of the intermediate result output by the first part of the model; the conversion unit 520 is specifically used to convert the nonlinear operation in the multi-party secure computation MPC into a linear operation based on the maximum or minimum value; based on the linear operation, the second intermediate result is operated.

[0130] For the convenience of description, the above devices are described in various units according to their functions. Of course, when implementing one or more of the present specification, the functions of each unit can be implemented in the same or more software and / or hardware, or the unit implementing the same function can be implemented by a combination of multiple sub-units or sub-modules, etc. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0131] The present specification also provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned method for processing intermediate results of a model when executed by a processor.

[0132] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the intermediate result processing method of the model is implemented.

[0133] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0134] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0135] The present specification also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the intermediate result processing method of the model.

[0136] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, graphene storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0137] In addition, the method provided in this specification can be implemented by a controller. The controller can be implemented in any appropriate manner. For example, the controller can take the form of a computer-readable medium, such as a microprocessor or processor and a computer-readable program code (such as software or firmware) that can be executed by the (micro) processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the device for implementing various functions included therein can also be regarded as a structure within the hardware component. Or even, the device for implementing various functions can be regarded as both a software module for implementing the method and a structure within the hardware component.

[0138] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a server system. Of course, the present application does not exclude that with the development of computer technology in the future, the computer that implements the functions of the above embodiments may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0139] Although one or more embodiments of the present specification provide the method operation steps described in the flow chart, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiment is only one way of executing the order of many steps, and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements. For example, if the first, second and other words are used to represent the name, they do not represent any specific order.

[0140] This specification is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0141] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0143] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0145] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0146] The above description is only an example of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. For those skilled in the art, one or more embodiments of the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims.

Claims

1. A method for processing an intermediate result of a model, the method being executed by a data owner, the data owner including at least a first part of the model, the data owner owning data including private content, the method comprising: S1 to S3 are executed repeatedly until the statistical values ​​of the parameters corresponding to the intermediate results of the first part of the model converge: S1 obtains samples; S2 inputs the sample into the first part model to obtain a first intermediate result; S3: updating the statistical value of the parameter according to the first intermediate result; Based on the statistical value of the parameter, a second intermediate result of the first partial model output relative to the data is transformed.

2. According to the method of claim 1, inputting the sample into the first partial model to obtain the first intermediate result comprises: Inputting a plurality of samples into the first part of the model to obtain a plurality of first intermediate results; The updating of the statistical value of the parameter according to the first intermediate result includes: Calculating a local statistical value of the parameter based on the plurality of first intermediate results; Based on the local statistical value and the cycle rounds corresponding to the multiple samples, the statistical value of the parameter is updated.

3. The method according to claim 2, wherein: The statistical value includes at least one of the following: mean and variance.

4. The method according to claim 3, wherein updating the statistical value of the parameter based on the local statistical value and the cycle rounds corresponding to the plurality of samples comprises: Determine a first variance based on a difference between a local mean and a mean of the parameter, and cycle rounds corresponding to the plurality of samples; The first variance is used to characterize the discreteness between the local mean and the mean of the parameter updated in the previous cycle; Update the variance of the parameter according to the first variance, the local variance and the cycle round; The variance of the parameter is updated according to the local mean and the cycle round.

5. The method according to claim 2, wherein: The parameter includes the norm of the difference between the intermediate results respectively output by the first part of the model for the two groups of data, and the statistical value of the parameter is used to determine the sensitivity of the intermediate result output by the first part of the model; The obtaining of samples comprises: Generate a random sample; For a randomly generated sample, any minimum changeable unit in the sample is changed multiple times to obtain multiple samples; The calculating the local statistical value of the parameter based on the plurality of first intermediate results comprises: For any two-by-two combinations of the plurality of first intermediate results, calculate the norm of each two-by-two combination; Calculate the statistics of the norms of multiple pairwise combinations as local statistics.

6. The method according to claim 5, wherein: The sensitivity is the minimum value of the L1 sensitivity and the L2 sensitivity of the intermediate result output by the first partial model.

7. The method according to claim 5, wherein: The method further involves a computing party, the computing party owning a second partial model including at least the model; The converting, based on the statistical value of the parameter, a second intermediate result of the first partial model relative to the data output comprises: Based on the sensitivity, perturbing the second intermediate result using a differential privacy method; The method further comprises: The disturbed second intermediate result is sent to the computing party, so that the computing party obtains the output of the model by using the second partial model according to the disturbed second intermediate result.

8. The method according to claim 1, wherein: The statistical value of the parameter includes the mean and variance of the intermediate result output by the first partial model, and the statistical value of the parameter is used to determine the mean, maximum value, minimum value and / or value range of the intermediate result output by the first partial model.

9. The method according to claim 5 or 8, wherein: The sensitivity, mean, maximum value, minimum value and / or value range of the intermediate result output by the first part of the model is determined according to the distribution obeyed by the parameter and the statistical value of the parameter.

10. The method according to claim 8, wherein: The statistical value of the parameter is used to determine the maximum or minimum value of the intermediate result output by the first part of the model; The converting, based on the statistical value of the parameter, a second intermediate result of the first partial model relative to the data output comprises: Based on the maximum value or the minimum value, convert the nonlinear operation in the multi-party secure computation MPC into a linear operation; Based on the linear operation, an operation is performed on the second intermediate result.

11. A computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 10 is implemented.

Images