A model intermediate result processing method, medium, device, and program product

Abstract

A model intermediate result processing method, device, equipment and program product. A data owner includes a first part model, and owns data containing privacy content. The data owner obtains a statistical value of a parameter corresponding to an intermediate result of the first part model by performing a loop. In each round of the loop, a sample is obtained, the sample is input into the first part model to obtain a first intermediate result, and the statistical value of the parameter is updated according to the first intermediate result until the statistical value of the parameter converges. After obtaining the statistical value of the parameter, a second intermediate result of the first part model relative to the input data is converted.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification belong to the technical field of computer application, and particularly relate to a model intermediate result processing method and device, equipment and program product. BACKGROUND

[0002] In some scenarios, there is a demand for obtaining an estimated value of a parameter corresponding to a certain intermediate result of a neural network model. For example, in a privacy computing scenario, multiple parties will cooperatively use large language models, tree models, and small neural network models, etc. In this scenario, data is sensitive, and certain measures need to be taken to protect data privacy. For example, the model can be horizontally or vertically split to protect data privacy.

[0003] In the horizontal splitting scenario, the data owner has the data, and the computing party has the model. The data owner obtains the intermediate result corresponding to the input data by using the part of the model owned by the computing party, and sends the intermediate result to the computing party, so that the computing party can perform subsequent calculation of the model to obtain the output. However, even if the data owner only transmits the intermediate result to the computing party, the computing party can launch a reconstruction attack according to the intermediate result, resulting in the leakage of the input data. In order to ensure data privacy, a common method is to use a differential privacy method to perturb the intermediate result. However, the differential privacy method requires determining the sensitivity of the intermediate result.

[0004] However, since there can be an infinite number of intermediate results of the model, it is impossible to directly determine the estimated value of the sensitivity, etc. of the intermediate result. In the prior art, methods such as clipping, traversal, or mathematical analysis are often used to analyze the estimated value of the parameter corresponding to the intermediate result.

[0005] Among them, clipping is a method of scaling data according to a pre-designed threshold in the differential privacy mechanism to ensure that the L1 or L2 sensitivity is not higher than the threshold. In this method, the setting of the clipping threshold is very important. If the clipping threshold is set too small, a large amount of information will be lost in the clipping process, and if the clipping threshold is set too large, excessive noise will be added.

[0006] Traversal refers to a method of traversing all cases and calculating statistical values. However, the traversal method has a large computational overhead, and in some cases, it is impossible to traverse due to the infinite input space.

[0007] The mathematical analysis method is a method of obtaining the estimated value of the parameter of the intermediate result by assuming some conditions and inequality scaling. However, the upper and lower bounds obtained by this method are often more relaxed.

[0008] As can be seen, in the prior art method, it is difficult to obtain an accurate estimated value of the parameter corresponding to the intermediate result. SUMMARY

[0009] An object of the present specification is to provide a model intermediate result processing method, device, equipment and program product.

[0010] The first aspect of the present specification provides a model intermediate result processing method, which is performed by a data owner, the data owner comprising at least a first partial model of a model, the data owner owning data comprising private content, the method comprising:

[0011] S1 to S3 are executed in cycles until the statistical value of the parameter corresponding to the intermediate result of the first partial model converges:

[0012] S1 obtains a sample;

[0013] S2 inputs the sample into the first partial model to obtain a first intermediate result;

[0014] S3 updates the statistical value of the parameter according to the first intermediate result;

[0015] Based on the statistical value of the parameter, a second intermediate result output by the first partial model with respect to the data is converted.

[0016] The second aspect of the present specification provides a computer readable storage medium having a computer program stored thereon, when the computer program is executed in a computer, the computer program causes the computer to execute the model intermediate result processing method of the first aspect.

[0017] The third aspect of the present specification provides a computing device comprising a memory and a processor, the memory storing executable code, and the processor executing the executable code to implement the model intermediate result processing method of the first aspect.

[0018] The fourth aspect of the present specification provides a computer program product comprising computer programs / instructions, which, when executed by a processor, implement the steps of the model intermediate result processing method of the first aspect.

[0019] In the intermediate result processing method provided in the embodiments of the present specification, the data owner includes a first part model and has data containing privacy content, and the data owner updates a statistical value of a parameter corresponding to the intermediate result of the first part model by performing a loop until the statistical value of the parameter converges. Compared with the pruning method, the above method can give a more accurate estimated value of the parameter, ensuring that pruning does not lose too much information and does not add too strong noise. Compared with the traversal method, the method provided in the present specification does not need to traverse all possible cases, and the statistical value of the parameter converges through the method of iteration by random sampling, so that the method can complete the estimation of the statistical value of the parameter with high efficiency. In addition, compared with the mathematical analysis method, the method of the present specification can obtain a more accurate statistical value of the parameter and can give a confidence interval of the statistical value. Therefore, the scheme in the embodiments of the present specification can obtain a more accurate statistical value of the parameter with high efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0021] Figure 1 is a schematic diagram of an application scenario;

[0022] Figure 2 is a schematic diagram of another application scenario;

[0023] Figure 3 is a flowchart of an intermediate result processing method of a model;

[0024] Figure 4 is a flowchart of an intermediate result processing method of a model in another embodiment;

[0025] Figure 5 is a block diagram of an intermediate result processing device of a model in an embodiment. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be described clearly and completely below with reference to the drawings in the embodiments of the present specification. Obviously, the described embodiments are only some embodiments of the present specification, not all embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present specification.

[0027] To solve the problems in the prior art, the present application provides a method for obtaining a statistical value of a parameter corresponding to an intermediate result of a model. The parameter can be the intermediate result itself, a maximum value of the intermediate result, a minimum value of the intermediate result, a norm of a difference between intermediate results corresponding to two similar input data sets, etc.

[0028] Specifically, a data owner includes a first part of a model and has data containing privacy content. The data owner obtains a statistical value of a parameter corresponding to an intermediate result of the first part of the model by performing a loop. In each round of the loop, a sample is obtained, the sample is input into the first part of the model to obtain a first intermediate result, and the statistical value of the parameter is updated according to the first intermediate result until the statistical value of the parameter converges. After obtaining the statistical value of the parameter, a second intermediate result of the first part of the model with respect to input data is converted.

[0029] The above method can efficiently obtain a more accurate statistical value of the parameter. In a privacy computing scenario, the second intermediate result is converted based on the statistical value of the parameter, which avoids leakage of the input data and protects data security.

[0030] Compared with the clipping method, the above method can give a more accurate estimate of the parameter, ensuring that clipping does not lose too much information and does not add too much noise.

[0031] Compared with the traversal method, the method provided in the present specification does not need to traverse all possible cases. The method iterates by random sampling and can obtain a convergent statistical value of the parameter from only part of the data, so that the method can efficiently estimate the statistical value of the parameter.

[0032] In addition, compared with the mathematical analysis method, the method of the present specification can obtain a more accurate statistical value of the parameter and can also give a confidence interval of the statistical value.

[0033] Next, a method for processing an intermediate result of a model proposed in the present specification will be described.

[0034] The method is performed by a data owner, the data owner including at least a first part of a model, and the data owner having data containing privacy content.

[0035] The first part of the model can be a model obtained by horizontally or vertically splitting the model, and the present specification does not limit the method for obtaining the first part of the model. Next, two examples will be used to illustrate the application scenarios of the present specification. It should be noted that the following two examples do not limit the present specification.

[0036] First, the present specification can be applied to a horizontally split scenario. In the horizontally split scenario, the model is horizontally split. For example,Figure 1 As shown, it can be Figure 1 The model shown is divided into three layers in the front and two layers in the back. Figure 1 The dashed box at the bottom center represents the first three layers. This dashed box area can also be considered the first part of the model, which belongs at least to the data owner. Figure 1 The dashed box at the top center represents the last two layers, which belong to the computation side of the model. In practical applications, the data owner can input data into the first part of the model to obtain intermediate results, and then add perturbations to the intermediate results using a differential privacy method. The perturbated intermediate results are then sent to the computation side, which obtains the output results through the last two layers of the model.

[0037] In a horizontally partitioned scenario, the computational party owns the model, and the data owner owns the data. The computational party shares the first part of the model with the data owner so that both parties can jointly acquire the output data. For example, the data owner might be a hospital, possessing patient test data and other data containing sensitive information. The computational party might be a data analytics company, possessing a model that can determine health status based on input test data. The data owner can obtain the first part of the computational party's model, such as the first few layers. The data owner can obtain estimates of the intermediate results obtained from inputting patient data into the first part of the model using the following methods. These estimates can then be used to transform the intermediate results obtained after inputting patient data into the first part of the model, thereby protecting the patient's privacy and preventing the computational party from obtaining the patient's privacy data through data reconstruction.

[0038] Secondly, this instruction manual can also be applied to scenarios involving vertical splitting. In vertical splitting scenarios, the model can be split vertically. For example, ... Figure 2 As shown, vertical splitting means dividing the first three layers of the model into two parts, as shown in the diagram. Figure 2 As shown by the dashed boxes, any one of the dashed boxes can represent the first part of the model described above. These two parts belong to two data owners, the first data owner and the second data owner, respectively. For example... Figure 2 As shown, the first data owner possesses the portion of the model corresponding to the left dashed box, and the second data owner possesses the portion of the model corresponding to the right dashed box. The two data owners input their respective data into the two parts to obtain intermediate results. These intermediate results are then encrypted and aggregated into a single participant for final calculation.

[0039] In the vertical splitting scenario, a possible problem is that two data owners jointly perform the calculation of the first several layers of the model in a secure multi-party computation (MPC) state and aggregate the final result. In this scenario, it is often necessary to perform nonlinear operations on ciphertexts, such as the Softmax function, and these nonlinear calculations will bring a large calculation overhead. In order to reduce the calculation overhead, it is necessary to convert the nonlinear calculation into an approximate linear calculation. For example, for the Softmax function, it is found that when the input data is in the range of 0-10, the calculation result is similar to the result of a certain linear calculation, so the nonlinear Softmax function can be converted into the above linear calculation. In order to convert into a linear calculation, the upper and lower bounds of the intermediate result need to be known. Similar to the horizontal splitting scenario, the parameters corresponding to the intermediate result also need to be obtained in the vertical splitting scenario.

[0040] For example, in the vertical splitting scenario, both data owners also have data including private content, and the two parties jointly perform calculations to obtain output results. For example, the first data owner wants to open a coffee shop at a certain address, in order to prevent poor sales after opening the shop, the first data owner needs to obtain the opening of coffee shops in the vicinity of the address. The second data owner has the opening of coffee shops in each area. The first data owner can obtain the opening of coffee shops near the address through the data of the second data owner. For the first data owner, opening a coffee shop at the address is private data and cannot be obtained by the second data owner, otherwise the second data owner may disclose the data of the first data owner to other competing stores that want to open a coffee shop at the address, and then other competing stores may also open a coffee shop at the address, affecting the opening of the first data owner's store.

[0041] In the above scenario, the first data owner can perform fuzzy address matching with the second data owner to query the opening of coffee shops near the address in the case where the second data owner cannot obtain the address input by the first data owner. In the above operation scenario, nonlinear operations between ciphertexts and ciphertexts or ciphertexts and plaintexts are involved, such as the Softmax function in neural networks. In the field of cryptography, such nonlinear operations will consume a large amount of computing resources. In order to reduce the calculation overhead, it is necessary to convert the above nonlinear operation into a linear operation with approximate results. For nonlinear operations, the approximate linear operation converted is different under different value ranges. Therefore, in order to more accurately convert the nonlinear operation into a linear operation, the maximum and / or minimum value of the intermediate result needs to be obtained. Therefore, the statistical value of the maximum and / or minimum value of the intermediate result can be obtained by the following method to convert the nonlinear operation of the intermediate result into an approximate linear operation.

[0042] Next, the model intermediate result processing method according to an example embodiment will be described in detail with reference to the accompanying drawings. Figure 3 The steps included in the intermediate result processing method of the model shown in the specification will be described in detail.

[0043] First of all, it needs to be pointed out that the data processed by the model in the specification can be text data, such as answering the user's question according to the input text data. The data processed by the model in the specification can also be image data, such as classifying image data. In addition, the data processed by the model in the specification can also be other types of data, such as transaction data, medical detection data, and address data. The specification does not limit this.

[0044] As shown in Figure 3 , the flowchart of the intermediate result processing method of a model according to an example embodiment shown in the specification includes: Figure 3

[0045] Step 301, loop execution S1 to S3 until the statistical value of the parameter corresponding to the intermediate result of the first part model converges: S1 obtains a sample;

[0046] S2 inputs the sample into the first part model to obtain a first intermediate result;

[0047] S3 updates the statistical value of the parameter according to the first intermediate result.

[0048] Step 301 is the method of obtaining the statistical value of the parameter. The method uses the idea of Monte Carlo method and Welford iteration method. Among them, the statistical value can be a numerical measure for describing the basic characteristics of the parameter, which can usually be used to describe the central tendency, dispersion, distribution form and other characteristics of the data. The statistical value can include at least one of the mean, median, mode, standard deviation, and variance.

[0049] Among them, the Monte Carlo method is also called statistical simulation method, a numerical calculation method guided by probability and statistics theory, which usually uses random numbers to solve many calculation problems. Generally speaking, Monte Carlo method will generate a large number of random samples or sample a large number of samples, and then calculate the corresponding results based on the calculation results of these samples. The sample obtained in S1 adopts Monte Carlo method, and the statistical value of the first intermediate result of multiple samples is used to estimate the statistical value of the parameter.

[0050] ​The Welford iteration method is an efficient algorithm for calculating the mean and variance of a data set online. The common method for calculating the mean and variance is based on all data, while the Welford method inputs a sample in one cycle and calculates the mean and variance through multiple cycle iterations. Compared with direct point-by-point calculation using the mean and variance formula, this method has advantages in numerical stability and calculation efficiency. In particular, when dealing with large data sets or data streams, the Welford iteration method is very useful because it does not need to store all data points. Through this method, more accurate statistical values can be obtained with high efficiency.

[0051] In the process of obtaining samples in step S1, input samples can be randomly generated according to the form of input data. For example, in the case of input data being sentences with a sentence length of n, a sentence with a sentence length of n can be randomly generated as an input sample according to a word table. In addition, samples can also be obtained from existing data, such as obtaining a data from an existing input data set as a sample.

[0052] The above is a description of the method of randomly obtaining a sample. In addition, in some cases, the Welford method can be improved to obtain multiple samples in S1, and the statistical values are updated through the data of multiple samples in S2 and S3. Next, the method of obtaining multiple samples will be introduced.

[0053] In the above, two application scenarios are introduced, one is to determine the sensitivity of the intermediate result in the horizontal split scenario as shown in Figure 1 , and the other is to determine the maximum and minimum values of the intermediate result in the vertical split scenario as shown in Figure 2 . Next, the parameters in different scenarios will be described in combination with specific scenarios, and the method of obtaining multiple samples in S1 will be described in combination with the selection of parameters.

[0054] First, in the scenario of Figure 1 , the first part of the model can be the first few layers of the model, such as the first 3 layers as shown in Figure 1 . The parameters of the intermediate result can include the norm of the difference between the intermediate results output by the first part of the model for the two groups of data, such as ||M(x)-M(x')||, where Δ(x,x')=1, that is, only one data is different between the two samples x and x'. The statistical value of this parameter can be used to determine the sensitivity of the intermediate result output by the first part of the model. The sensitivity includes L1 sensitivity and L2 sensitivity, and at least one of the L1 sensitivity and the L2 sensitivity can be calculated when calculating the statistical value.

[0055] The reason for selecting this parameter is that in the scenario of Figure 1 , the maximum value max and the minimum value min need to be calculatedx,x′；Δ(x,x′)＝1 ||M(x)-M(x')||. If the maximum value of the norm is directly calculated, even if the current updated statistical value has converged, a larger norm value can be obtained in the next cycle, which can cause the cycle to stop. Therefore, the statistical value of the norm (such as mean or variance, etc.) is calculated, and the maximum value is determined according to the mean and variance according to the distribution to which the norm is subjected, and then the sensitivity is determined. In this way, the accuracy can be guaranteed.

[0056] Correspondingly, in the case of Figure 1 In the case of Δ(x, x') = 1, the input sample needs to include multiple samples, and only one minimum changeable unit is different between any two sample sets.

[0057] Correspondingly, the process of obtaining the sample can include: randomly generating a sample; for the randomly generated sample, changing any minimum changeable unit in the sample multiple times to obtain multiple samples. For example, for a model with text input, if b input samples are needed, 1 sentence can be randomly generated, and one character in the sentence can be randomly replaced to obtain the remaining b-1 sentences.

[0058] Correspondingly, the process of obtaining the local statistical value can include: for any two combinations of the plurality of first intermediate results, calculating the norm of each two combination; calculating the statistical value of the norm of the plurality of two combinations as the local statistical value.

[0059] The norm of the two combinations is ||M(x)-M(x')||.

[0060] Second, in the case of Figure 2 In the case shown in the figure, the first part model can be a vertically split model, which can be Figure 2 The left or right dashed box in the figure. In this case, the data owner described in the foregoing can be the first data owner or the second data owner. Correspondingly, another data owner, the second data owner or the first data owner, can also exist in this case. The two data owners each own part of the data. The parameter can be the intermediate result itself, and correspondingly, the parameter can be used to determine the extreme value of the intermediate result.

[0061] In addition, in the case where the intermediate result itself is used as the parameter, the statistical value of the parameter can include the mean of the intermediate result output by the first part model, and the statistical value of the parameter can be used to determine the mean of the intermediate result output by the first part model.

[0062] In the case that the intermediate result itself is taken as a parameter, and the statistical value of the parameter includes the mean and variance of the intermediate result output by the first part of the model, in addition to being able to determine the mean of the intermediate result, the statistical value of the parameter can also be used to determine the maximum value, minimum value and / or value range of the intermediate result output by the first part of the model. The specific determination method will be described below in step 303.

[0063] In Figure 2 In the scenario shown, because there is no restriction on the relationship between the samples, the plurality of samples can all be randomly generated. The plurality of samples can also be a plurality of samples randomly selected from an existing data set, and the present specification does not limit the method of obtaining the plurality of samples. Correspondingly, the calculation of the local statistical value can directly calculate the statistical value of the first intermediate result of the plurality of samples. The reason for calculating the statistical value of the intermediate result instead of calculating the statistical value of the maximum or minimum value of the first intermediate result in the plurality of samples is the same as the first, and will not be described here.

[0064] In addition, there can be a need to calculate the mean or value range of the intermediate result in other scenarios. In these scenarios, a similar calculation method as the second can be used, that is, the statistical value of the intermediate result is directly calculated. How to use the statistical value to obtain the extreme value, value range, etc. of the intermediate result will be described below in combination with step 303, and will not be described here.

[0065] In step S2, that is, by the operation of the first part of the model, the first intermediate result corresponding to the input sample is obtained. In step S3, that is, the Wilks iteration method is used to update the parameter value.

[0066] Taking the statistical value including the variance and the mean as an example, the specific updating method can refer to the Wilks iteration method, that is, the following formulas (1) and (2):

[0067]

[0068] Wherein, μ k is the mean value obtained by the kth cycle update, μ k-1 is the mean value obtained by the (k-1)th cycle update, x k is the first intermediate result corresponding to the parameter of the sample input in this cycle, and k is the cycle number. Correspondingly, σ k is the variance obtained by the kth cycle update, σ k-1 is the variance obtained by the (k-1)th cycle update. In addition, for the first cycle, the variance can be directly set to 0, and the mean value is set to x k .

[0069] In addition, for x kThe maximum value, the minimum value, and the sensitivity value of the data to be obtained. In the calculation of the statistical value, the maximum value cannot be obtained by one input data in one round of iteration. In order to facilitate the calculation, the maximum value can not be taken as a parameter, but x k The parameter itself. After calculating the mean and variance of the parameter, the maximum value can be determined by the mean and several times the variance according to the distribution of the parameter. The specific method of obtaining the maximum value according to the parameter will be described below, which will not be described here.

[0070] Compared with the point-by-point calculation of the mean and the variance, the above formula (1) and formula (2) update the obtained mean and variance according to one input. This calculation method is more efficient for computers, and can ensure that the stability of the data is not lost during the floating-point number calculation process, so that the calculation result is more accurate.

[0071] In the above example, the parameter statistical value is updated by one sample in one round of iteration. In another optional implementation, the Wilford iteration method can be improved, such as inputting multiple samples in one round of iteration, and updating the statistical value of the parameter by multiple samples in each round of iteration. In this way, since more data participates in the calculation in one round of iteration, the number of loop rounds is reduced to a certain extent, the calculation accuracy is ensured, and the calculation efficiency is improved. The number of samples input in one round of iteration should be less than a certain value, so that each round of iteration does not need to store a large amount of data, and the advantages of the Wilford method in reducing storage and ensuring statistical value accuracy can be achieved.

[0072] Next, the method of updating the statistical value by multiple samples in one round of iteration will be described.

[0073] The sample obtained in S1 can be multiple samples. Correspondingly, the process of obtaining the first intermediate result in S2 can be: inputting multiple samples into the first part of the model to obtain multiple first intermediate results. Then the process of updating the statistical value in S3 can be: based on the multiple first intermediate results, calculating the local statistical value of the parameter; based on the local statistical value and the loop round corresponding to the multiple samples, updating the statistical value of the parameter.

[0074] The local statistical value is the statistical value of the parameter corresponding to the first intermediate result of the multiple samples in the current round of iteration. For example, the parameter is the first intermediate result itself, and the local statistical value is the local mean of the first intermediate result. Then the local statistical value is the mean of the multiple first intermediate results.

[0075] It should be noted that in the two application scenarios introduced above, the statistical value can be updated by the following method.

[0076] In an optional embodiment, the statistical value comprises at least one of: mean value, variance.

[0077] For the specific method of updating the statistical value, firstly, for the mean value, the local mean value can be taken as x k in formula (1) to update the mean value in a similar way as formula (1).

[0078] Secondly, for the variance, since multiple samples are input in each round of loop, in each round of loop, the variance will be affected by the dispersion degree of the first intermediate results of the samples in the round of loop in addition to the dispersion degree of the local mean value of the round of loop compared with the mean value updated in the last round of loop as shown in formula (2). Therefore, in the variance updating, the dispersion degree of the data in the round of loop needs to be further introduced in formula (2).

[0079] Specifically, the method of updating the variance can comprise: determining a first variance based on the difference between the local mean value and the mean value of the parameter and the round of loop corresponding to the multiple samples; the first variance is used to represent the dispersion between the local mean value and the mean value of the parameter updated in the last round of loop; updating the variance of the parameter according to the first variance, the local variance and the round of loop; updating the variance of the parameter according to the local mean value and the round of loop.

[0080] wherein the role of the first variance is similar to the role of in formula (2), both of which are used to describe the dispersion degree between the parameter obtained in the round of loop and the parameter updated in the past loop. The local variance is the variance between the data obtained in the round of loop, and the dispersion degree between the first intermediate results of the samples in the round of loop can be described by the local variance.

[0081] wherein in an optional embodiment, the variance and the mean value can be updated by the following formulas (3) and (4):

[0082]

[0083] wherein the same symbols in formula (3) and formula (4) have the same meanings as in formula (1) and (2). new is the local mean value, and new is the local variance. is used to represent the first variance.

[0084] Step 303: converting the second intermediate result output by the first part of the model with respect to the data based on the statistical value of the parameter.

[0085] After the statistical value of the parameter is obtained, the intermediate result can be converted by the statistical value through step 303. Specifically, the estimated value of the parameter can be determined by the statistical value of the parameter, and the second intermediate result is further converted according to the estimated value of the parameter to meet the needs of different scenarios.

[0086] It should be further pointed out that steps 301 and 303 can be executed continuously or discontinuously.

[0087] Next, the specific implementation method of step 303 in different scenarios will be described. Still taking the two examples mentioned in the foregoing as an example.

[0088] First, in the scenario shown in FIG. 3, the sensitivity can be estimated according to the statistical value of the parameter first. Figure 1

[0089] For the sensitivity, the sensitivity of the intermediate result output by the first part model is determined according to the distribution to which the parameter conforms and the statistical value of the parameter. For example, assuming that the parameter conforms to a normal distribution, the size of the sensitivity can be determined according to the sum of the determined mean and variance of the parameter.

[0090] For example, the statistical value μ+3σ corresponding to the parameter can be taken as the sensitivity, μ is the mean, and σ is the variance. According to the normal distribution, it can be determined that the failure probability of the sensitivity μ+3σ is 2.7×10 -3 In addition, the sensitivity can also be determined according to the requirement for the failure probability of the sensitivity. For example, if the sensitivity requirement is lower, the sensitivity can be determined as μ+2σ, and the failure probability of the sensitivity is 4.66×10 -2 .

[0091] In addition, the sensitivity includes L1 sensitivity and L2 sensitivity, and the two sensitivities are calculated by setting the parameter as the L1 norm and L2 norm of ||M(x)-M(x')|| respectively. When calculating the sensitivity, only the statistical value of the parameter corresponding to the L1 sensitivity or the statistical value of the parameter corresponding to the L2 sensitivity can be calculated.

[0092] Alternatively, the L1 norm and L2 norm of ||M(x)-M(x')|| can also be taken as parameters, the statistical values of the two parameters are updated in each loop, and the L1 sensitivity and L2 sensitivity are determined according to the statistical values of the two parameters. And the minimum value among them is taken as the final sensitivity value. In other words, the sensitivity is the minimum value of the L1 sensitivity and the L2 sensitivity of the intermediate result output by the first part model. Such sensitivity can be more appropriate.

[0093] ​After calculating the sensitivity, the intermediate results can be perturbed based on the sensitivity to prevent the computing party from launching a reconstruction attack based on the intermediate results, thus protecting the security of the data owner's data.

[0094] In other words, the method also involves a computation party, the owner of which includes at least the second part of the model; step 303 includes: perturbing the second intermediate result using a differential privacy method based on the sensitivity. After obtaining the perturbed second intermediate result, the perturbed second intermediate result can also be sent to the computation party, so that the computation party can obtain the output of the model using the second part of the model based on the perturbed second intermediate result.

[0095] The second part of the model refers to the portion of the model other than the first part. The input to the second part can be intermediate results, and the output is the combined output of the first and second parts. After determining the sensitivity, the intermediate results of the first part's output can be perturbed. This perturbed intermediate result is then sent to the computation unit, allowing it to input the intermediate results into the second part to obtain the model's output.

[0096] The difference between the first and second intermediate results lies in the input data, although both are obtained in the same way. The first intermediate result is obtained by inputting the samples obtained in each iteration into the first part of the model, while the second intermediate result is obtained by inputting the data owned by the data owner into the first part of the model.

[0097] Second, in Figure 2 In the scenario shown, similar to the sensitivity determination method, the maximum and / or minimum values ​​of the intermediate results output by the first part of the model can be determined based on the distribution that the parameters follow and the statistical values ​​of the parameters.

[0098] In addition, in other scenarios where there is a need to determine the mean and / or range of intermediate results output by the first part of the model, it can also be determined based on the distribution that the parameters follow and the statistical values ​​of the parameters.

[0099] For example, assuming the parameters follow a normal distribution, the mean μ of the parameter statistics can be used as the mean of the intermediate results output by the first part of the model. The sum of the parameter mean μ and a certain multiple of the variance σ can be used as the maximum value of the intermediate results. For example, μ + 3σ can be used as the maximum value of the intermediate results output by the first part of the model, and the failure probability of this maximum value is 2.7 × 10⁻⁶. -3The difference between the mean of the parameters and several times the variance of the parameters can be taken as the minimum of the above intermediate result. For example, μ-3σ can be taken as the minimum of the intermediate result of the first part model output, and the failure probability of the minimum is also 2.7x10 -3 For the value range of the intermediate result of the first part model output, the value range can also be determined according to the determined minimum value and maximum value.

[0100] In the vertical splitting scenario, after the maximum value and / or the minimum value of the intermediate result are determined, step 303 can include: converting a nonlinear operation in multi-party secure computation (MPC) into a linear operation based on the maximum value or the minimum value; and performing an operation on the second intermediate result based on the linear operation.

[0101] As described above, in the vertical splitting scenario, because the nonlinear operation needs to consume more computing resources, it is necessary to convert the nonlinear operation into a linear operation composed of addition, subtraction, multiplication, and division and the like, which has a similar operation result, to complete the required calculation in the MPC scenario. For different nonlinear operations, the linear operation with a similar operation result is different under different value ranges. Therefore, by determining the maximum value and / or the minimum value, it is more convenient to determine what kind of linear operation is converted.

[0102] The above two scenarios are examples and do not limit the present application. It is easy to understand that in other scenarios, if a parameter of a certain intermediate result needs to be obtained, the above method can also be used to obtain the parameter.

[0103] Next, the method shown in the specification will be described by determining the L2 sensitivity in differential privacy as a specific example. In this scenario, the data that needs to be protected by differential privacy is the intermediate result obtained by inputting data into the first part model in the horizontal splitting scenario. The model in this scenario can be a model for processing text data.

[0104] As shown in FIG. 4, the scenario includes the following steps. Figure 4

[0105] Step 401, determine a given model M that needs to calculate a statistical value and a corresponding statistical value G. In the sensitivity scenario, G=max x,x′；Δ(x,x′)＝1 ||M(x)-M(x′)||. M is the first part model.

[0106] Step 402, generate b model input samples {x1,x2,…,x b}. Specifically, a sentence is first randomly generated, and the remaining b-1 sentences are obtained by randomly replacing each word in the sentence.

[0107] ​Step 403, calculate the local statistical value of the batch of sentence samples. Pairwise combine the b sentences, calculate the L2 norm of the difference between the intermediate results of the two sentences in each combination in the corresponding model. And calculate the average value μ of all L2 norms of the differences new and the variance σ new .

[0108] Step 404, judge whether it is the first iteration:

[0109] Step 404a. If it is the first iteration, take the local average value and the local variance of the L2 norm of the current batch of data as the overall average value and the overall variance. And jump to step 402.

[0110] Step 404b. If it is not the first iteration, update the average value and the variance by the recursive formula. Refer to formula (3) and formula (4) for the average value and variance update formula.

[0111] Step 405, judge whether it converges. Judge whether the average value and the variance of the current round are close to 0 compared with the average value and the variance of the last round. If they are close to 0, it means that it has converged, and jump to step 407. If it does not converge, continue to execute step 406

[0112] For example, judge whether the difference between the average value of the current round and the average value of the last round is less than a first preset threshold value, and judge whether the difference between the variance of the current round and the variance of the last round is less than a second preset threshold value. The first preset threshold value and the second preset threshold value can be the same or different. When both differences are less than the threshold value, it is confirmed that it has converged.

[0113] Step 406, judge whether the upper limit of the number of iterations has been reached. If the upper limit of the number of iterations set in advance has not been reached, jump back to step 402 to continue execution. If the upper limit of the number of iterations set in advance has been reached, continue to execute step 407.

[0114] Step 407, output μ and σ.

[0115] After step 407, the sensitivity G can be calculated according to the method mentioned in step 403 above, and the intermediate results can be processed. The specific processing method is not repeated here.

[0116] Through the above method, in the case that manpower cannot accurately calculate statistical values such as large models, a method of experimental science is proposed as a supplement, which can give a more accurate statistical value estimate and give a confidence level. Moreover, the improved batch Welford iteration method calculates the statistical quantity of the cumulative sample without storing all the samples, and supports batch processing to improve the calculation speed. In the case of limited computing power, better results can also be achieved. An adaptive model statistical value calculation method is proposed: for the model, the statistical quantity is infinitely approximated; for L1 and L2 sensitivity, the norm with less influence is freely selected. The whole method is flexible and lightweight, and is suitable for various scenes.

[0117] In the 1990s, it was quite obvious to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structures of diodes, transistors, switches, etc.) or in software (improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) such as a field programmable gate array (FPGA) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by ordering a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating integrated circuit chips, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code before compilation must also be written in a specific programming language, which is called a hardware description language (HDL), and there are many such languages, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should be aware that, as long as the method flow is logically programmed in the above-mentioned hardware description languages and programmed into an integrated circuit, a hardware circuit that implements the logical method flow can be easily obtained.

[0118] The specification also provides a model intermediate result processing apparatus applied to a data owner, the data owner comprising at least a first partial model of a model, the data owner owning data comprising private content. A block diagram of the apparatus is shown in Figure 5 The apparatus is configured to perform the steps of the method described above. The apparatus comprises: Figure 3 or Figure 4 The apparatus comprises:

[0119] A loop unit 510 is configured to looply perform S1-S3 until a statistical value of a parameter corresponding to an intermediate result of the first partial model converges. S1: obtain a sample; S2: input the sample into the first partial model to obtain a first intermediate result; and S3: update the statistical value of the parameter according to the first intermediate result.

[0120] A conversion unit 520 is configured to convert a second intermediate result output by the first partial model with respect to the data based on the statistical value of the parameter.

[0121] In an optional embodiment, the loop unit 510 is specifically configured to obtain a sample; input a plurality of samples into the first partial model to obtain a plurality of first intermediate results; calculate a local statistical value of the parameter based on the plurality of first intermediate results; and update the statistical value of the parameter based on the local statistical value and a loop round corresponding to the plurality of samples.

[0122] In an optional embodiment, the statistical value comprises at least one of a mean value and a variance.

[0123] In an optional embodiment, the loop unit 510 is specifically configured to obtain a sample; input a plurality of samples into the first partial model to obtain a plurality of first intermediate results; calculate a local statistical value of the parameter based on the plurality of first intermediate results; determine a first variance based on a difference between a local mean value and a mean value of the parameter and a loop round corresponding to the plurality of samples, the first variance being used to represent a dispersion between the local mean value and the mean value of the parameter updated in a previous loop round; update a variance of the parameter according to the first variance, the local variance and the loop round; and update the variance of the parameter according to the local mean value and the loop round.

[0124] In an optional implementation, the parameter comprises a norm of a difference between intermediate results respectively output by the first partial model for two groups of data, and the statistical value of the parameter is used to determine a sensitivity of the intermediate result output by the first partial model; the loop unit 510 is specifically configured to randomly generate a sample; for the randomly generated sample, any minimum changeable unit in the sample is changed multiple times to obtain multiple samples; for any two groups of the multiple first intermediate results, a norm of each two-group combination is calculated; a statistical value of the multiple two-group combination norms is calculated as a local statistical value. Based on the local statistical value and a loop round corresponding to the multiple samples, the statistical value of the parameter is updated.

[0125] In an optional implementation, the sensitivity is a minimum value of L1 sensitivity and L2 sensitivity of the intermediate result output by the first partial model.

[0126] In an optional implementation, the method further involves a computing party, and the computing party at least comprises a second partial model of the model; the conversion unit 520 is specifically configured to disturb the second intermediate result based on the sensitivity by using a differential privacy method. The apparatus further comprises a sending unit 330 (not shown in the figure) configured to send the disturbed second intermediate result to the computing party, so that the computing party obtains the output of the model by using the second partial model based on the disturbed second intermediate result.

[0127] In an optional implementation, the statistical value of the parameter comprises a mean value and a variance of the intermediate result output by the first partial model, and the statistical value of the parameter is used to determine a mean value, a maximum value, a minimum value and / or a value range of the intermediate result output by the first partial model.

[0128] In an optional implementation, the sensitivity, the mean value, the maximum value, the minimum value and / or the value range of the intermediate result output by the first partial model are determined according to a distribution to which the parameter conforms and the statistical value of the parameter.

[0129] In an optional implementation, the statistical value of the parameter is used to determine a maximum value or a minimum value of the intermediate result output by the first partial model; the conversion unit 520 is specifically configured to convert a nonlinear operation in multi-party secure computation (MPC) into a linear operation based on the maximum value or the minimum value; and the second intermediate result is operated based on the linear operation.

[0130] For ease of description, the above apparatus is described as various units in function to describe separately. Of course, when implementing one or more of the present specification, the functions of each unit can be implemented in one or more software and / or hardware, and units implementing the same function can be implemented by a combination of multiple sub-units or sub-modules, etc. The above-described apparatus embodiments are only illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0131] The present specification also provides a computer program product, including computer programs / instructions, which, when executed by a processor, implement the steps of the above-described model intermediate result processing method.

[0132] The present specification also provides a computing device, including a memory and a processor, the memory stores executable codes, and the processor executes the executable codes to implement the above-described model intermediate result processing method.

[0133] In a typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memories.

[0134] The memory can include non-permanent memory in a computer readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer readable medium.

[0135] The present specification also provides a computer readable storage medium, which stores a computer program, and when the computer program is executed in a computer, the computer executes the above-described model intermediate result processing method.

[0136] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage, graphene storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0137] In addition, the method provided by the present specification can be implemented by a controller. The controller can be implemented in any appropriate manner, for example, the controller can take the form of, for example, a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g. software or firmware) executable by the (micro)processor, logic gates, switches, application specific integrated circuits (ASIC), programmable logic controllers and embedded microcontrollers, examples of the controller include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, the same function can be achieved by logically programming the method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, such a controller can be considered as a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, even the means for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0138] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0139] While one or more embodiments of this specification provide the steps of the method described in the flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual devices or end products, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0140] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0141] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a processFigure 1 one or more processes and / or functions described in the one or more blocks. Figure 1 one or more blocks.

[0142] These computer program instructions can also be loaded into computer or other programmable data processing devices to cause a series of operational steps to be performed on the computer or other programmable devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable devices provide steps for implementing the functions described in the flowcharts Figure 1 one or more processes and / or functions described in the one or more blocks. Figure 1 one or more blocks.

[0143] Those skilled in the art will appreciate that one or more embodiments of the present description can be provided as a method, a system or a computer program product. Accordingly, one or more embodiments of the present description can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0144] One or more embodiments of the present description can be described in the general context of computer-executable instructions, such as program modules, being executed by computers. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. One or more embodiments of the present description can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0145] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, the system embodiments are described simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments. In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the specification. The illustrative description of the above terms in the specification does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0146] The above only describes the embodiments of one or more embodiments of the specification and does not limit one or more embodiments of the specification. Those skilled in the art can make various changes and changes to one or more embodiments of the specification. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the specification shall be included in the scope of claims.

Claims

1. A method for processing intermediate results of a model, the method being performed by a data owner, the data owner comprising at least a first part of the model, the data owner possessing data including privacy-related content, the method comprising: Repeat steps S1 through S3 until the statistical values ​​of the parameters corresponding to the intermediate results of the first part of the model converge: S1 obtains the sample; S2 inputs the sample into the first part of the model to obtain the first intermediate result; S3 updates the statistical value of the parameter based on the first intermediate result; Based on the statistical values ​​of the parameters, the second intermediate result of the first part of the model is transformed relative to the data output.

2. The method according to claim 1, wherein inputting the sample into the first partial model to obtain the first intermediate result includes: Multiple samples are input into the first part of the model to obtain multiple first intermediate results; The step of updating the statistical value of the parameter based on the first intermediate result includes: Based on the multiple first intermediate results, local statistical values ​​of the parameters are calculated; The statistical values ​​of the parameters are updated based on the local statistical values ​​and the cycle number corresponding to the multiple samples.

3. The method according to claim 2, wherein, The statistical values ​​include at least one of the following: mean and variance.

4. The method according to claim 3, wherein updating the statistical value of the parameter based on the local statistical value and the cycle number corresponding to the plurality of samples includes: The first variance is determined based on the difference between the local mean and the mean of the parameters, and the number of cycles corresponding to the multiple samples. The first variance is used to characterize the dispersion between the local mean and the mean of the parameters obtained from the previous iteration; the local mean is the mean of the parameters corresponding to the first intermediate results of multiple samples obtained in this iteration; The variance of the parameters is updated based on the first variance, the local variance, and the iteration number; the local variance is the variance between the parameters corresponding to the first intermediate results of multiple samples obtained in this iteration. The variance of the parameters is updated based on the local mean and the number of iterations.

5. The method according to claim 2, wherein, The parameters include the norm of the difference between the intermediate results output by the first part of the model for the two sets of data, and the statistical value of the parameters is used to determine the sensitivity of the intermediate results output by the first part of the model. The acquisition of samples includes: Randomly generate a sample; For a randomly generated sample, multiple samples are obtained by repeatedly changing any smallest modifiable unit in that sample. The step of calculating the local statistical value of the parameter based on the plurality of first intermediate results includes: For any pairwise combination of the plurality of first intermediate results, calculate the norm of each pairwise combination; Calculate the norm statistics of multiple pairwise combinations as local statistics.

6. The method according to claim 5, wherein, The sensitivity is the minimum of the L1 sensitivity and L2 sensitivity of the intermediate results output by the first part of the model.

7. The method according to claim 5, wherein, The method also involves a computational approach, which includes at least a second part of the model; The transformation of the second intermediate result of the first part of the model relative to the data output based on the statistical values ​​of the parameters includes: Based on the aforementioned sensitivity, the second intermediate result is perturbed using a differential privacy method; The method further includes: The perturbated second intermediate result is sent to the computing party so that the computing party can obtain the output of the model using the second part of the model based on the perturbated second intermediate result.

8. The method according to claim 1, wherein, The statistical values ​​of the parameters include the mean and variance of the intermediate results output by the first part of the model. The statistical values ​​of the parameters are used to determine the mean, maximum value, minimum value and / or range of the intermediate results output by the first part of the model.

9. The method according to claim 5 or 8, wherein, The sensitivity, mean, maximum, minimum and / or range of the intermediate results output by the first part of the model are determined based on the distribution that the parameters follow and the statistical values ​​of the parameters.

10. The method according to claim 8, wherein, The statistical values ​​of the parameters are used to determine the maximum or minimum value of the intermediate results output by the first part of the model; The transformation of the second intermediate result of the first part of the model relative to the data output based on the statistical values ​​of the parameters includes: Based on the maximum or minimum value, the nonlinear operation in Multi-Party Secure Computation (MPC) is converted into a linear operation; The second intermediate result is then processed based on the linear operation described above.

11. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-10.

Images