Data processing method and data processing apparatus

Abstract

The application discloses a data processing method and a data processing device, and belongs to the field of vertical federated learning. The data processing method comprises the following steps: obtaining target industrial feature data; performing encryption processing on the target industrial feature data to obtain an initial intermediate result, and sending the initial intermediate result to a privacy computing system for encryption processing to obtain a target intermediate result output by the privacy computing system; and obtaining a target label corresponding to the target intermediate result based on the target intermediate result; wherein the privacy computing system comprises a first server, a second server participating in additive sharing and an auxiliary computing server used for auxiliary computing. The data processing method can significantly reduce the risk of obtaining the target industrial feature data based on the intermediate result in the vertical federated learning process, thereby reducing the risk of data privacy leakage of the feature provider caused by leakage of the intermediate result, and effectively improving the data security in the vertical federated learning process.

Description

Technical Field

[0001] This application belongs to the field of vertical federated learning, and in particular relates to a data processing method and a data processing apparatus. Background Technology

[0002] With the advent of the big data era, the Industrial Internet involves an increasing amount of data, making effective privacy protection a pressing issue. One relevant technology employs a split-learning approach for federated learning, using industrial feature data to obtain corresponding labels, thereby protecting the privacy of local industrial feature data. However, this split-learning method still carries a significant risk of local data leakage. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a data processing method and a data processing apparatus to effectively improve data security during the longitudinal federated learning process.

[0004] Firstly, this application provides a data processing method, which includes:

[0005] Acquire target industrial characteristic data;

[0006] The target industrial feature data is encrypted to obtain an initial intermediate result, and the initial intermediate result is sent to a privacy computing system for encryption to obtain the target intermediate result output by the privacy computing system.

[0007] Based on the target intermediate result, obtain the target label corresponding to the target intermediate result;

[0008] The privacy computing system includes a first server and a second server participating in addition sharing, as well as an auxiliary computing server for assisting computing.

[0009] According to the data processing method of this application, in the process of vertical federated learning, the feature provider performs encryption processing on the initial intermediate results based on addition sharing to obtain the target intermediate results, and sends the target intermediate results to the label provider to predict the target label based on the target intermediate results. The intermediate results can be efficiently encrypted without multiple encryption and decryption, and the encryption effect is good. It can significantly reduce the risk of reverse inferring the target industrial feature data based on the intermediate results, thereby reducing the risk of data privacy leakage of the feature provider due to leakage of intermediate results, and effectively improving the data security in the process of vertical federated learning.

[0010] According to one embodiment of this application, the privacy computing system includes a sequentially connected basic operator layer, a differentiable operator layer, and a model layer. The step of sending the initial intermediate result to the privacy computing system for encryption processing and obtaining the target intermediate result output by the privacy computing system includes:

[0011] The initial intermediate result is input into the basic operator layer to obtain the encrypted data output by the basic operator layer;

[0012] The encrypted data is input into the differentiable operator layer for operator format packaging, and the differential data output by the differentiable operator layer is obtained.

[0013] The differential data is input into the model layer to obtain the target intermediate result output by the model layer.

[0014] According to one embodiment of this application, the step of inputting the initial intermediate result into the basic operator layer and obtaining the encrypted data output by the basic operator layer includes:

[0015] The initial intermediate results are subjected to linear encryption and nonlinear element-by-element function calculations to obtain the encrypted data.

[0016] According to one embodiment of this application, the step of inputting the encrypted data into a differentiable operator layer for operator format packaging and obtaining the differential data output by the differentiable operator layer includes:

[0017] Determine the dependencies between multiple encrypted data;

[0018] Based on the aforementioned dependency relationship, the operation functions are sequentially called to process the multiple encrypted data and obtain the differential data.

[0019] According to one embodiment of this application, the step of encrypting the target industrial feature data to obtain an initial intermediate result, and sending the initial intermediate result to a privacy computing system for encryption processing to obtain a target intermediate result output by the privacy computing system; and obtaining a target tag corresponding to the target intermediate result based on the target intermediate result, includes:

[0020] The target industrial feature data is input into the target first sub-model among multiple first sub-models, and the target intermediate result output by the target first sub-model is obtained;

[0021] The target intermediate result is input into the second sub-model to obtain the target label output by the second sub-model;

[0022] The first target sub-model is trained based on sample industrial feature data corresponding to the first target sub-model and sample intermediate results corresponding to the sample industrial feature data. The second sub-model is trained based on the sample intermediate results and sample labels corresponding to the sample industrial feature data. The sample intermediate results are data that has been encrypted by the privacy computing system.

[0023] According to one embodiment of this application, the intermediate sample results are determined in the following manner:

[0024] The sample industrial feature data is encrypted to obtain a first intermediate result;

[0025] The first intermediate result is sent to the privacy computing system for encryption processing, and the sample intermediate result output by the privacy computing system is obtained.

[0026] Secondly, this application provides a data processing apparatus, which includes:

[0027] According to the data processing apparatus of this application, during the vertical federated learning process, the feature provider performs encryption processing on the initial intermediate results based on addition sharing to obtain the target intermediate results, and sends the target intermediate results to the label provider to predict the target label based on the target intermediate results. The intermediate results can be efficiently encrypted without multiple encryption and decryption, and the encryption effect is good. It can significantly reduce the risk of reverse inferring the target industrial feature data based on the intermediate results, thereby reducing the risk of data privacy leakage of the feature provider due to leakage of intermediate results, and effectively improving the data security in the vertical federated learning process.

[0028] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data processing method described in the first aspect above.

[0029] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method as described in the first aspect above.

[0030] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the data processing method as described in the first aspect.

[0031] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the data processing method as described in the first aspect above.

[0032] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects:

[0033] By having the feature provider perform addition-based encryption on the initial intermediate results during the vertical federated learning process to obtain the target intermediate results, and then sending the target intermediate results to the label provider to predict the target labels based on the target intermediate results, the intermediate results can be efficiently encrypted without multiple encryption and decryption. The encryption effect is good and can significantly reduce the risk of reverse-engineering the target industrial feature data based on the intermediate results. This reduces the risk of data privacy leakage of the feature provider due to the leakage of intermediate results, and effectively improves the data security in the vertical federated learning process.

[0034] Furthermore, encrypted data is obtained by encrypting the initial intermediate results through a basic operator layer, and then the encrypted data is packaged into differentiable operators through a differentiable operator layer. Finally, the differentiable operators are processed through a model layer to package them into operators suitable for the model, so that the final encrypted target intermediate results can be directly used in the longitudinal federated learning model. It can provide a high-level automatic differentiation model interface for convenient privacy modeling based on the underlying basic operators, and can provide a complete, developable, extensible and automatically differentiable high-level machine learning modeling interface, with high universality and ease of use.

[0035] Furthermore, by performing linear encryption on the initial intermediate results and nonlinear element-wise function calculations to obtain encrypted data, the initial intermediate results can be encrypted from multiple dimensions with good encryption effect, significantly improving the privacy of intermediate results. This enhances the data security and privacy of feature providers in the longitudinal federated learning process, without requiring multiple encryption and decryption operations, making the operation simple and convenient.

[0036] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0037] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0038] Figure 1 This is one of the flowcharts illustrating the data processing method provided in the embodiments of this application;

[0039] Figure 2 This is a second schematic flowchart of the data processing method provided in the embodiments of this application;

[0040] Figure 3This is the third flowchart illustrating the data processing method provided in the embodiments of this application;

[0041] Figure 4 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application;

[0042] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0043] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0044] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0045] The data processing method, data processing device, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0046] The data processing method can be applied to the terminal, and can be executed by the hardware or software in the terminal.

[0047] The data processing method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the data processing method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The data processing method provided in this application embodiment is described below using an electronic device as the execution subject.

[0048] It should be noted that this data processing method can be applied to the field of industrial big data processing.

[0049] like Figure 1 As shown, the data processing method includes steps 110, 120 and 130.

[0050] Step 110: Obtain target industrial characteristic data;

[0051] In this step, the target industrial characteristic data can be user characteristic data, product characteristic data, production characteristic data, or other arbitrary characteristic data, such as the construction year of the user's residential community and the average house price in the community. This application does not limit this.

[0052] Taking target industrial characteristic data such as the construction year of the user's residential community and the average house price in the community as an example, this characteristic data can be used to improve the probability of predicting whether the user will purchase a certain type of product in the next month.

[0053] Understandably, in the process of vertical federated learning, multiple feature providers are involved, each of which has corresponding industrial feature data. The industrial feature data provided by each feature provider is not completely the same and varies greatly.

[0054] It should be noted that the feature providers do not share information with each other.

[0055] Continuing with the example of target industrial characteristic data such as the construction year of the user's residential community and the average house price in the community, the relevant characteristic data such as the construction year of the user's residential community may be provided by company A, and the characteristic data such as the average house price in the community may be provided by company B, and there is no information sharing between company A and company B.

[0056] In this step, the target feature provider can be any one of multiple feature providers, and the target industrial feature data is the industrial feature data provided by the target feature provider.

[0057] Step 120: Encrypt the target industrial feature data to obtain the initial intermediate result, and send the initial intermediate result to the privacy computing system for encryption processing to obtain the target intermediate result output by the privacy computing system;

[0058] In this step, such as Figure 2 As shown, the privacy computing system includes a first server P0, a second server P1, and an auxiliary computing server P2 for assisting in the computing process.

[0059] The target intermediate result is the feature data provided by the feature provider to the label provider.

[0060] The initial intermediate result is the data obtained by the feature provider after encrypting and aligning its corresponding industrial feature data. This initial intermediate result can maintain data alignment with the initial intermediate results provided by other feature providers, thereby facilitating the label provider to perform federated training based on the initial intermediate results provided by multiple feature providers to generate labels corresponding to each industrial feature data.

[0061] The target intermediate result is a new intermediate result generated after performing addition-based sharing encryption on the initial intermediate result.

[0062] In this application, the initial intermediate results are encrypted using an addition-based sharing encryption process to perform secondary encryption on the initial intermediate results. This prevents the reverse engineering of the target industrial feature data based on the initial intermediate results, thus avoiding privacy leaks of the target industrial feature data and improving data security.

[0063] During the research and development process, the inventors discovered that there are methods in related technologies that use cryptographic methods or secure multi-party computation protocols to perform secure computation layer by layer to encrypt the splitting model. However, these methods require multiple encryption and decryption operations, and the method of sharing secrets to calculate the activation function also incurs significant overhead. This makes it unsuitable for complex nonlinear activation functions such as Sigmoid or Tanh, limiting its application scope and increasing its cost.

[0064] In this application, a third-party auxiliary computing server is introduced to assist the first and second servers in performing privacy-preserving computations based on addition sharing. This eliminates the need for multiple encryption and decryption operations, and the resulting intermediate target is not easily cracked, thus providing high privacy and security.

[0065] In some embodiments, the privacy computing system may include a sequentially connected layer of basic operators, a layer of differentiable operators, and a model layer. Sending initial intermediate results to the privacy computing system for encryption processing and obtaining the target intermediate results output by the privacy computing system may include:

[0066] The initial intermediate results are input into the basic operator layer to obtain the encrypted data output by the basic operator layer;

[0067] Encrypted data is input into the differentiable operator layer for operator format packaging, and the differential data output by the differentiable operator layer is obtained.

[0068] The differential data is input into the model layer to obtain the target intermediate result output by the model layer.

[0069] In this embodiment, the basic operator layer includes a first server, a second server, and an auxiliary computing server for assisting computation.

[0070] The basic operator layer provides basic operators.

[0071] Basic operators include fundamental operators needed for machine learning, such as addition, subtraction, multiplication, and division; as well as some local tensor computations, such as shape transformation and shape broadcasting.

[0072] The differentiable operator layer is used to perform differentiable processing on encrypted data, so as to wrap the basic operators into differentiable operators, thereby realizing the automatic differentiation mechanism required for backpropagation.

[0073] The data structure of the operators in the differentiable operator layer is represented as: [Input node list (InNodes), Output node (OutNode), Operation function (f)].

[0074] During normal forward propagation, the differentiable operator obtains the value of the output node through the operation function, that is:

[0075] OutNode.value=f(InNodes[0].value,InNodes[1].value,…).

[0076] When defining a differentiable operator, you need to specify the gradient function for that operator. The return value of the gradient function is also a list of operators, whose input node list contains the original operator's input node list, as well as the gradient returned from the output.

[0077] It should be noted that in some embodiments, there are multiple differentiable operators, and each differentiable operator can be arbitrarily combined to form a combined differentiable operator.

[0078] In some embodiments, inputting encrypted data into a differentiable operator layer for operator format packaging and obtaining the differential data output by the differentiable operator layer may include:

[0079] Determine the dependencies between multiple encrypted data;

[0080] Based on dependencies, the operation functions are called sequentially to process multiple encrypted data and obtain differential data.

[0081] In this embodiment, the operator has a combinatorial function.

[0082] For example, for three inputs: x1, x2, x3, we can first use the addition operator to get h1 = x1 + x2, and then use the multiplication operator to get y = h1 × x3 = (x1 + x2) × x3.

[0083] The computation of combined operators is the same as that of ordinary operators; the result of the output node is obtained by calling the operation function.

[0084] For combination operators, the operation functions need to be called sequentially according to their dependencies.

[0085] The gradient of a combinatorial operator can be generated using the chain rule.

[0086] Model layers are used to wrap differentiable operators into commonly used neural network layers, such as linear layers, activation function layers, and loss function layers, to facilitate model building.

[0087] It should be noted that the model itself is also an operator, and its input nodes include the model's input and the model's parameters.

[0088] In the case of backpropagation, the gradient on the input is not calculated, only the gradient on the parameters is calculated, and then optimizations such as gradient descent are performed according to the gradient.

[0089] According to the data processing method provided in the embodiments of this application, encrypted data is obtained by encrypting the initial intermediate results through a basic operator layer, and the encrypted data is packaged into differentiable operators through a differentiable operator layer. Then, the differentiable operators are processed through a model layer to package them into operators suitable for the model, so that the final encrypted target intermediate results can be directly used in the longitudinal federated learning model. It can provide a high-level automatic differentiation model interface for convenient privacy modeling based on the underlying basic operators, and can provide a complete, developable, extensible and automatically differentiable high-level machine learning modeling interface, which has high universality and ease of use.

[0090] In some embodiments, inputting the initial intermediate result into the basic operator layer and obtaining the encrypted data output by the basic operator layer may include: performing linear encryption processing on the initial intermediate result and nonlinear element-wise function calculation to obtain the encrypted data.

[0091] In this embodiment, the basic operator layer may include a linear computation layer and a nonlinear element-wise computation layer.

[0092] The basic operator layer includes the fundamental operators required for machine learning.

[0093] The linear computation layer includes operators for basic operations such as addition, subtraction, and multiplication. This layer is used for linear encryption processing, which includes operations based on addition, subtraction, and multiplication, such as... Figure 2 As shown.

[0094] The nonlinear element-wise computation layer is used to perform nonlinear element-wise function computations, where the computation flow for nonlinear element-wise function computation is as follows: Figure 3 As shown.

[0095] Nonlinear element-wise computation is a key aspect of the numerous nonlinear computations in neural networks. Understandably, most nonlinear computations are element-wise, meaning each element in a tensor is computed individually, such as activation functions like ReLU, Sigmoid, and Tanh.

[0096] It should be noted that, in this application, the inputs to the functions that are input to the nonlinear element-wise computation layer are all values ​​that have already been shared by addition between P0 and P1.

[0097] In some embodiments, the basic operator layer may also include a data type conversion layer.

[0098] The data type conversion layer is used to convert between floating-point numbers and integers.

[0099] It should be noted that when performing calculations at the basic operator layer, integer data must be used to ensure safety (i.e., the elements shared by addition must be Z2L elements over the integer finite field); however, the data used in practical applications (such as machine learning) is floating-point data.

[0100] The following explains how to convert between floating-point numbers and integers.

[0101] For example, let p be the number of decimal places, then the fixed-point number corresponding to the floating-point number xfloat can be:

[0102]

[0103] Where xfixed is the integer obtained by converting the floating-point number xfloat; p is the number of decimal places of the floating-point number xfloat; x is the floating-point number xfloat; and L is the maximum number of digits of the fixed-point number xfixed.

[0104] In some embodiments, L can be set to 64 to conform to the property that most programs can support 64-bit integer operations.

[0105] The above conversion method can represent floating-point numbers in the range [-2L-1-p, 2L-1-p).

[0106] In some embodiments, a fixed-point number xfixed can be converted to a floating-point number using the following formula:

[0107]

[0108] Where xfloat is the floating-point number converted from the fixed-point number xfixed; p is the number of decimal places of the floating-point number xfloat; L is the maximum number of digits of the fixed-point number xfixed; and x is the fixed-point number xfixed.

[0109] According to the data processing method provided in the embodiments of this application, encrypted data is obtained by linearly encrypting the initial intermediate results and performing nonlinear element-wise function calculations. This method can perform privacy encryption on the initial intermediate results from multiple dimensions, achieving good encryption results and significantly improving the privacy of the intermediate results. This enhances the data security and privacy of the feature provider in the longitudinal federated learning process, and eliminates the need for multiple encryption and decryption operations, making the operation simple and convenient.

[0110] The specific implementation process of linear and nonlinear calculations is explained below.

[0111] In some embodiments, linear encryption processing of the initial intermediate results and nonlinear element-wise function calculation may include:

[0112] If the data type of the initial intermediate result is inconsistent with the target data type, the initial intermediate result is input to the data type conversion layer to obtain the first data of the target data type output by the data type conversion layer.

[0113] The first data is input into the linear computation layer, and the first encrypted data output by the linear computation layer is obtained.

[0114] The first encrypted data is input into the nonlinear element-by-element computation layer to obtain the target encrypted data output by the nonlinear element-by-element computation layer.

[0115] In this embodiment, the target data type is the data type required for addition sharing, that is, an integer type.

[0116] If the initial intermediate result is a floating-point number, it is input to the data type conversion layer, which converts it to an integer to obtain the first data. Then, the first data is input to the linear calculation layer, which performs addition to generate the first encrypted data. Finally, the first encrypted data is input to the nonlinear element-wise calculation layer for function calculation to generate the target first encrypted data.

[0117] If the initial intermediate result is an integer, the initial intermediate result can be directly determined as the first data and input into the linear computation layer. The linear computation layer will then perform addition and sharing to generate the first encrypted data. Finally, the first encrypted data will be input into the nonlinear element-wise computation layer to perform function calculation and generate the target first encrypted data.

[0118] In some embodiments, inputting first data into a linear computation layer and obtaining the first encrypted data output by the linear computation layer may include:

[0119] Split the first data to obtain at least two subsequences;

[0120] The first encrypted data is determined based on at least two subsequences.

[0121] In this embodiment, at least two subsequences are obtained by splitting the first data based on addition or subtraction.

[0122] The following is a detailed explanation of sharing based on addition and subtraction.

[0123] In actual implementation, such as Figure 2 As shown, a first server P0, a second server P1, and an auxiliary computing server P2 can be configured to perform addition sharing. All variables in the privacy computation are added and shared between P0 and P1.

[0124] For example, for an integer variable x∈Z2, P0 holds [x]0, P1 holds [x]1, and [x]0 + [x]1 = x. Here, x is the first data, and [x]0 and [x]1 are two subsequences.

[0125] At the same time, for any x, [x]0 and [x]1 are uniformly distributed in On top.

[0126] When it is necessary to rebuild an addition-shared value, simply add the addition-shared values ​​of P0 and P1 together.

[0127] For example, continue to refer to Figure 2 Let x and y share the addition between P0 and P1. To calculate the addition share of z = x + y, we can calculate [z] = [x]0 + [y]0 through P0 and [z]1 = [x]1 + [y]1 through P1, thus obtaining the addition share of z.

[0128] Subtraction is calculated using a similar method, which will not be elaborated here.

[0129] After adding and sharing the first data to obtain at least two subsequences, the at least two subsequences can be further multiplied and shared to obtain the first encrypted data.

[0130] The following section provides a detailed explanation of sharing based on multiplication.

[0131] In some embodiments, determining the first encrypted data based on at least two sub-sequences may include:

[0132] Obtain the target multiplication triple;

[0133] The first encrypted data is determined based on the target multiplication triple and at least two subsequences.

[0134] In this embodiment, the target multiplication triple can be a user-defined multiplication triple. For example, the multiplication triple can be defined as: u, v, w, used to represent three numbers that satisfy w = uv, and these three numbers are all Integers above.

[0135] In this embodiment, the triples u, v, w can be randomly generated by the auxiliary computing server P2, and then added and shared with P0 and P1.

[0136] Continue to refer to Figure 2 Suppose that P0 and P1 have already shared the three numbers in the triplet through addition, where P0 shares [x]0 - [u]0 and P1 shares [x]1 - [u]1, and [x]0 + [x]1 = x, [u]0 + [u]1 = u. Now, we need to multiply the x and y that they shared through addition. The process is as follows:

[0137] P0 and P1 send their respective shared [x]0-[u]0 and [x]1-[u]1 to each other, so that both of them have the value of xu.

[0138] Using a similar method, P1 and P2 can also obtain the value of yv.

[0139] P0 calculates (xu)(yv)+[u]0(yv)+(xu)[v]0+[w]0=[z]0;

[0140] P1 calculates [u]1(yv)+(xu)[v]1+[w]1=[z]1.

[0141] Therefore:

[0142] [z]0+[z]1=(xu)(yv)+([u]0+[u]1)(yv)+(xu)([v]0+[v]1)+[w]0+[w]=x(yv)+

[0143] (xu)v+uv=xy

[0144] That is, P0 and P1 have obtained the shared value of z = xy (i.e., the first encrypted data), and the calculation is complete.

[0145] In some embodiments, where P0 directly possesses x and P1 directly possesses y, it can be directly assumed that:

[0146] [x]0=x,[x]1=0,[y]0=0,[y]1=y,

[0147] Then, based on a similar method described above, it is converted into a secret-shared multiplication to obtain the first encrypted data.

[0148] Understandably, in a secret-shared multiplication, xu and yv are public, while u and v are unknown. Therefore, the public values ​​do not reveal information about x and y.

[0149] Meanwhile, since [w]0 and [w]1 are uniformly distributed, the final result (i.e., the first encrypted data) is also completely random for one party.

[0150] In some embodiments, the above-described multiplication-based sharing can also be applied to vector and matrix scenarios. To perform x and y multiplication, the triples u and v need to have the same shape as x and y, respectively. The specific multiplication sharing steps are similar to those described above and will not be repeated here.

[0151] In some embodiments, if one of the two numbers x and y being multiplied is public, such as y, then P0 only needs to calculate [z]0 = [x0]0y and P1 needs to calculate [z]1 = [x]1y to obtain the final first encrypted data, without revealing information about x.

[0152] According to the data processing method provided in the embodiments of this application, by generating first encrypted data through secret sharing based on addition, subtraction and multiplication of the first data, the leakage of the first data can be avoided and the probability of obtaining the first data by reverse deduction based on the first encrypted data can be significantly reduced, thereby improving the security and privacy of the intermediate results.

[0153] like Figure 3 As shown, in some embodiments, inputting the first encrypted data into a nonlinear element-wise computation layer to obtain the target encrypted data output by the nonlinear element-wise computation layer may include:

[0154] Obtain the first shared value and the second shared value corresponding to the first encrypted data;

[0155] The first and second shared values ​​are processed separately using a target random permutation to obtain the third and fourth shared values;

[0156] The element-wise function result is determined based on the third and fourth sharing values;

[0157] Split the element-wise function result to obtain the fifth and sixth shared values;

[0158] The fifth and sixth sharing values ​​are reversed to determine the target encrypted data.

[0159] In this embodiment, the input to the function to be calculated is a value that has already been shared by addition between P0 and P1, which is the first encrypted data.

[0160] The first shared value and the second shared value are the values ​​shared by the first server P0 and the second server P1, which are participating in the addition sharing.

[0161] The target random permutation can be a permutation randomly generated by the system.

[0162] Understandably, the number of permutations grows exponentially. For example, for a permutation of 10 elements, more than 3 million possible permutations can be randomly generated, and the target random permutation can be any one of these permutations.

[0163] Nonlinear element-wise function calculation, such as Figure 4 As shown.

[0164] For example, such as Figure 3 As shown, in actual execution, for vectors If the first share value in P0 is [x]0 and the second share value in P1 is [x]1, then P0 and P1 can negotiate a target random permutation π of length n to obtain the third share value π([x]0) corresponding to P0 and the fourth share value π([x]1) corresponding to P1; then send π([x]0) and π([x]1) to P2 respectively.

[0165] Continue to refer to Figure 3 P2 can determine π[xfloat] based on the obtained third and fourth share values, then calculate the corresponding element-wise function result f(π([xfloat])=π(f[xfloat]), and then convert it into a fixed-point number and send it to P0 and P1 in an additive sharing manner.

[0166] After P0 receives the fifth shared value from P2 and P1 receives the sixth shared value from P2, P0 and P1 then obtain the required result f(xfloat)fixed, i.e., the target encrypted data, by reversing the order.

[0167] According to the data processing method provided in the embodiments of this application, the first shared value and the second shared value are processed by random arrangement, which can improve the randomness of the generated shared value, thereby further reducing the possibility of deducing the original valid information from the generated target encrypted data, significantly improving the privacy and security of the intermediate results, and thus improving the data privacy of split learning.

[0168] In some embodiments, processing the first and second shared values ​​using a target random permutation to obtain the third and fourth shared values ​​may include:

[0169] Obtain the target randomly flipped vector;

[0170] Based on the target random permutation and target random flip vector, the first and second shared values ​​are processed respectively to obtain the third and fourth shared values.

[0171] In this embodiment, the target random flip vector is used to calculate a specific nonlinear element-wise function to avoid exposing the distribution of elements in the data after random permutation (such as exposing the distribution of the last layer labels of the neural network), thereby further protecting privacy.

[0172] The target random flip vector is any random flip vector.

[0173] For example, continuing with the above embodiment, for vectors If the first share value in P0 is [x]0 and the second share value in P1 is [x]1, then P0 and P1 can negotiate a target random permutation π of length n and a target random flip vector σ of length n, thereby generating a third share value π ([σ⊙x]0) and a fourth share value π ([σ⊙x]1); where the elements in σ randomly take the value 1 or -1.

[0174] Then send π([σ⊙x]0) and π([σ⊙x]1) to P2 respectively.

[0175] Where, σ⊙x=(σ1x1,...,σnxn).

[0176] P2 can obtain π[σ⊙xfloat] based on the third and fourth shared values, then calculate the element-wise function result f(π[σ⊙xfloat])=π(f[σ⊙xfloat]) corresponding to π[σ⊙xfloat], and then convert it into a fixed-point number and send it to P0 and P1 in an additive sharing manner.

[0177] P0 and P1 are then reversed to obtain the desired result f(σ⊙xfloat)fixed.

[0178] In some embodiments, for certain specific element-wise functions, f(-x) = g(f(x)), such as for the Sigmoid function, Sigmoid(-x)⁻¹-Sigmoid(x); for Tanh, Tanh(-x) = -Tanh(x).

[0179] In this case, P0 and P1 can further calculate the value of f(x) from f(σ⊙x).

[0180] According to the data processing method provided in the embodiments of this application, by processing the first shared value and the second shared value by using a random flipping method based on random permutation, it is possible to improve randomness while avoiding exposure of the distribution of each element in the random permutation, thereby further protecting data privacy, preventing leakage of intermediate results, and significantly improving the data security and privacy of the feature provider in the vertical federated learning process.

[0181] In some embodiments, the basic operator layer may further include a communication module.

[0182] In this embodiment, when P0, P1, and P2 are running simultaneously, the communication module performs asynchronous communication monitoring at the same time.

[0183] The communication protocol can be TCP.

[0184] The message sending format is: [message header, message body].

[0185] After receiving the message, the listening servers of each party categorize the message according to the sender and message header, and then store it in the message buffer in chronological order.

[0186] When any party needs to receive a message, it retrieves the message from the message cache of the listening server.

[0187] If the sender and message header queues corresponding to the current message buffer are empty, wait until there is a message in them.

[0188] In some embodiments, the basic operator layer may also include other local computation modules.

[0189] In this embodiment, other local computation modules are used to perform calculations such as tensor shape transformation, summation, and averaging.

[0190] For example, to transform a 10×100 tensor X into a 20×50 shape, it is only necessary to transform [X]0 into a 20×50 shape by P0 and transform [X]1 into a 20×50 shape by P1.

[0191] The above operations can be performed locally without any additional communication overhead.

[0192] Step 130: Based on the target intermediate results, obtain the target labels corresponding to the target intermediate results.

[0193] In this step, the target label is a label that matches the target intermediate result, predicted based on the target intermediate result.

[0194] This target label can be approximated as the label corresponding to the target industrial characteristic data.

[0195] After the feature provider generates the target intermediate result, the target intermediate result is sent to the label provider so that the label provider can predict the target label corresponding to the target intermediate result based on the target intermediate result.

[0196] After receiving the encrypted output values ​​(i.e., target intermediate results) from all feature providers, the label provider uses the target intermediate results to make predictions and obtain the target labels.

[0197] Continuing with the example of industrial characteristic data such as the construction year and average housing price of a user's residential community, Company A encrypts the industrial characteristic data, such as the construction year, of the user's residential community in region A to obtain an initial intermediate result. Company A then performs addition-based sharing encryption on the initial intermediate result to obtain the target intermediate result and sends it to the tag provider (e.g., Company C). Simultaneously, Company B encrypts the industrial characteristic data, such as the average housing price and construction year, of the residential community in region A to obtain an initial intermediate result. Company B then performs addition-based sharing encryption on the initial intermediate result to obtain the target intermediate result and sends it to the tag provider (e.g., Company C).

[0198] After receiving the target intermediate results sent by Company A and Company B, Company C makes a comprehensive prediction based on all the received target intermediate results to obtain the corresponding label, such as whether users in region A are willing to buy products (such as houses) in region A.

[0199] According to the data processing method provided in the embodiments of this application, during the vertical federated learning process, the feature provider performs encryption processing on the initial intermediate results based on addition sharing to obtain the target intermediate results, and sends the target intermediate results to the label provider to predict the target label based on the target intermediate results. The intermediate results can be efficiently encrypted without multiple encryption and decryption, and the encryption effect is good. It can significantly reduce the risk of reverse inferring the target industrial feature data based on the intermediate results, thereby reducing the risk of data privacy leakage of the feature provider due to leakage of intermediate results, and effectively improving the data security in the vertical federated learning process.

[0200] In actual execution, steps 120 and 130 can be performed based on a neural network model.

[0201] The neural network model can be a vertical federated learning model, which includes multiple first sub-models and second sub-models, with the output of each first sub-model connected to the input of the second sub-model.

[0202] Each first sub-model corresponds to a feature provider, and the second sub-model corresponds to a label provider.

[0203] There is no data sharing between the first sub-models.

[0204] The input data for the first sub-model is the industrial feature data provided by the feature provider corresponding to the first sub-model, and the output data for the first sub-model is the target result data.

[0205] The input data for the second sub-model consists of the target result data provided by each feature provider, and the output data of the second sub-model consists of the target label predicted based on the input target result data.

[0206] For example, a typical two-layer split learning model can be represented as the following function:

[0207]

[0208] in, Let Mt represent the model prediction result, Mhi represent the tail model (i.e., the second sub-model), Mhi represent the i-th head model (i.e., the first sub-model), and xi represent the input feature corresponding to the i-th head model.

[0209] In the context of split learning, Mt, Mh1, ..., Mhn and their corresponding inputs: model labels y and different input features x1, ..., xn are often owned by different stakeholders.

[0210] In this application, the participant who possesses the tail model Mt and the label y is called the label provider, and the participant who possesses the head model Mhi and its corresponding input xi is called the i-th feature provider.

[0211] The following section uses a vertical federated learning model as an example to illustrate the specific implementation of this application.

[0212] In some embodiments, the target industrial feature data is encrypted to obtain an initial intermediate result, and the initial intermediate result is sent to a privacy computing system for encryption to obtain the target intermediate result output by the privacy computing system; based on the target intermediate result, the target label corresponding to the target intermediate result is obtained, which may include:

[0213] Input the target industrial feature data into the target first sub-model among multiple first sub-models, and obtain the target intermediate results output by the target first sub-model;

[0214] Input the intermediate target result into the second sub-model and obtain the target label output by the second sub-model;

[0215] The first sub-model is trained based on the sample industrial feature data corresponding to the first sub-model and the sample intermediate results corresponding to the sample industrial feature data. The second sub-model is trained based on the sample intermediate results and the sample labels corresponding to the sample industrial feature data. The sample intermediate results are data that has been encrypted by a privacy computing system.

[0216] In this embodiment, the first sub-model of the target is the first sub-model corresponding to the target feature provider that provides the target feature data.

[0217] The first sub-model of the target is a model obtained by training the target feature provider separately. The training samples of the first sub-model of the target include the sample industrial feature data provided by the target feature provider.

[0218] The intermediate sample result is the data obtained by further encrypting the initial encrypted data obtained by the feature provider through routine encryption of the sample industrial feature data in federated learning.

[0219] The second sub-model is a model trained by the label provider. The training samples of the second sub-model include intermediate sample results provided by the feature providers corresponding to the multiple first sub-models associated with the second sub-model, as well as sample labels corresponding to the sample industrial feature data of each feature provider.

[0220] It should be noted that during the training process, each first sub-model is trained independently, and the intermediate sample results output by the first sub-model are sent to the second sub-model to train the second sub-model.

[0221] According to the data processing method provided in the embodiments of this application, by optimizing the splitting model during the vertical federated learning process, the intermediate results output by the head model in the splitting model are the results after secondary encryption, which reduces the risk of data privacy leakage caused by back-deriving feature data through intermediate results, significantly improves data security in the vertical federated learning process, and has good data processing effect and strong learning ability.

[0222] The training process of the federated learning model is explained below.

[0223] In some embodiments, intermediate sample results can be determined in the following way:

[0224] The sample industrial feature data is encrypted to obtain the first intermediate result;

[0225] The first intermediate result is sent to the privacy computing system for encryption, and the sample intermediate result output by the privacy computing system is obtained.

[0226] In this embodiment, the first intermediate result is the intermediate result obtained by performing conventional encryption processing on the sample industrial feature data in federated learning.

[0227] The specific encryption methods of the privacy computing system have been described in the above embodiments and will not be repeated here.

[0228] For example, the i-th feature provider first obtains its corresponding sample industrial feature data Xi and inputs it into the first sub-model Mhi before optimization to obtain the first intermediate result Hi; wherein, the first sub-model Mhi before optimization is only used to perform basic encryption processing on the sample industrial feature data. The first sub-model Mhi before optimization is trained with the sample industrial feature data as the sample and the initial intermediate result of the sample corresponding to the sample industrial feature data as the sample label. The initial intermediate result of the sample is the intermediate result without encryption processing.

[0229] After obtaining the first intermediate result Hi, the first intermediate result Hi is encrypted using addition-based sharing to obtain the sample intermediate result H″i, and the sample intermediate result H″i is sent to the tag provider.

[0230] The data processing method provided in this application can be executed by a data processing device. This application uses an example of a data processing device executing the data processing method to illustrate the data processing device provided in this application.

[0231] This application also provides a data processing apparatus.

[0232] like Figure 4 As shown, the data processing device includes: a first processing module 410, a second processing module 420 and a third processing module 430.

[0233] The first processing module 410 is used to acquire target industrial characteristic data;

[0234] The second processing module 420 is used to encrypt the target industrial feature data to obtain the initial intermediate result, and send the initial intermediate result to the privacy computing system for encryption processing to obtain the target intermediate result output by the privacy computing system.

[0235] The third processing module 430 is used to obtain the target label corresponding to the target intermediate result based on the target intermediate result;

[0236] The privacy computing system includes a first server and a second server that participate in addition sharing, as well as an auxiliary computing server for auxiliary computing.

[0237] According to the data processing apparatus provided in the embodiments of this application, during the vertical federated learning process, the feature provider performs encryption processing on the initial intermediate results based on addition sharing to obtain the target intermediate results, and sends the target intermediate results to the label provider to predict the target label based on the target intermediate results. The intermediate results can be efficiently encrypted without multiple encryption and decryption, and the encryption effect is good. It can significantly reduce the risk of reverse inferring the target industrial feature data based on the intermediate results, thereby reducing the risk of leakage of the intermediate results leading to the leakage of the feature provider's data privacy, and effectively improving the data security in the vertical federated learning process.

[0238] In some embodiments, the privacy computing system may include a layer of basic operators, a layer of differentiable operators, and a model layer connected in sequence, and the second processing module 420 may also be used for:

[0239] The initial intermediate results are input into the basic operator layer to obtain the encrypted data output by the basic operator layer;

[0240] Encrypted data is input into the differentiable operator layer for operator format packaging, and the differential data output by the differentiable operator layer is obtained.

[0241] The differential data is input into the model layer to obtain the target intermediate result output by the model layer.

[0242] In some embodiments, the second processing module 420 may also be used for:

[0243] The initial intermediate results are subjected to linear encryption and nonlinear element-wise function calculations to obtain encrypted data.

[0244] In some embodiments, the second processing module 420 may also be used for:

[0245] Determine the dependencies between multiple encrypted data;

[0246] Based on dependencies, the operation functions are called sequentially to process multiple encrypted data and obtain differential data.

[0247] In some embodiments, the device may further include:

[0248] The fourth processing module is used to input the target industrial feature data into the target first sub-model among multiple first sub-models, and obtain the target intermediate results output by the target first sub-model;

[0249] The fifth processing module is used to input the intermediate results of the target into the second sub-model and obtain the target label output by the second sub-model;

[0250] The first sub-model is trained based on the sample industrial feature data corresponding to the first sub-model and the sample intermediate results corresponding to the sample industrial feature data. The second sub-model is trained based on the sample intermediate results and the sample labels corresponding to the sample industrial feature data. The sample intermediate results are data that has been encrypted by a privacy computing system.

[0251] In some embodiments, the device may further include:

[0252] The sixth processing module is used to encrypt the sample industrial feature data and obtain the first intermediate result;

[0253] The seventh processing module is used to send the first intermediate result to the privacy computing system for encryption processing and obtain the sample intermediate result output by the privacy computing system.

[0254] The data processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0255] The data processing device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0256] The data processing device provided in this application embodiment can achieve... Figures 1 to 3 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0257] In some embodiments, such as Figure 5As shown, this application embodiment also provides an electronic device 500, including a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, it implements the various processes of the above-described data processing method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0258] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0259] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described data processing method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0260] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0261] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described data processing method.

[0262] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0263] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above data processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0264] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0265] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0266] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0267] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0268] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0269] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A data processing method, characterized in that, include: Acquire target industrial characteristic data; The target industrial feature data is encrypted to obtain an initial intermediate result, and the initial intermediate result is sent to a privacy computing system for encryption to obtain the target intermediate result output by the privacy computing system. Based on the target intermediate result, obtain the target label corresponding to the target intermediate result; The privacy computing system includes a first server and a second server participating in addition sharing, as well as an auxiliary computing server for assisting computing. The process involves encrypting the target industrial feature data to obtain an initial intermediate result, sending the initial intermediate result to a privacy computing system for further encryption, and obtaining the target intermediate result output by the privacy computing system. Based on the target intermediate result, the process also involves obtaining the target label corresponding to the target intermediate result, including: The target industrial feature data is input into the target first sub-model among multiple first sub-models, and the target intermediate result output by the target first sub-model is obtained; The target intermediate result is input into the second sub-model to obtain the target label output by the second sub-model; The first sub-model of the target is trained based on the sample industrial feature data corresponding to the first sub-model of the target and the sample intermediate results corresponding to the sample industrial feature data. The second sub-model is trained based on the sample intermediate results and the sample labels corresponding to the sample industrial feature data. The sample intermediate results are data that has been encrypted by the privacy computing system. The intermediate results of the samples are determined in the following manner: The sample industrial feature data is encrypted to obtain a first intermediate result; The first intermediate result is sent to the privacy computing system for encryption processing, and the sample intermediate result output by the privacy computing system is obtained.

2. The data processing method according to claim 1, characterized in that, The privacy computing system comprises a sequentially connected layer of basic operators, a layer of differentiable operators, and a model layer. The initial intermediate result is sent to the privacy computing system for encryption processing. The target intermediate result output by the privacy computing system is then obtained, including: The initial intermediate result is input into the basic operator layer to obtain the encrypted data output by the basic operator layer; The encrypted data is input into the differentiable operator layer for operator format packaging, and the differential data output by the differentiable operator layer is obtained. The differential data is input into the model layer to obtain the target intermediate result output by the model layer.

3. The data processing method according to claim 2, characterized in that, The step of inputting the initial intermediate result into the basic operator layer and obtaining the encrypted data output by the basic operator layer includes: The initial intermediate results are subjected to linear encryption and nonlinear element-by-element function calculations to obtain the encrypted data.

4. The data processing method according to claim 2, characterized in that, The step of inputting the encrypted data into a differentiable operator layer for operator format packaging and obtaining the differential data output by the differentiable operator layer includes: Determine the dependencies between multiple encrypted data; Based on the aforementioned dependency relationship, the operation functions are sequentially called to process the multiple encrypted data and obtain the differential data.

5. A data processing apparatus, characterized in that, The data processing apparatus is used to implement the data processing method as described in any one of claims 1-4, comprising: The first processing module is used to acquire target industrial characteristic data; The second processing module is used to encrypt the target industrial feature data to obtain an initial intermediate result, and send the initial intermediate result to the privacy computing system for encryption processing to obtain the target intermediate result output by the privacy computing system; The third processing module is used to obtain the target label corresponding to the target intermediate result based on the target intermediate result; The privacy computing system includes a first server and a second server participating in addition sharing, as well as an auxiliary computing server for assisting computing.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the data processing method as described in any one of claims 1-4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the data processing method as described in any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the data processing method as described in any one of claims 1-4.

Images