Data processing method

Abstract

This application discloses a data processing method. The method includes: acquiring multiple first data subsets from multiple federated learning participants, wherein each federated learning participant provides one first data subset; processing the multiple first data subsets using a distance-based local differential privacy algorithm to obtain multiple second data subsets, wherein the distance is the distance between data within the first data subsets; combining the multiple second data subsets into a training data set, and inputting the training data set into a decision tree learning model for training to obtain a target decision tree.

Description

Technical Field

[0001] This application relates to the field of machine learning, and more specifically, to a data processing method. Background Technology

[0002] Gradient boosting decision trees, exemplified by XGBoost (Extreme Gradient Boosting), combine the advantages of decision trees—ease of understanding, fast prediction speed, robustness against outliers, and ease of parallelization—with the overfitting resistance of boosting techniques. They are common machine learning algorithms in e-commerce. In today's big data landscape, there is a growing demand for federated learning among data entities. This involves combining data from various sources to achieve joint modeling and prediction while ensuring data security and privacy. Gradient boosting decision trees are often the preferred decision tree learning model for clients.

[0003] While decision tree federated learning schemes have been proposed in related technologies, these schemes have several drawbacks: 1) They require significant modifications to the decision tree algorithm, making it impossible to directly use existing decision tree algorithms for training, thus introducing substantial engineering costs; 2) They generally require the introduction of cryptographic techniques such as homomorphic encryption and secure multi-party computation, which significantly reduces performance; 3) In vertical federated learning, the computation process involves complex sorting and comparison operations, so intermediate results are usually decrypted during the learning process in practical applications, reducing the security of the scheme and making it difficult to theoretically discuss the security or privacy protection level of the scheme; 4) They generally only support joint training and cannot support joint prediction.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a data processing method to at least solve the technical problems in related technologies where federated learning requires significant modifications to the decision tree algorithm and the introduction of complex encryption techniques, resulting in high costs, low security, and the inability to achieve joint prediction rather than joint training.

[0006] According to one aspect of the embodiments of this application, a data processing method is provided, comprising: acquiring a plurality of first data subsets from a plurality of federated learning participants, wherein each federated learning participant provides a first data subset; processing the plurality of first data subsets using a distance-based local differential privacy algorithm to obtain a plurality of second data subsets, wherein the distance is the distance between data in the first data subsets; forming a training data set from the plurality of second data subsets, and inputting the training data set into a decision tree learning model for training to obtain a target decision tree.

[0007] According to another aspect of the embodiments of this application, another data processing method is also provided, including: obtaining multiple fifth data subsets from multiple federated learning participants, wherein each federated learning participant provides a fifth data subset, the fifth data subset including feature data; processing the multiple fifth data subsets with a distance-based local differential privacy algorithm to obtain multiple sixth data subsets, wherein the distance is the distance between data in the fifth data subsets; forming a prediction data set from the multiple sixth data subsets, and inputting the prediction data set into a target decision tree for prediction to obtain label data corresponding to the feature data.

[0008] According to another aspect of the embodiments of this application, another data processing method is also provided, including: obtaining multiple first data subsets and multiple fifth data subsets from multiple federated learning participants, wherein each federated learning participant provides one first data subset and one fifth data subset; processing the multiple first data subsets with a distance-based local differential privacy algorithm to obtain multiple second data subsets, wherein the distance is the distance between data in the first data subsets; processing the multiple fifth data subsets with a distance-based local differential privacy algorithm to obtain multiple sixth data subsets, wherein the distance is the distance between data in the fifth data subsets; forming a training data set from the multiple second data subsets, and inputting the training data set into a decision tree learning model for training to obtain a target decision tree; forming a prediction data set from the multiple sixth data subsets, and inputting the prediction data set into the target decision tree to obtain a prediction result.

[0009] According to another aspect of the embodiments of this application, a data processing apparatus is also provided, comprising: an acquisition module, configured to acquire multiple first data subsets from multiple federated learning participants, wherein each federated learning participant provides a first data subset; a processing module, configured to process the multiple first data subsets using a distance-based local differential privacy algorithm to obtain multiple second data subsets, wherein the distance is the distance between data in the first data subsets; and a training module, configured to combine the multiple second data subsets into a training data set, and input the training data set into a decision tree learning model for training to obtain a target decision tree.

[0010] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute the above-described data processing method.

[0011] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a processor and a memory, the memory being connected to the processor and configured to provide the processor with instructions for processing the following processing steps: acquiring multiple first data subsets from multiple federated learning participants, wherein each federated learning participant provides a first data subset; processing the multiple first data subsets using a distance-based local differential privacy algorithm to obtain multiple second data subsets, wherein the distance is the distance between data in the first data subsets; forming a training data set from the multiple second data subsets, and inputting the training data set into a decision tree learning model for training to obtain a target decision tree.

[0012] In this embodiment, a distance-based local differential privacy algorithm is introduced into decision tree training. Leveraging the partial order-preserving properties of this algorithm, the privacy and accuracy of decision tree training are balanced. Since the entire training process only applies distance-based local differential privacy processing to the input data, and the data type and format remain unchanged after processing, existing decision tree algorithm libraries can be directly called to complete the training. Furthermore, the method of adding noise based on the distance-based local differential privacy algorithm is simple, easy to understand, and readily applicable to distributed processing. It can simultaneously support horizontal / vertical federated training and horizontal / vertical federated prediction. This application effectively solves the technical problems of related technologies that require significant modifications to the decision tree algorithm and the introduction of complex encryption techniques for federated learning, resulting in high costs, low security, and the inability to achieve joint prediction while only enabling joint training. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0014] Figure 1 This is a schematic diagram of the structure of a computer terminal according to an embodiment of this application;

[0015] Figure 2 This is a flowchart illustrating a data processing method according to an embodiment of this application;

[0016] Figure 3 This is a flowchart illustrating another data processing method according to an embodiment of this application;

[0017] Figure 4 This is a flowchart illustrating another data processing method according to an embodiment of this application;

[0018] Figure 5 This is a schematic diagram of a decision tree federated learning process according to an embodiment of this application;

[0019] Figure 6 This is a schematic diagram of the structure of a data processing device according to an embodiment of this application. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0023] 1. Differential Privacy: This is a privacy protection algorithm that reduces the impact of individual data on the aggregation result by adding a certain amount of noise to the aggregated data, thereby protecting personal privacy while allowing data owners to publish relatively accurate aggregated data.

[0024] Specifically, assume that the data owner holds the real data for each individual, forming a dataset D. Let M be a noisy algorithm that publishes aggregated data results on dataset D, and let Pr[M(D)∈S] represent the data results published by M on dataset D in set S. If for any two neighboring datasets D and D' (i.e., the two datasets differ by only one individual's data), for any set S, Pr[M(D)∈S]≤e^ε·Pr[M(D')∈S], then algorithm M is said to satisfy ε-differential privacy.

[0025] Here, ε is called the privacy budget, which is directly proportional to the accuracy of the data output of the differential privacy algorithm and inversely proportional to the privacy protection level of the output data. That is, the more budget is spent, the more accurate the output of algorithm M is, but the worse the privacy protection level is; the less budget is spent, the less accurate the output of algorithm M is, but the better the privacy protection level is.

[0026] 2. Local Differential Privacy: In differential privacy, the data owner is completely trustworthy and holds the real data of each individual; while in local differential privacy, the data owner is not completely trustworthy, and each individual adds a certain amount of noise to the data before uploading it to the data owner.

[0027] Specifically, given the set X of all possible values ​​of the data held by an individual, let M be an algorithm for noise processing on the input data x∈X, and let Pr[M(x)∈S] denote the data results published by M on dataset D in set S. If for any possible pair of input data x, x'∈X, Pr[M(x)∈S]≤e^ε·Pr[M(x')∈S], then the algorithm M is said to satisfy ε-local differential privacy.

[0028] 3. Distance-based Local Differential Privacy: Local differential privacy requires that for any set of data x, x' ∈ X, the algorithm M must satisfy Pr[M(x) ∈ S] ≤ e^ε·Pr[M(x') ∈ S]. However, when x is continuous data, in order to ensure that the definition of local differential privacy holds for any x, x' ∈ X, the noise level introduced by the algorithm M is generally related to the maximum value of x. This privacy definition is too strict in many application scenarios.

[0029] Distance-based local differential privacy is a weakened version of local differential privacy proposed in this application. Its basic idea is: if x and x' are close, then M(x) and M(x') are difficult to distinguish; if x and x' are far apart, then M(x) and M(x') are relatively easy to distinguish. For example, X represents the geographical location of an individual. If individual x and individual x' work in the same city and on the same street, and their locations are relatively close, then this relatively accurate location information should be protected; conversely, if individual x and individual x' work in different cities, and their locations are relatively far apart, then this relatively ambiguous location information has relatively low sensitivity.

[0030] Specifically, given the set X of all possible values ​​of the data held by an individual, let M be an algorithm for noise processing on the input data x∈X, and let Pr[M(x)∈S] denote the data results published by M on dataset D in set S. If for any possible pair of input data x, x'∈X, Pr[M(x)∈S]≤e^(ε·D(x,x'))·Pr[M(x')∈S], then the algorithm M is said to satisfy ε-local differential privacy, where D(x,x') is the distance between data x and x', i.e., D(x,x')=|x-x'|.

[0031] 4. Decision Tree: A decision tree is a predictive model represented as a tree. Each node in the tree represents an object, each branch path represents a possible attribute value, and each leaf node corresponds to the value of the object represented by the path from the root node to that leaf node. A decision tree has only a single output, but multiple decision tree models can be built to output multiple possible values. Decision trees are a commonly used technique for both regression and classification. The machine learning technique that generates decision trees from data is called decision tree learning, and is often simply referred to as a decision tree.

[0032] The most significant drawback of decision trees is their susceptibility to overfitting when used alone. To address this issue, decision trees are typically combined with techniques such as boosting and bagging. The machine learning algorithm that combines decision trees with boosting is called Gradient Boosting Decision Tree (GBDT), and the classic machine learning algorithm XGBoost also belongs to the GBDT category. The machine learning algorithm that combines decision trees with bagging is called Random Forest.

[0033] 5. Federated Learning: A distributed machine learning technique that aims to achieve collaborative modeling among multiple participants while ensuring data privacy, security, and legal compliance. In federated learning, each participating enterprise is referred to as a participant. Based on the different data distributions among the participants, federated learning is divided into three categories: horizontal federated learning, vertical federated learning, and federated transfer learning. Federated transfer learning is primarily applicable to scenarios using deep neural networks as the base model and is not considered in this application. Horizontal federated learning essentially involves the joint use of samples and is suitable for scenarios where participants have similar business models but reach different customers, i.e., when there is significant feature overlap but minimal user overlap. For example, banks in different regions offer similar services (similar features) but have different users (different samples). Vertical federated learning essentially involves the joint use of features and is suitable for scenarios with significant user overlap but minimal feature overlap. For example, supermarkets and banks in the same region reach residents of that region (same samples) but offer different services (different features).

[0034] 6. Data Discretization: Also known as data grouping, this refers to grouping continuous data into discrete intervals. This simplifies model training, reduces the risk of overfitting, and the discretized features are highly robust to outliers. Common data discretization methods include: equal-width discretization, which determines a fixed width based on the number of groups, ensuring that the variables in each group have equal widths. While simple to implement, it is sensitive to outliers and can unevenly distribute attribute values ​​across intervals, resulting in some intervals containing more variables than others; and equal-frequency discretization, also called quantile grouping, which ensures that each group has the same number of variables, but assigns identical variable values ​​to different groups. This application primarily uses equal-width discretization in its embodiments.

[0035] Example 1

[0036] According to an embodiment of this application, a data processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0037] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method is shown. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0038] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0039] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-mentioned application vulnerability detection method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0040] The transmission module 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0041] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0042] Under the aforementioned operating environment, this application provides a data processing method applied to decision tree federated learning, such as... Figure 2 As shown, the method includes steps S202-S206, wherein:

[0043] Step S202: Obtain multiple first data subsets from multiple federated learning participants, wherein each federated learning participant provides one first data subset.

[0044] In some optional embodiments of this application, it is first necessary to obtain training data provided by multiple federated learning participants, and then use the training data provided by each participant as a first data subset. For example, when shopping malls, banks, etc. in a certain region conduct decision tree federated learning, the user data provided by each shopping mall and bank is used as a first data subset, which is then processed and used to train the decision tree learning model.

[0045] In this context, for any first data subset, the subset includes feature data that serves as input to the decision tree learning model, such as the number of a user's family members and geographical location. Meanwhile, among the multiple first data subsets provided by multiple federated learning participants, at least one first data subset includes label data that serves as output to the decision tree learning model, such as the user's consumption level. The decision tree learning model can predict the user's consumption level using feature data such as the number of a user's family members and geographical location.

[0046] Step S204: Process multiple first data subsets using a distance-based local differential privacy algorithm to obtain multiple second data subsets, wherein the distance is the distance between data in the first data subsets.

[0047] In decision tree training, there is a very special property: if the feature data is a continuous attribute, the model training process of the decision tree learning algorithm only depends on the order of the feature data, rather than the values of the feature data. Therefore, if the original data is processed for privacy protection and the processed result is ensured to satisfy the requirement of unchanged order as much as possible, relatively accurate decision tree federated learning can be achieved on the basis of privacy protection.

[0048] In some optional embodiments of the present application, a distance-based local differential privacy algorithm is introduced to process the data: First, for any first data subset, calculate the distances between the data in the first data subset; according to the distances and the local differential privacy algorithm, determine the distance-based local differential privacy of the distance-based local differential privacy algorithm; determine the noise data according to the distance-based local differential privacy algorithm, and add the distance-based local differential privacy noise data to the data in the first data subset.

[0049] Among them, the distance between data is the distance in space. For example, if a certain feature data is a geographical location, the distance between two data is the actual distance between the two geographical locations in the physical space; this distance can also be understood as the difference between data. For example, if a certain feature data is the number of family members, the distance between two data is the difference in the number of family members.

[0050] Due to the requirements of the distance-based local differential privacy algorithm, for any possible pair of input data x, x' ∈ X, there is Pr[M(x) ∈ S] ≤ e^(ε·D(x,x'))·Pr[M(x') ∈ S], (where D(x,x') represents the distance between x and x', that is, D(x,x') = |x - x'|). That is to say, for any possible pair of input data x, x' ∈ X, the larger D(x,x') is, the more likely M(x) and M(x') are to have different results; this also means that if x < x', the larger D(x,x') is, the higher the probability that M(x) < M(x'). Therefore, the distance-based local differential privacy algorithm not only satisfies the privacy protection processing of the original data, but also ensures that the processed result satisfies the requirement of unchanged order as much as possible.

[0051] In some optional embodiments of this application, a privacy budget ε for a distance-based local differential privacy algorithm can be determined first. This privacy budget is directly proportional to the accuracy of the data output by the algorithm and inversely proportional to the privacy protection level of the output data. Specifically, a larger ε results in better algorithm orderliness and more accurate output data, but weaker privacy protection; conversely, a smaller ε results in poorer algorithm orderliness and less accurate output data, but stronger privacy protection. During training, the size of ε can be adjusted to balance data orderliness and privacy protection strength. Then, noise with parameter ε is added to the data in the first data subset. The noise type includes at least one of the following: Laplace distribution noise, geometric distribution noise, and step distribution noise. For example, given the j-th column data x_(i,j) of the i-th individual, adding Laplace distribution noise with parameter ε to x_(i,j) yields training data x'_(i,j).

[0052] It's important to note that because the noise in distance-based local differential privacy follows a continuous distribution, if a column contains a large number of data points with identical values, adding noise will greatly increase the probability that these values ​​will be different. In decision tree training, identical data points are treated as a whole; therefore, adding noise to transform identical data into different data points will significantly impact the model's accuracy. To address this issue, the data can be discretized before applying the distance-based local differential privacy algorithm.

[0053] In some optional embodiments of this application, the data in the first data subset is first discretized with equal width according to a preset unit distance to obtain a third data subset; then, noisy data is determined according to a distance-based local differential privacy algorithm, and noisy data is added to the data in the third data subset to obtain a fourth data subset; then, the data in the fourth data subset is dediscretized according to a preset unit distance to obtain a second data subset.

[0054] For example, in federated learning, each participant defines a preset unit distance Δd for a column of data with identical values ​​(in vertical federated learning, the participant holding this column of data defines its own preset unit distance Δd). Then, x_i is discretized by calculating y_i = x_i / Δd and rounding it down to obtain y_i. After applying a distance-based local differential privacy algorithm to y_i, the result with added noise is rounded down. Then, x'_i = y'_i·Δd is calculated again to de-discrete the data, thus obtaining data x'_i that satisfies distance-based local differential privacy. By discretizing the data, the introduced noise is also transformed into discrete values, thereby significantly reducing the possibility of identical data being transformed into different data.

[0055] The preset unit distance Δd can be defined based on the meaning of the data in that column. For example, if a column's feature data is the number of family members, a difference of 2 family members can be used as the dividing point, in which case the preset unit distance Δd = 2. If a column's feature data is consumption level, the participants can also consider the geographical location of the sample when defining Δd. If it is a first-tier city, Δd = 1000 can be defined, and if it is a second-tier city, Δd = 500 can be defined. It should be noted that the above data are all illustrative and do not constitute a limitation on the actual value of the preset unit distance Δd.

[0056] In some optional embodiments of this application, for any first data subset, when the data in the first data subset is of an enumeration type, the enumeration type data is processed using a local differential privacy algorithm in related technologies.

[0057] Specifically, if a feature is of an enumeration type, and the feature enumerates all possible values ​​|d|, then when processing the enumeration type data, the true value of the data is retained with a first probability e^ε / (e^ε+|d|-1), and the data is transformed into other possible values ​​with a second probability 1 / (e^ε+|d|-1). For example, suppose the value range of a feature is {male, female, unknown} (|d|=3), and suppose the true value of this feature for an individual is "male". Then, the individual's feature value is kept unchanged as "male" with a probability of e^ε / (e^ε+3-1), the individual's feature value is changed to "female" with a probability of 1 / (e^ε+3-1), and the individual's feature value is changed to "unknown" with a probability of 1 / (e^ε+3-1).

[0058] Optionally, when processing the data of this enumeration type, the data of this enumeration type can be one-hot encoded, and then the one-hot encoded data can be processed by the local differential privacy algorithm in related technologies to obtain multiple sets of data. The multiple sets of data obtained are then added to the second data subset. For example, assuming that the value range of a certain feature is {male, female, unknown}, and assuming that the true value of this feature for a certain individual is "male", then after one-hot encoding of the feature data of this enumeration type, it will become three columns of data: feature_male:1, feature_female:0, feature_unknown:0. Then, for each value, it is changed from 1 to 0 with a probability of e^ε / (e^ε+1), and changed from 0 to 1 with a probability of 1 / (e^ε+1). Finally, the three columns of features are added to the second data subset for training.

[0059] It should be noted that the above process can be used not only to process feature data of enumeration type, but also to process label data of enumeration type.

[0060] Step S206: Combine multiple second data subsets into a training data set, and input the training data set into the decision tree learning model for training to obtain the target decision tree.

[0061] In some optional embodiments of this application, when assembling multiple second data subsets into a training data set, it is necessary to first determine the identifier information of each data in the multiple second data subsets to identify the sample object from which the data originates, such as the user's ID, mobile phone number, etc.; then, based on the identifier information of each data, the multiple second data subsets are securely aligned using methods such as privacy set intersection to obtain the training data set.

[0062] It should be noted that since the essence of horizontal federated learning is the union of samples, while the essence of vertical federated learning is the union of features, the above sample alignment process is only used for vertical federated learning. If it is horizontal federated learning, sample alignment is not required.

[0063] Finally, the training dataset is input into the decision tree learning model, with the feature data as the algorithm input and the label data as the algorithm output, thereby training the decision tree learning model to obtain the target decision tree.

[0064] In this embodiment, a distance-based local differential privacy algorithm is introduced into decision tree training. Leveraging the partial order-preserving properties of this algorithm, the privacy and accuracy of decision tree training are balanced. Since the entire training process only applies distance-based local differential privacy processing to the input data, and the data type and format remain unchanged after processing, existing decision tree algorithm libraries can be directly called to complete the training. Furthermore, the method of adding noise based on the distance-based local differential privacy algorithm is simple, easy to understand, and readily applicable to distributed processing. It can simultaneously support horizontal / vertical federated training, effectively solving the technical problem in related technologies where federated learning requires significant modifications to the decision tree algorithm and the introduction of complex encryption techniques, resulting in high costs and low security.

[0065] Example 2

[0066] According to an embodiment of this application, another data processing method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0067] The data processing method provided in this application embodiment can also be run in... Figure 1 The flowchart of the operating environment shown is as follows: Figure 3 As shown, the method includes steps S302-S306, wherein:

[0068] Step S302: Obtain multiple fifth data subsets from multiple federated learning participants, wherein each federated learning participant provides a fifth data subset, and the fifth data subset includes feature data.

[0069] In some optional embodiments of this application, feature data for prediction, such as the number of a user's family members and geographical location, is first obtained from multiple federated learning participants. Then, the feature data provided by each participant is used as a fifth data subset, which is processed and input into the target decision tree. The target decision tree will predict the corresponding label data, such as the user's consumption level, based on the feature data such as the number of the user's family members and geographical location.

[0070] Step S304: Process multiple fifth data subsets using a distance-based local differential privacy algorithm to obtain multiple sixth data subsets, wherein the distance is the distance between data in the fifth data subsets.

[0071] The processing of the fifth data subset is similar to that of the first data subset in Example 1: First, the privacy budget ε that satisfies the distance-based local differential privacy algorithm is determined. The size of ε can be adjusted to balance the order of data and the strength of privacy protection. Then, noise with parameter ε is added to the data in the fifth data subset. The noise type includes at least one of the following: Laplace distribution noise, geometric distribution noise, and step distribution noise. For example, given the j-th column data x_(i,j) of the i-th individual, by adding Laplace distribution noise with parameter ε to x_(i,j), the data x'_(i,j) used for prediction is obtained.

[0072] In some optional embodiments of this application, if a column contains a large number of data with the same value, the data in that column can be discretized first, and then distance-based local differential privacy can be used. Specifically, firstly, the data in the fifth data subset is discretized with equal width according to a preset unit distance to obtain the seventh data subset; then, noise that satisfies the distance-based local differential privacy algorithm is added to the data in the seventh data subset to obtain the eighth data subset; then, the data in the eighth data subset is dediscretized according to the preset unit distance to obtain the sixth data subset.

[0073] In some optional embodiments of this application, if a certain feature data is of an enumeration type, and the feature data enumerates all possible values ​​of |d|, then when processing the data of this enumeration type, the true value of the data is retained with a first probability e^ε / (e^ε+|d|-1), and the data is transformed into other possible values ​​with a second probability 1 / (e^ε+|d|-1). Optionally, when processing the data of this enumeration type, the data of this enumeration type can also be one-hot encoded, and then the one-hot encoded data can be processed by a local differential privacy algorithm in related technologies to obtain multiple sets of data, and the obtained multiple sets of data can be added to a sixth data subset.

[0074] Step S306: Combine multiple sixth data subsets into a prediction data set, and input the prediction data set into the target decision tree for prediction to obtain label data corresponding to the feature data.

[0075] If we are performing vertical federated learning, when combining multiple sixth data subsets into a prediction dataset, we need to first determine the identifier information of each data in the multiple sixth data subsets, such as user ID, mobile phone number, etc.; then, based on the identifier information of each data, we use methods such as privacy set intersection to perform secure sample alignment processing on the multiple sixth data subsets to obtain the prediction dataset.

[0076] In some optional embodiments of this application, when using a target decision tree for classification, the label data is processed using a local differential privacy algorithm from related technologies. The true value of the label data is retained with a first probability, and the label data is transformed into other values ​​with a second probability. That is, during prediction, local differential privacy processing of the prediction data can only be performed using the method of "retaining the true value with a probability of e^ε / (e^ε+|d|-1), and randomizing it to other possible values ​​with a probability of 1 / (e^ε+|d|-1)".

[0077] In this embodiment, a distance-based local differential privacy algorithm is introduced into decision tree prediction. By leveraging the partial order-preserving properties of the distance-based local differential privacy algorithm, the privacy and accuracy of decision tree prediction are balanced. Since the entire prediction process only performs distance-based local differential privacy processing on the input data, and the data type and format remain unchanged after processing, existing decision tree algorithm libraries can be directly called to complete the training. At the same time, the method of adding noise based on the distance-based local differential privacy algorithm is simple, easy to understand, and easy to distribute, and can support both horizontal and vertical federated prediction. This effectively solves the technical problem in related technologies that federated learning requires significant modifications to the decision tree algorithm and the introduction of complex encryption techniques, resulting in high costs and low security.

[0078] Example 3

[0079] According to an embodiment of this application, another data processing method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0080] The data processing method provided in this application embodiment can also be run in... Figure 1 The flowchart of the operating environment shown is as follows: Figure 4 As shown, the method includes steps S402-S406, wherein:

[0081] Step S402: Obtain multiple first data subsets and multiple fifth data subsets from multiple federated learning participants, wherein each federated learning participant provides one first data subset and one fifth data subset.

[0082] In some optional embodiments of this application, it is first necessary to obtain training data and prediction data provided by multiple federated learning participants. The training data provided by each participant is taken as a first data subset, and the prediction data is taken as a fifth data subset. The first data subset includes feature data and label data used to train the model, and the fifth data subset includes feature data used for prediction.

[0083] Step S404: Process multiple first data subsets using a distance-based local differential privacy algorithm to obtain multiple second data subsets, wherein the distance is the distance between data in the first data subsets.

[0084] Step S406: Process multiple fifth data subsets using a distance-based local differential privacy algorithm to obtain multiple sixth data subsets, wherein the distance is the distance between data in the fifth data subsets.

[0085] The two data processing procedures described above are similar. First, a privacy budget ε that satisfies the distance-based local differential privacy algorithm is determined. The size of ε is adjusted to make a trade-off between the order of data and the strength of privacy protection. Then, noise with parameter ε is added to the data in the first and fifth data subsets. The noise type includes at least one of the following: Laplace distribution noise, geometric distribution noise, and step distribution noise.

[0086] In some optional embodiments of this application, if a column contains a large number of data with the same value, the data in that column can be discretized first according to a preset unit distance, and then noise that satisfies the distance-based local differential privacy algorithm can be added to the discretized data. After that, the data with added noise can be dediscretized according to the preset unit distance.

[0087] If a data point is an enumeration type, and it enumerates all possible values ​​|d|, then when processing this enumeration type of data, the true value of the data is retained with a first probability e^ε / (e^ε+|d|-1), and the data is transformed into other possible values ​​with a second probability 1 / (e^ε+|d|-1). Optionally, when processing this enumeration type of data, it can also be one-hot encoded, and then the one-hot encoded data can be processed using a local differential privacy algorithm in related technologies to obtain multiple sets of data. These multiple sets of data are then added to the training data or prediction data.

[0088] Step S408: Combine multiple second data subsets into a training data set, and input the training data set into the decision tree learning model for training to obtain the target decision tree.

[0089] Step S410: Combine multiple sixth data subsets into a prediction data set, and input the prediction data set into the target decision tree to obtain the prediction result.

[0090] If performing vertical federated learning, when combining multiple second data subsets into a training dataset or multiple sixth data subsets into a prediction dataset, it is necessary to first determine the identifier information of each data in the multiple second data subsets or sixth data subsets, such as user ID, mobile phone number, etc.; then, based on the identifier information of each data, a method such as privacy set intersection is used to perform secure sample alignment processing on the multiple second data subsets or sixth data subsets.

[0091] The training dataset is input into the decision tree learning model, with the feature data as the algorithm input and the label data as the algorithm output. This process is used to train the decision tree learning model and obtain the target decision tree. The prediction dataset is then input into the target decision tree for prediction. The target decision tree predicts the corresponding label data based on the input feature data.

[0092] Figure 5 This paper illustrates a complete decision tree federated learning process. The left and right sides represent two participants in the federated learning process. F1, ..., Fm represent the feature data held by the left participant, F1', ..., Fn' represent the feature data held by the right participant, and L represents the label data held by the right participant. The overall process is divided into a training phase and a prediction phase. Figure 6 The upper half of the dotted line represents the training phase, and the lower half represents the prediction phase. Specifically:

[0093] Training Phase: 1) The left / right participants discretize the training data (first data subset); 2) The left / right participants call the distance-based local differential privacy algorithm in the algorithm library to perform distance-based local differential privacy processing on the training data (to obtain the second data subset); 3) The left participant sends the processed training data to the right participant through the distributed dataset in the communication interface buffer; 4) The right participant merges the training data (training data set) and directly uses the merged data to call the machine learning algorithm to train the decision tree learning model to obtain the target decision tree.

[0094] Prediction Phase: 1) The left / right participants discretize the prediction data (the fifth data subset); 2) The left / right participants call the distance-based local differential privacy algorithm in the algorithm library to perform distance-based local differential privacy processing on the prediction data (to obtain the sixth data subset); 3) The left participant sends the processed prediction data to the right participant through the distributed dataset in the communication interface buffer; 4) The right participant merges the prediction data (prediction data set) and directly calls the target decision tree to obtain the predicted label data.

[0095] In this embodiment, a distance-based local differential privacy algorithm is introduced into decision tree training. Leveraging the partial order-preserving properties of this algorithm, the privacy and accuracy of decision tree training are balanced. Since the entire training process only applies distance-based local differential privacy processing to the input data, and the data type and format remain unchanged after processing, existing decision tree algorithm libraries can be directly called to complete the training. Furthermore, the method of adding noise based on the distance-based local differential privacy algorithm is simple, easy to understand, and readily applicable to distributed processing. It can simultaneously support horizontal / vertical federated training and horizontal / vertical federated prediction. This application effectively solves the technical problems of related technologies that require significant modifications to the decision tree algorithm and the introduction of complex encryption techniques for federated learning, resulting in high costs, low security, and the inability to achieve joint prediction while only enabling joint training.

[0096] Example 4

[0097] According to embodiments of this application, a data processing apparatus for implementing the above-described data processing method is also provided, such as... Figure 6 As shown, the device includes at least an acquisition module 60, a processing module 62, and a training module 64, wherein:

[0098] The acquisition module 60 is used to acquire multiple first data subsets from multiple federated learning participants, wherein each federated learning participant provides one first data subset.

[0099] The processing module 62 is used to process multiple first data subsets using a distance-based local differential privacy algorithm to obtain multiple second data subsets, wherein the distance is the distance between data in the first data subsets.

[0100] Training module 64 is used to combine multiple second data subsets into a training data set, and input the training data set into the decision tree learning model for training to obtain the target decision tree.

[0101] Optionally, the data processing device further includes a second acquisition module 66, a second processing module 68, and a training module 610, wherein:

[0102] The second acquisition module 66 is used to acquire multiple fifth data subsets from multiple federated learning participants, wherein each federated learning participant provides a fifth data subset, and the fifth data subset includes feature data.

[0103] The second processing module 68 is used to process multiple fifth data subsets using a distance-based local differential privacy algorithm to obtain multiple sixth data subsets, wherein the distance is the distance between data in the fifth data subset.

[0104] The prediction module 610 is used to combine multiple sixth data subsets into a prediction data set, and input the prediction data set into the target decision tree for prediction to obtain label data corresponding to the feature data.

[0105] It should be noted that the modules in the data processing device in this embodiment correspond to the implementation steps of the data processing methods in embodiments 1, 2, and 3. Since they have been described in detail before, some details not shown in this embodiment can be referred to the above embodiments, and will not be repeated here.

[0106] Example 5

[0107] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored program, wherein the program controls the device where the non-volatile storage medium is located to execute the above-described data processing method during runtime.

[0108] Optionally, during program execution, the device containing the non-volatile storage medium is controlled to perform the following steps: obtaining multiple first data subsets from multiple federated learning participants, wherein each federated learning participant provides a first data subset; processing the multiple first data subsets using a distance-based local differential privacy algorithm to obtain multiple second data subsets, wherein the distance is the distance between data in the first data subsets; combining the multiple second data subsets into a training data set, and inputting the training data set into a decision tree learning model for training to obtain a target decision tree.

[0109] Optionally, during program execution, the device containing the non-volatile storage medium is controlled to perform the following steps: obtaining multiple fifth data subsets from multiple federated learning participants, wherein each federated learning participant provides a fifth data subset, and the fifth data subset includes feature data; processing the multiple fifth data subsets using a distance-based local differential privacy algorithm to obtain multiple sixth data subsets, wherein the distance is the distance between data in the fifth data subsets; combining the multiple sixth data subsets into a prediction data set, and inputting the prediction data set into the target decision tree for prediction to obtain label data corresponding to the feature data.

[0110] Optionally, during program execution, the device containing the non-volatile storage medium is controlled to perform the following steps: acquiring multiple first data subsets and multiple fifth data subsets from multiple federated learning participants, wherein each federated learning participant provides one first data subset and one fifth data subset; processing the multiple first data subsets using a distance-based local differential privacy algorithm to obtain multiple second data subsets, wherein the distance is the distance between data in the first data subsets; processing the multiple fifth data subsets using a distance-based local differential privacy algorithm to obtain multiple sixth data subsets, wherein the distance is the distance between data in the fifth data subsets; combining the multiple second data subsets into a training data set, and inputting the training data set into a decision tree learning model for training to obtain a target decision tree; combining the multiple sixth data subsets into a prediction data set, and inputting the prediction data set into the target decision tree to obtain the prediction result.

[0111] Example 6

[0112] According to an embodiment of this application, an electronic device is also provided, comprising a processor and a memory, wherein: the memory is connected to the processor and is used to provide the processor with instructions to perform the following processing steps: acquiring multiple first data subsets from multiple federated learning participants, wherein each federated learning participant provides a first data subset; processing the multiple first data subsets using a distance-based local differential privacy algorithm to obtain multiple second data subsets, wherein the distance is the distance between data in the first data subsets; forming a training data set from the multiple second data subsets, and inputting the training data set into a decision tree learning model for training to obtain a target decision tree.

[0113] Optionally, the memory also stores instructions for processing the following steps: obtaining multiple fifth data subsets from multiple federated learning participants, wherein each federated learning participant provides a fifth data subset, the fifth data subset including feature data; processing the multiple fifth data subsets using a distance-based local differential privacy algorithm to obtain multiple sixth data subsets, wherein the distance is the distance between data in the fifth data subsets; combining the multiple sixth data subsets into a prediction data set, and inputting the prediction data set into a target decision tree for prediction to obtain label data corresponding to the feature data.

[0114] Optionally, the memory also stores instructions for processing the following steps: obtaining multiple first data subsets and multiple fifth data subsets from multiple federated learning participants, wherein each federated learning participant provides one first data subset and one fifth data subset; processing the multiple first data subsets using a distance-based local differential privacy algorithm to obtain multiple second data subsets, wherein the distance is the distance between data in the first data subsets; processing the multiple fifth data subsets using a distance-based local differential privacy algorithm to obtain multiple sixth data subsets, wherein the distance is the distance between data in the fifth data subsets; combining the multiple second data subsets into a training data set, and inputting the training data set into a decision tree learning model for training to obtain a target decision tree; combining the multiple sixth data subsets into a prediction data set, and inputting the prediction data set into the target decision tree to obtain a prediction result.

[0115] In some optional embodiments of this application, the specific structure of the above-described electronic device can be referred to as follows: Figure 1 The computer terminal shown. It should be noted that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. The structure of this electronic device can be similar to... Figure 1 The computer terminals shown are the same, but may also include those shown. Figure 1 The more or fewer components shown may also have the same Figure 1 The different configurations shown.

[0116] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0117] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0119] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0122] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data processing method comprising: obtaining a plurality of first data subsets from a plurality of federated learning participants, wherein each federated learning participant provides one of the first data subsets; processing the plurality of first data subsets with a distance-based local differential privacy algorithm to obtain a plurality of second data subsets, wherein the distance is a distance between data in the first data subset, the distance-based local differential privacy algorithm has a partial order preserving property, and the closer the distance between two data, the more difficult it is to distinguish two results obtained by performing differential privacy processing on the two data respectively; combining the plurality of second data subsets into a training data set, and inputting the training data set into a decision tree learning model for training to obtain a target decision tree.

2. The method of claim 1, wherein, processing the plurality of first data subsets with a distance-based local differential privacy algorithm comprises: calculating a distance between data in any of the first data subsets; determining the distance-based local differential privacy algorithm according to the distance and the local differential privacy algorithm; determining noise data according to the distance-based local differential privacy algorithm, and adding the noise data to the data in the first data subset.

3. The method of claim 2, wherein, determining noise data according to the distance-based local differential privacy algorithm, and adding the noise data to the data in the first data subset comprises: determining a privacy budget satisfying the distance-based local differential privacy algorithm, wherein the privacy budget is proportional to an accuracy of a data result output by the distance-based local differential privacy algorithm, and is inversely proportional to a privacy protection level of the data result output; adding noise with a parameter of the privacy budget to the data in the first data subset, wherein the noise type at least includes one of the following: Laplace distribution noise, geometric distribution noise, and step distribution noise.

4. The method of claim 2, wherein, determining noise data according to the distance-based local differential privacy algorithm, and adding the noise data to the data in the first data subset comprises: performing equal-width discretization processing on the data in the first data subset according to a preset unit distance to obtain a third data subset; determining noise data according to the distance-based local differential privacy algorithm, and adding the noise data to the data in the third data subset to obtain a fourth data subset; performing de-discretization processing on the data in the fourth data subset according to the preset unit distance to obtain the second data subset.

5. The method of claim 1, wherein combining the plurality of second data subsets into a training data set comprises: determining identification information of each data in the plurality of second data subsets, the identification information being used to identify a sample object from which the data originates; performing sample alignment processing on the plurality of second data subsets according to the identification information of each data to obtain the training data set, wherein the sample alignment processing comprises a privacy set intersection.

6. The method of claim 1, wherein, processing the plurality of first data subsets with a distance-based local differential privacy algorithm comprises: For any of the first data subsets, when data in the first data subset is of an enumeration type, processing the data of the enumeration type with a local differential privacy algorithm.

7. The method of claim 6, wherein, Processing the data in the first data subset with a local differential privacy algorithm includes: For the data of the enumeration type, retaining a true value of the data with a first probability and converting the data to other values with a second probability.

8. The method of claim 7, wherein, The method further includes: One-hot encoding the data of the enumeration type, and then processing the one-hot encoded data with a local differential privacy algorithm to obtain multiple groups of data, and adding the obtained multiple groups of data to the second data subset.

9. The method of any of claims 1-8, wherein: Any of the first data subsets includes feature data, and at least one of the first data subsets in the plurality of first data subsets includes label data, wherein the feature data is input to the decision tree learning model and the label data is output from the decision tree learning model.

10. A data processing method, comprising: obtaining a plurality of fifth data subsets from a plurality of federated learning participants, wherein each federated learning participant provides one of the fifth data subsets, and the fifth data subset includes feature data; processing the plurality of fifth data subsets with a distance-based local differential privacy algorithm to obtain a plurality of sixth data subsets, wherein the distance is a distance between data in the fifth data subset, and the distance-based local differential privacy algorithm has a partial order preserving property, and the closer the distance between two data, the more difficult it is to distinguish two results obtained by respectively processing the two data with differential privacy; combining the plurality of sixth data subsets into a prediction data set, and inputting the prediction data set into a target decision tree for prediction to obtain label data corresponding to the feature data.

11. The method of claim 10, wherein, The method further includes: When using the target decision tree for classification, processing the label data with a local differential privacy algorithm, retaining a true value of the label data with a first probability, and converting the label data to other values with a second probability.

12. A data processing method, comprising: obtaining a plurality of first data subsets and a plurality of fifth data subsets from a plurality of federated learning participants, wherein each federated learning participant provides one of the first data subsets and one of the fifth data subsets; processing the plurality of first data subsets with a distance-based local differential privacy algorithm to obtain a plurality of second data subsets, wherein the distance is a distance between data in the first data subset, and the distance-based local differential privacy algorithm has a partial order preserving property, and the closer the distance between two data, the more difficult it is to distinguish two results obtained by respectively processing the two data with differential privacy; processing the plurality of fifth data subsets with a distance-based local differential privacy algorithm to obtain a plurality of sixth data subsets, wherein the distance is a distance between data in the fifth data subset. The multiple second data subsets are grouped into a training data set, and the training data set is input into a decision tree learning model for training to obtain a target decision tree; The multiple sixth data subsets are grouped into a prediction data set, and the prediction data set is input into the target decision tree to obtain a prediction result.

Images