Data processing method and device, equipment and storage medium

Abstract

The application discloses a data processing method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: a first module receives gradient serialization data sent by a second device; the first module sends the gradient serialization data to a second module; the second module respectively constructs N first feature ciphertext histograms based on the gradient serialization data and N first features corresponding to at least one training sample, and obtains a ciphertext histogram object; the second module sends first information to the first module; the first module acquires the ciphertext histogram object from a storage location of the ciphertext histogram object in the first device according to the first information; and the first module sends the ciphertext histogram object to the second device. The method improves the processing efficiency of a device in training a decision tree based on federated learning.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, device, and storage medium. Background Technology

[0002] With the development of computer technology, a large amount of interconnected data has been generated across different fields and platforms. Mining this data can yield valuable information, providing crucial support for decision support, business optimization, and scientific research. Among related technologies, federated learning allows multiple data stakeholders to collaboratively mine and utilize the value of their data without sharing the original data.

[0003] In this technology, the business side and multiple data providers train multiple decision trees using the GBDT (Gradient Boosting Decision Tree) algorithm based on federated learning. For any decision tree, the business side calls a C++ client to homomorphically encrypt the first and second gradients of the training samples, obtaining gradient ciphertext. The C++ client serializes the gradient ciphertext and sends it to the business side's Java client, which then sends the serialized gradient ciphertext to multiple data providers. Each data provider's Java client sends its serialized gradient ciphertext to the C++ client, which deserializes the serialized gradient ciphertext and constructs a ciphertext histogram based on it. The C++ client sends the ciphertext histogram back to the Java client, which then sends it back to the business side. Finally, the business side trains the decision tree based on the ciphertext histograms from multiple data providers.

[0004] In the process of jointly training any decision tree by the business side and the data side, multiple serialization, deserialization, and ciphertext transmissions are required. The frequent construction of ciphertext histograms will generate a lot of additional overhead, affecting the processing efficiency of the device in training decision trees based on federated learning. Summary of the Invention

[0005] This application provides a data processing method, apparatus, device, and storage medium. The technical solutions provided by this application are as follows:

[0006] According to one aspect of the embodiments of this application, a data processing method is provided, the method being executed by a first device, the first device including a first module and a second module, the first module and the second module being implemented based on different programming languages, the method comprising:

[0007] The first module receives gradient serialized data sent by the second device. The gradient serialized data is a serialized gradient ciphertext object, which is homomorphically encrypted gradient data. The gradient data includes sample gradients corresponding to at least one training sample.

[0008] The first module sends the gradient serialized data to the second module;

[0009] The second module constructs a ciphertext histogram for each of the N first features based on the gradient serialization data and the at least one training sample, resulting in a ciphertext histogram object. The ciphertext histogram object includes the ciphertext histograms for each of the N first features. The ciphertext histogram of the first feature is used to indicate the distribution of the at least one training sample in at least one bin of the first feature, where N is a positive integer.

[0010] The second module sends the first information to the first module, and the first information is used to determine the storage location of the encrypted histogram object in the first device;

[0011] The first module obtains the ciphertext histogram object from its storage location in the first device based on the first information.

[0012] The first module sends the encrypted histogram object to the second device.

[0013] According to one aspect of the embodiments of this application, a data processing method is provided, the method being executed by a second device, the second device including a first module and a second module, the first module and the second module being implemented based on different programming languages, the method comprising:

[0014] The first module sends gradient data to the second module, and the gradient data includes sample gradients corresponding to at least one training sample.

[0015] The second module performs homomorphic encryption on the gradient data to obtain a gradient ciphertext object;

[0016] The second module performs a serialization operation on the gradient ciphertext object to obtain gradient serialized data, which is the serialized gradient ciphertext object.

[0017] The second module sends the second information to the first module, and the second information is used to determine the storage location of the gradient serialization data in the second device;

[0018] The first module obtains the gradient serialization data from the storage location of the gradient serialization data in the second device based on the second information;

[0019] The first module sends the gradient serialized data to at least one first device.

[0020] According to one aspect of the embodiments of this application, a data processing apparatus is provided, the apparatus including a first module and a second module, wherein the first module and the second module are implemented based on different programming languages;

[0021] The first module is used to receive gradient serialization data sent by the second device. The gradient serialization data is a serialized gradient ciphertext object, which is homomorphically encrypted gradient data. The gradient data includes sample gradients corresponding to at least one training sample.

[0022] The first module is further configured to send the gradient serialized data to the second module;

[0023] The second module is used to construct a ciphertext histogram for each of the N first features based on the gradient serialization data and the N first features corresponding to the at least one training sample, and obtain a ciphertext histogram object. The ciphertext histogram object includes the ciphertext histograms for each of the N first features. The ciphertext histogram of the first feature is used to indicate the distribution of the at least one training sample in at least one bin of the first feature, where N is a positive integer.

[0024] The second module is further configured to send first information to the first module, wherein the first information is used to determine the storage location of the encrypted histogram object in the first device;

[0025] The first module is further configured to obtain the ciphertext histogram object from the storage location of the ciphertext histogram object in the first device based on the first information;

[0026] The first module is further configured to send the encrypted histogram object to the second device.

[0027] According to one aspect of the present invention, a data processing apparatus is provided, the apparatus comprising a first module and a second module, the first module and the second module being implemented based on different programming languages;

[0028] The first module is used to send gradient data to the second module, wherein the gradient data includes sample gradients corresponding to at least one training sample;

[0029] The second module is used to perform homomorphic encryption on the gradient data to obtain a gradient ciphertext object;

[0030] The second module is further configured to perform a serialization operation on the gradient ciphertext object to obtain gradient serialized data, wherein the gradient serialized data is the serialized gradient ciphertext object;

[0031] The second module is further configured to send second information to the first module, wherein the second information is used to determine the storage location of the gradient serialization data in the second device;

[0032] The first module is further configured to obtain the gradient serialization data from the storage location of the gradient serialization data in the second device based on the second information;

[0033] The first module is also configured to send the gradient serialized data to at least one first device.

[0034] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described data processing method.

[0035] According to one aspect of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the storage medium, the computer program being loaded and executed by a processor to implement the above-described data processing method.

[0036] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program, the computer program being loaded and executed by a processor to implement the above-described data processing method.

[0037] The technical solutions provided in this application have at least the following beneficial effects:

[0038] During the construction of the encrypted histogram, modules implemented in different programming languages ​​call each other. The first information is transmitted between different modules, indicating the storage location of the constructed encrypted histogram. Different modules can obtain the constructed encrypted histogram based on the first information, which reduces the amount of processing required for the device to transmit the encrypted histogram, thereby effectively improving the processing efficiency of the device in training decision trees based on federated learning. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of a computer system provided in one embodiment of this application;

[0040] Figure 2 This is an interactive flowchart of a data processing method provided in one embodiment of this application;

[0041] Figure 3 This is a flowchart of a data processing method provided in one embodiment of this application;

[0042] Figure 4 This is a flowchart of constructing a ciphertext histogram provided in one embodiment of this application;

[0043] Figure 5 This is a schematic diagram of concurrent gradient accumulation calculation based on a single embodiment of this application, which involves segmenting and then summarizing the results.

[0044] Figure 6 This is a schematic diagram of a target decision tree provided in one embodiment of this application;

[0045] Figure 7 This is a flowchart of a data processing method provided in another embodiment of this application;

[0046] Figure 8 This is a flowchart of determining the optimal split point provided in one embodiment of this application;

[0047] Figure 9 This is a bar chart showing the average training time of a decision tree provided in one embodiment of this application;

[0048] Figure 10 This is a block diagram of a data processing apparatus provided in one embodiment of this application;

[0049] Figure 11 This is a block diagram of a data processing apparatus provided in another embodiment of this application;

[0050] Figure 12 This is a structural block diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0052] Please refer to Figure 1 The diagram illustrates a computer system provided in one embodiment of this application. The computer system may include at least one first device 10 and a second device 20. The number of second devices 20 may be one.

[0053] In this application, the second device 20 and at least one first device 10 jointly train multiple decision trees using a federated machine learning algorithm, enabling the trained decision trees to function as a federated learning model capable of performing prediction tasks. A federated learning model is an ensemble model obtained by integrating multiple decision trees. The second device 20 is the participant that leads the training of the multiple decision trees and possesses the label information of each training sample for the prediction task. In some embodiments, the prediction task is related to the business needs of the second device 20. The first device 10 is the participant that provides additional feature data to the training samples and possesses features not available to the second device 20. In some embodiments, different first devices 10 possess different features.

[0054] Vertical federated learning is a common federated learning model in cross-enterprise collaborations. Taking two parties as an example, only one party possesses the machine learning label information, referred to as the Guest party, while the party without label information is called the Host party. Through vertical federated learning, the Guest party can leverage the feature data of the Host party to improve the capabilities of the machine learning model, while simultaneously protecting the data privacy of all participating parties. In this application, the first device 10 corresponds to the Host party, and the second device 20 corresponds to the Guest party.

[0055] The first device 10 is an electronic device with data computing, processing, and storage functions. The first device 10 can be either a terminal device or a server. The first device 10 can include, but is not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, game consoles, wearable devices, multimedia playback devices, augmented reality (AR) devices, virtual reality (VR) devices, cloud technology platforms, intelligent robots, intelligent transportation terminal systems, and driving control systems, among other electronic devices.

[0056] The second device 20 is an electronic device with data computing, processing, and storage functions. The second device 20 can be either a terminal device or a server. The second device 20 can include, but is not limited to, electronic devices such as mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, game consoles, wearable devices, multimedia playback devices, augmented reality devices, virtual reality devices, cloud technology platforms, intelligent robots, intelligent transportation terminal systems, and driving control systems.

[0057] In some embodiments, the solutions provided in this application can be applied to at least one of the following scenarios: e-commerce operations, medical and health research, urban traffic management, financial risk control, credit assessment, advertising recommendation, financial marketing, joint data analysis, etc., and can also be applied to other scenarios, which are not limited in this application embodiment.

[0058] For example, the server mentioned above can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, but it is not limited to these.

[0059] In the federated GBDT algorithm, multiple devices jointly train multiple decision trees, which then serve as a federated learning model. The decision trees are constructed sequentially, with each tree relying on pre-trained trees for training. The training process for any single decision tree is as follows:

[0060] 1. Gradient Encryption: The second device 20 performs gradient calculation based on the current predicted value and label information of at least one training sample to obtain gradient data of at least one training sample, and then encrypts the gradient data and sends it to at least one first device 10.

[0061] The current predicted value of a training sample refers to the predicted value obtained based on the already trained decision tree. The label information refers to the true value of the training sample for the prediction task.

[0062] 2. Constructing gradient histograms: For each tree node in the decision tree, the second device 20 and at least one first device 10 construct gradient histograms based on their own feature data, and at least one first device 10 sends the constructed gradient histograms to the second device 20.

[0063] 3. Finding the optimal split point: The second device 20 decrypts the gradient histogram of at least one of the first devices and, based on the split gain calculation formula, finds the globally optimal split point of the decision tree. If the optimal split point belongs to any of the first devices 10, the second device 20 needs to return the split information to that first device 10 for parsing. The split information indicates the location of the optimal split point.

[0064] 4. Split Tree Node: The first device 10 or the second device 20 with the optimal split point splits the sample subset on the leaf node of the current decision tree and sends the split result to the other party to update the index between the sample and the tree node.

[0065] 5. Update the predicted value: The second device 20 calculates the predicted weight value of the leaf node according to the weight calculation formula and updates the sample predicted value; at least one first device 10 cannot know the weight of the leaf node of the decision tree.

[0066] This application proposes an optimization method for constructing gradient histograms or ciphertext histograms, aiming to reduce the transmission overhead during the construction process.

[0067] Please refer to Figure 2 This illustrates an interactive flowchart of a data processing method provided in one embodiment of this application. The following is in conjunction with... Figure 2 Taking a single first device 10 as an example, the interaction process of constructing a ciphertext histogram between the first device 10 and the second device 20 in this embodiment of the application is briefly described as follows:

[0068] S1. The first module of the second device 20 transmits the gradient data of at least one training sample in plaintext to the second module of the second device 20 via JNI (Java Native Interface) call;

[0069] The gradient data for each training sample includes the first-order gradient and the second-order gradient of that training sample.

[0070] S2, the second module of the second device 20 encrypts the plaintext of the gradient data of at least one training sample into ciphertext to obtain a gradient ciphertext object;

[0071] S3. The second module of the second device 20 serializes the gradient ciphertext object through a serialization framework to obtain the serialized gradient ciphertext object.

[0072] S4. The second module of the second device 20 passes the serialized gradient ciphertext object to the first module of the second device 20.

[0073] In some embodiments, the second module of the second device 20 sends second information to the first module of the second device 20, the second information being used to indicate the storage location of the gradient ciphertext object in the second device 20.

[0074] In some embodiments, the first module of the second device 20 is implemented using the Java language, and the second module of the second device 20 is implemented using the C++ language.

[0075] In some embodiments, the first module and the second module in the second device 20 call each other via JNI.

[0076] S5. The first module of the second device 20 transmits the serialized gradient ciphertext object to the remote first device 10 via message communication.

[0077] In some embodiments, the gradient ciphertext object includes gradient data of a subset of samples corresponding to at least one leaf node of the decision tree.

[0078] S6. After receiving the serialized gradient ciphertext object, the first module of the first device 10 transmits the serialized gradient ciphertext object to the second module of the first device 10 through a JNI call.

[0079] S7. The second module of the first device 10 then deserializes the serialized gradient ciphertext object back into a gradient ciphertext object.

[0080] S8. The second module of the first device 10 performs the construction of a ciphertext histogram on the gradient ciphertext object to obtain a ciphertext histogram object.

[0081] S9. The second module of the first device 10 serializes the constructed ciphertext histogram object to obtain a serialized ciphertext histogram object; the second module of the first device 10 stores the serialized ciphertext histogram object in the storage location indicated by the first information.

[0082] S10, the second module of the first device 10 transmits the first information to the first module of the first device 10. The first information is used to indicate the storage location of the encrypted histogram in the first device 10.

[0083] S11, The first module of the first device 10 transmits the serialized ciphertext histogram object to the remote second device 20 via message communication;

[0084] S12. After receiving the serialized ciphertext histogram object, the first module of the second device 20 transmits the serialized ciphertext histogram object to the second module of the second device 20 through a JNI call.

[0085] S13, the second module of the second device 20 then deserializes the serialized ciphertext histogram object back into a ciphertext histogram object;

[0086] S14, The second module of the second device 20 performs decryption of the ciphertext histogram object on the ciphertext histogram object;

[0087] S15, the second module of the second device 20 passes the decrypted ciphertext histogram object to the first module of the second device 20.

[0088] Please refer to Figure 3 The diagram illustrates a flowchart of a data processing method provided in one embodiment of this application. The execution entity for each step of the method can be a computer device; for example, the computer device can be... Figure 1 The first device 10 in the illustrated implementation environment. The method may include at least one of the following steps (310-360):

[0089] Step 310: The first module receives gradient serialization data sent by the second device. The gradient serialization data is a serialized gradient ciphertext object, which is homomorphically encrypted gradient data. The gradient data includes the sample gradient corresponding to at least one training sample.

[0090] In some embodiments, the first device includes a first module and a second module, which are implemented using different programming languages.

[0091] Federated learning is a distributed machine learning technique whose core objective is to achieve collaborative modeling of data from multiple parties while protecting data privacy and security. In some embodiments, federated learning includes horizontal federated learning and vertical federated learning. It is important to note that this application's embodiment applies to vertical federated learning and describes the process of training multiple decision trees based on vertical federated learning and GBDT. This process is simply referred to as the federated GBDT training process.

[0092] GBDT is a popular machine learning algorithm, and one typical implementation, XGBoost (eXtreme Gradient Boosting), is a highly optimized distributed machine learning algorithm. Since its outstanding performance in various competitions, it has gradually become a hot topic in academic research, and a number of more efficient implementations have emerged, such as LightGBM (Light Gradient Boosting Machine) and CatBoost (Categorical Boosting). At the same time, these implementations have wide applications in various problems such as classification, regression, and ranking.

[0093] The first device and the second device are different participants in the federated learning process. In some embodiments, the participants in the federated learning process include at least one of the following: business parties, data providers, and third-party service providers, and may also include other participants, which are not limited in this application embodiment. In some embodiments, the number of first devices is at least one, and the number of second devices is one.

[0094] The business party, also known as the Guest party, refers to the role in federated learning that proposes business requirements for the federated learning model and applies the model. In this application, the business party corresponds to the second device. In some embodiments, the business party provides label information and partial feature data for the training samples. The label information of the training samples is used to indicate the true values ​​of the training samples in response to the business requirements.

[0095] The data provider, also known as the host, refers to a party that possesses some feature data but lacks label information or complete training samples. Feature data refers to the data of the training samples on the corresponding features. Features refer to the attributes of the training samples. In this application, the data provider corresponds to the first device.

[0096] The third-party service provider acts as the technical coordinator in federated learning, responsible for ensuring the privacy, security, and efficiency of the training process. In some embodiments, the services provided by the third-party service provider may include at least one of the following: computing services, encryption services, storage services, etc., and may also include other services, which are not limited in this application embodiment.

[0097] During the training of Federated GBDT, the sample gradient of the training sample is used to measure the error of the current federated learning model on that training sample and to guide the training direction of subsequent decision trees. In some embodiments, the current federated learning module includes a pre-trained decision tree. In some embodiments, the sample error of the training sample is determined based on the label information of the training sample and the prediction value of the training sample on the current federated learning model. A decision tree is a weak learner based on a tree structure. In some embodiments, the decision tree is CART (Classification and Regression Tree).

[0098] Gradient data refers to the set of sample gradients corresponding to at least one training sample. Gradient ciphertext objects are ciphertext objects obtained by encrypting gradient data using a homomorphic encryption algorithm.

[0099] Homomorphic encryption algorithms refer to algorithms that allow direct computational operations on encrypted data without prior decryption. In some embodiments, homomorphic encryption algorithms include partially homomorphic encryption (PHE) and fully homomorphic encryption (FHE). A partially homomorphic algorithm is one that encrypts data using only addition or multiplication operations. In some embodiments, partially homomorphic algorithms include at least one of the following: Paillier (a partially homomorphic encryption algorithm proposed by Paillier), ElGamal (a partially homomorphic encryption algorithm proposed by ElGamal), OU (a partially homomorphic encryption algorithm proposed by Okamoto-Uchiyama), etc., and may also include other partially homomorphic algorithms; this application does not limit the specific examples. A fully homomorphic algorithm is one that encrypts data using both addition and multiplication operations. In some embodiments, the fully homomorphic encryption algorithm includes at least one of the following: BGV (a fully homomorphic encryption algorithm proposed by Brakerski, Gentry, and Vaikuntanathan), BFV (a fully homomorphic encryption algorithm proposed by Brakerski and Fan-Vercauteren respectively), and CKKS (a fully homomorphic encryption algorithm proposed by Cheon-Kim-Kim-Song), etc., and may also include other fully homomorphic encryption algorithms, which are not limited in this application embodiment.

[0100] Gradient serialized data is serialized data obtained by performing a serialization operation on a gradient ciphertext object. Serialization is the process of converting an object or data structure into a storable or transmittable byte stream or other standardized format. In some embodiments, serialization is implemented in any of the following ways: JSON (JavaScript Object Notation), Protocol Buffers (Protobuf), MessagePack (MsgPack), FlatBuffers, Java Serializable (Java Serializable Interface), etc., and may also include other implementation methods, which are not limited in this application embodiment.

[0101] At least one training sample is used to train and generate at least one decision tree. In some embodiments, prior to federated learning, a data alignment operation is performed to obtain at least one training data sample. The data alignment operation is used to determine the common data portion of the second device and at least one first device. In some embodiments, each training sample includes label information and feature data corresponding to at least one different feature. In some embodiments, the label information in each training sample originates from the second device, and the feature data of the at least one different feature corresponding to each training sample originates from either the second device or at least one first device. In some embodiments, the data alignment operation includes feature alignment and sample alignment. Feature alignment refers to data alignment between the second device and at least one first device along the feature dimension, applied to horizontal federated learning. Sample alignment refers to data alignment between the second device and at least one first device along the training sample dimension, applied to vertical federated learning. In some embodiments, sample alignment can be implemented as PrivateSet Intersection (PSI). PrivateSet Intersection is used by participating parties to calculate the intersection between the data of multiple participating parties without obtaining additional information from each other (information other than the intersection).

[0102] The first module and the second module are two different functional modules in the first device used to implement the functions of the data provider. In some embodiments, the first module and the second module are used to implement different functions. In some embodiments, the computational efficiency of the second module is higher than that of the first module.

[0103] In some embodiments, the first module is used to implement business logic. In some embodiments, the first module is used to implement at least one of the following: data preprocessing, key management, business interaction, etc., and may also be used to implement other functions, which are not limited in this application embodiment. Data preprocessing refers to the process of initializing data related to training samples. In some embodiments, data preprocessing includes at least one of the following: data alignment, data cleaning, normalization, standardization, data transformation, binning, etc., and may also include other data preprocessing, which are not limited in this application embodiment. Key management refers to generating and managing keys used for encryption. Business interaction refers to the processes of data flow, logical processing, and function calls during the training process of federated learning. In some embodiments, business interaction may include at least one of the following: interaction processes within a device, interaction processes between devices, interaction processes between a device and a service, etc., and may also include other interactions, which are not limited in this application embodiment.

[0104] In some embodiments, the second module is used to implement computationally intensive tasks. In some embodiments, the second module is used to implement at least one of the following: encryption computation, decryption computation, serialization operation, deserialization operation, etc., and may also be used to implement other functions, which are not limited in this application embodiment. Encryption computation refers to the process of encrypting data related to training samples based on an encryption algorithm. Decryption computation refers to the reverse process of encryption computation, the process of decrypting data related to encrypted training samples based on an encryption algorithm. In some embodiments, the encryption algorithm is a homomorphic encryption algorithm.

[0105] In some embodiments, the first module is implemented in Java, and the second module is implemented in C++ or C. The first module and the second module call each other through JNI (Java Native Interface).

[0106] Java Native Interface (JNI) is an interface mechanism used to enable bidirectional interaction between Java code and code implemented in other programming languages. By using Java Native Interface (JNI) to write programs, JNI ensures easy portability across different platforms. JNI allows Java code to call native code (such as native code implemented in C / C++) and also supports native code calling Java methods, forming a bidirectional communication bridge.

[0107] By implementing different functional modules using different programming languages ​​and enabling them to call each other via JNI, the advantages of each programming language can be leveraged to improve the efficiency of federated learning training. The second module is implemented using C++ or C. Since C++ and C are faster in computationally intensive tasks, they can effectively improve the efficiency of federated learning training.

[0108] Step 320: The first module sends the gradient serialization data to the second module.

[0109] In other words, the first module transmits the gradient serialization data to the second module for processing. In some embodiments, the first module sends the complete gradient serialization data to the second module.

[0110] In some embodiments, the first module sends fourth information to the second module, the fourth information being used to indicate the storage location of the gradient serialization data in the first device.

[0111] Step 330: The second module constructs ciphertext histograms for each of the N first features based on the gradient serialization data and at least one training sample, resulting in a ciphertext histogram object. The ciphertext histogram object includes ciphertext histograms for each of the N first features. The ciphertext histograms of the first features are used to indicate the distribution of at least one training sample in at least one bin of the first feature, where N is a positive integer.

[0112] The first feature refers to the feature possessed by the first device. N first features are the N different first features possessed by the first device. Different first features are used to indicate different attributes of the training samples.

[0113] Constructing a gradient histogram refers to the process of building a histogram based on the plaintext gradient of at least one training sample and the feature data of at least one feature corresponding to that training sample. Constructing a ciphertext histogram refers to the process of building a histogram based on the ciphertext gradient of at least one training sample and the feature data of at least one feature corresponding to that training sample.

[0114] For any one of the N first features, the gradient histogram of that first feature is used to analyze the distribution of the sample gradients corresponding to each of the at least one training sample on that first feature. The ciphertext histogram of that first feature is used to analyze the distribution of the encrypted sample gradients corresponding to each of the at least one training sample on that first feature. In some embodiments, for any one of the N first features, homomorphic decryption is performed on the ciphertext histogram of that first feature to obtain the gradient histogram corresponding to that first feature.

[0115] Binning refers to dividing a continuous variable or a high-cardinality discrete variable into discrete intervals. In this application, the continuous variable or high-cardinality discrete variable can be the value range of any feature. In some embodiments, any one of the M bins of a first feature corresponds to a value range of the first feature. In some embodiments, for the aforementioned N first features, the number of bins for different first features can be the same or different. In some embodiments, for any one of the aforementioned N first features, the bins corresponding to that first feature are pre-defined by those skilled in the art, or can be divided based on preset division rules; this application does not limit this. In some embodiments, the division rules can include at least one of the following: equal-frequency binning, equal-interval binning, chi-square binning, etc., and may also include other division rules; this application does not limit this. Equal-frequency binning refers to dividing the data into feature intervals evenly according to frequency. Equal-interval binning divides feature intervals at equal intervals according to numerical ranges. Chi-square binning merges adjacent regions of the first feature based on statistical significance.

[0116] In some embodiments, the second module performs a deserialization operation on the gradient serialized data to obtain a gradient ciphertext object; the second module constructs ciphertext histograms for each of the N first features based on the gradient ciphertext object and at least one training sample, to obtain a ciphertext histogram object.

[0117] Deserialization is the inverse process of serialization, which is the process of restoring byte streams or standardized format data back to the original object or data structure.

[0118] N represents the number of first features possessed by the first device. In some embodiments, different first devices possess different numbers of first features.

[0119] In some embodiments, for any one of the N first features, the second module divides at least one training sample into M bins of the first feature to obtain the sample binning result of the first feature. The sample binning result is used to indicate the training samples corresponding to each of the M bins, where M is an integer greater than 1. Based on the sample binning result of the first feature, the second module calculates the cumulative value corresponding to each of the M bins. The cumulative value corresponding to each bin is the cumulative value of the sample gradient of the training sample corresponding to the bin. Based on the cumulative values ​​corresponding to each of the M bins, the second module constructs a ciphertext histogram to obtain the ciphertext histogram of the first feature. Based on the ciphertext histograms of the N first features, the second module obtains a ciphertext histogram object.

[0120] In some embodiments, the at least one training sample includes feature data corresponding to the first feature; for any one of the at least one training samples, the second module determines the binning corresponding to the training sample based on the feature data corresponding to the first feature in the training sample and M binnings of the first feature; the second module obtains the sample binning result of the first feature based on the binnings corresponding to each of the at least one training sample.

[0121] In some embodiments, for any one of the at least one bins of the first feature, the second module sums the sample gradients of the training samples corresponding to that bin based on the sample binning result of the first feature to obtain the cumulative value corresponding to that bin; the second module then obtains the ciphertext histogram of the first feature based on the cumulative values ​​corresponding to each of the at least one bin of the first feature. It should be noted that during the process of constructing the ciphertext histogram based on the gradient ciphertext object, the sample gradient corresponding to the training sample is the homomorphically encrypted sample gradient.

[0122] By using the encrypted gradient ciphertext object to construct the ciphertext histogram in the above manner, the gradient and label information of the training samples can be effectively protected. While avoiding information leakage, the ciphertext histogram can be constructed effectively.

[0123] In addition, the process of constructing a ciphertext histogram based on gradient ciphertext objects involves ciphertext computation using homomorphic encryption. Since ciphertext computation is a relatively time-consuming process, having the second module, i.e., the second module implemented in C++ or C, perform the ciphertext computation can effectively reduce the time consumed in constructing the ciphertext histogram and effectively improve the training efficiency of federated learning.

[0124] In some embodiments, the construction of gradient histograms or ciphertext histograms may include concurrent construction along the feature dimension and / or concurrent construction along the binning dimension. Concurrent construction along the feature dimension means that gradient histograms or ciphertext histograms can be constructed in parallel for different features. Concurrent construction along the binning dimension means that the computation of each bin within each feature can be performed in parallel.

[0125] However, the distribution of dense features varies greatly across different bins, resulting in an extremely uneven amount of data processed by different threads. This is especially true in time-consuming computational environments like homomorphic encryption, where the advantages of concurrency are not significant. However, parallel processing directly at the bin dimension can lead to thread safety issues.

[0126] Step 340: The second module sends the first information to the first module. The first information is used to determine the storage location of the encrypted histogram object in the first device.

[0127] The first information is the location information of the encrypted histogram object in the first device. In some embodiments, the first information and the fourth information described above indicate the same storage location. That is, the first module and the second module can transmit data through the same storage location. In some embodiments, the first information includes first pointer information of the encrypted histogram object, which is used to indicate the storage location of the encrypted histogram object in the first device.

[0128] The first pointer information is a pointer used to locate the encrypted histogram object. In some embodiments, the type of the first pointer information includes at least one of the following: memory pointer, shared memory pointer, inter-process communication handle, etc., and may also include other types, which are not limited in this application embodiment. A memory pointer is a variable used in a programming language to store memory addresses, through which data in memory can be directly accessed or manipulated. The essence of a memory pointer is the number of a memory unit. A shared memory pointer points to a memory region shared by multiple processes. A shared memory segment is created through a system call and mapped to the process address space. A handle is a resource identifier provided by the operating system. Inter-process communication handles are used to abstractly reference IPC (Inter-Process Communication) objects. An inter-process communication handle is an index or abstract identifier of a kernel object. In some embodiments, the type of IPC handle includes at least one of the following: pipe handle, shared memory identifier, message queue identifier, etc., and may also include other types, which are not limited in this application embodiment.

[0129] For example, the first pointer information is a pointer to a memory-shared defined structure (typedef void*CipherHandle;). typedef void*CipherHandle aliases the generic pointer void* as CipherHandle, which is essentially an untyped pointer. A pointer defined in this way can point to any memory address.

[0130] In some embodiments, the first information may further include at least one of the following: path information, index information, distributed storage address, memory address, URL (Uniform Resource Locator), etc., and may also include other information, which is not limited in this application embodiment. The path information of the encrypted histogram object is used to indicate the storage location of the encrypted histogram object in the storage system of the first device. In some embodiments, the path information includes absolute paths and / or relative paths. The index information is used to indicate the location of the encrypted histogram object in the database or storage system. The distributed storage address is used to indicate the storage location of the encrypted histogram object in the distributed system. The memory address is used to indicate the storage location of the encrypted histogram object in the memory of the first device. The URL is used to indicate the specific location of the encrypted histogram object on the network or locally.

[0131] In this way, the second module does not directly transmit the encrypted histogram object to the first module. Instead, it transmits pointer information indicating the encrypted histogram object to the first module. The first module can directly obtain the encrypted histogram object based on the pointer information, reducing the overhead of transmitting the encrypted histogram object between the first and second modules and effectively improving the training efficiency of federated learning. Step 350: The first module retrieves the encrypted histogram object from its storage location in the first device based on the first information.

[0132] Since both the first module and the second module are implemented in the first device and can access the storage system of the first device, after the first module receives the first information, it can obtain the encrypted histogram object in the storage system of the first device based on the first information.

[0133] In some embodiments, the first module and the second module share a first memory region in the first device, and the ciphertext histogram object is stored in the first memory region.

[0134] The first memory region is a region within the memory of the first device. In some embodiments, the first memory region may be created by a first module or a second module, and this application embodiment does not limit this. In some embodiments, gradient serialization data is also stored in the first memory region; the second module obtains the gradient serialization data from its storage location in the first memory region based on the fourth information.

[0135] In some embodiments, the histogram serialized data is stored at a storage location in a first memory region indicated by the first information.

[0136] In some embodiments, the first module obtains the ciphertext histogram object at a storage location in the first memory region based on the first information.

[0137] In some embodiments, before executing step 340, the second module performs a serialization operation on the ciphertext histogram object to obtain a serialized ciphertext histogram object; the second module stores the serialized ciphertext histogram in the storage location indicated by the first information; and the first module retrieves the serialized ciphertext histogram from the storage location indicated by the first information and sends it to the second device.

[0138] The storage location indicated by the first information stores a serialized ciphertext histogram object, and the ciphertext histogram object obtained from the storage location indicated by the first information in step 350 is a serialized ciphertext histogram object.

[0139] In some embodiments, the first module obtains the serialized ciphertext histogram object at a storage location in the first memory region based on the first information.

[0140] By using shared memory, the first module can directly obtain the serialized ciphertext histogram from the corresponding memory based on the first information. This reduces the transmission and copying of serialized data between the first and second modules, effectively reducing the overhead of additional data transmission and helping to improve the efficiency of federated learning training.

[0141] Step 360: The first module sends the encrypted histogram object to the second device.

[0142] In some embodiments, the first module sends the serialized encrypted histogram object to the second device. In some embodiments, the first module sends the encrypted histogram object to the second device in one or more of the following ways: message queue communication, HTTP (Hypertext Transfer Protocol) communication, Socket communication, gRPC (Google Remote Procedure Calls) communication, etc., and other methods may also be used, which are not limited in this application embodiment.

[0143] In summary, the technical solution provided in this application embodiment allows modules implemented in different programming languages ​​to call each other during the construction of the encrypted histogram. By transmitting first information between different modules, the first information indicates the storage location of the constructed encrypted histogram. Different modules can obtain the constructed encrypted histogram based on this first information, reducing the processing load of the device in transmitting the encrypted histogram, thereby effectively improving the processing efficiency of the device in training decision trees based on federated learning.

[0144] The process of constructing a ciphertext histogram is described below.

[0145] In some embodiments, the sample gradient corresponding to the training sample includes a first-order gradient and a second-order gradient. In some embodiments, the training sample can be any one of the at least one training sample described above.

[0146] During Federated GBDT training, each tree is optimized based on the residuals of the previous tree. In some embodiments, the first and second gradients of a training sample are used to measure the residuals of that training sample in the previous tree. In some embodiments, the first gradient of a training sample is used to measure the error between the current federated learning model's prediction for that training sample and the label information corresponding to that training sample. In some embodiments, the second gradient of a training sample is used to measure the rate of change of the gradient of the training sample. In some embodiments, the second gradient of a training sample is used to adjust the training step size of the Federated GBDT.

[0147] In some embodiments, such as Figure 4 As shown, it illustrates a flowchart of constructing a ciphertext histogram according to an embodiment of this application. The process of constructing the ciphertext histogram includes at least one of the following steps (410-440).

[0148] Step 410: For any one of the N first features, the second module divides at least one training sample into M bins of the first feature to obtain the sample binning result of the first feature. The sample binning result is used to indicate the training samples corresponding to each of the M bins, where M is an integer greater than 1.

[0149] Sample binning results refer to the mapping relationship after at least one training sample is divided into different bins according to the value range of a certain feature (such as the first feature). In other words, sample binning results are used to indicate which training samples are contained in each of the M bins for the aforementioned first feature.

[0150] M represents the number of bins for the first feature. In some embodiments, the number of bins for different first features may be the same or different.

[0151] In some embodiments, the at least one training sample includes feature data corresponding to the first feature; for any one of the at least one training samples, the second module determines the binning corresponding to the training sample based on the feature data corresponding to the first feature in the training sample and M binnings of the first feature; the second module obtains the sample binning result of the first feature based on the binnings corresponding to each of the at least one training sample.

[0152] For example, assuming the value range of the first feature is [0, 100], the value range of the first feature is divided into 5 bins: bin 1, bin 2, bin 3, bin 4, and bin 5. Bin 1 is [0, 20), bin 2 is [20, 40), bin 3 is [40, 60), bin 4 is [60, 80), and bin 5 is [80, 100]. Assuming the feature data of the first feature of the first training sample is 42, then the first training sample belongs to bin 3, and is therefore assigned to bin 3.

[0153] Step 420: Based on the sample binning results of the first feature, the second module allocates M bins to S computing units. The S computing units then calculate the first and second cumulative values ​​corresponding to each of the M bins in parallel. The first cumulative value corresponding to a bin is the cumulative value of the first gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to a bin is the cumulative value of the second gradient of the training sample corresponding to the bin. S is an integer greater than 1.

[0154] In some embodiments, the second module allocates M bins to S computing units based on the sample binning results of the first feature and a preset allocation rule. The S computing units then calculate the first and second cumulative values ​​corresponding to each of the M bins in parallel. The preset allocation rule refers to a pre-defined rule for allocating bins. In some embodiments, the preset allocation rule is pre-set or designed by relevant technical personnel, and this application embodiment does not limit this. In some embodiments, the preset allocation rule can be any of the following: round-robin allocation, uniform allocation, allocation by sample number, etc., and may also include other allocation rules, which this application embodiment does not limit. Round-robin allocation means allocating one bin to each computing unit at a time to evenly distribute the load. Uniform allocation means evenly distributing multiple bins to multiple computing units. Allocation by sample number means dynamic allocation, ensuring that the total number of samples in each computing unit is approximately equal.

[0155] S represents the number of computing units used for parallel computation of the first and second accumulated values ​​of each of the M bins. A computing unit refers to a basic processing unit that performs a computational task. In some embodiments, a computing unit may include at least one of the following: a thread, a GPU (Graphics Processing Unit) core, a process, a distributed computing node, etc., and may also include other computing units, which are not limited in this application embodiment.

[0156] In some embodiments, the second module obtains T bins based on the sample binning results of the first feature. The T bins are obtained by dividing at least one training sample along the binning dimension, and each bin includes at least one training sample belonging to the same bin. The second module assigns the T bins to S computing units, and the S computing units calculate the first and second cumulative values ​​corresponding to each of the T bins in parallel. The first cumulative value corresponding to the bin is the cumulative value of the first gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to the bin is the cumulative value of the second gradient of the training sample corresponding to the bin. The second module calculates the first and second cumulative values ​​corresponding to each of the M bins based on the first and second cumulative values ​​corresponding to each of the T bins.

[0157] A bin is a subset of the training samples corresponding to any one of the M bins. In some embodiments, the number of training samples corresponding to each of the T bins is the same. In some embodiments, the second module determines the training samples corresponding to each of the M bins based on the sample binning result of the first feature; for any one of the M bins, the second module divides the training samples corresponding to that bin into at least one bin according to a preset number; the second module determines the aforementioned T bins based on the bins corresponding to each of the M bins.

[0158] In some embodiments, the second module allocates the T bins to S computing units according to a preset allocation rule, and the S computing units calculate the first and second cumulative values ​​corresponding to each of the T bins in parallel. It should be noted that the number of training samples allocated to the S computing units is approximately the same. T represents the number of bins obtained from the M bins.

[0159] In some embodiments, for any one of the T bins, the second module adds the first gradients of the training samples corresponding to the bin to obtain the first cumulative value corresponding to the bin; the second module adds the second gradients of the training samples corresponding to the bin to obtain the second cumulative value corresponding to the bin.

[0160] In some embodiments, the second module traverses T sub-blocks and adds the first cumulative values ​​of the sub-blocks corresponding to each of the M sub-blocks to obtain the first cumulative value corresponding to each of the M sub-blocks; the second module traverses T sub-blocks and adds the second cumulative values ​​of the sub-blocks corresponding to each of the M sub-blocks to obtain the second cumulative value corresponding to each of the M sub-blocks.

[0161] When constructing the gradient histogram for any feature, the bins are first divided into blocks or segments. The cumulative values ​​of the first and second gradients of each bin are calculated in parallel. Finally, the cumulative values ​​of the first and second gradients of all bins are summarized serially to obtain the final cumulative values ​​of the first and second gradients for each bin. This method ensures that the number of training samples allocated to each computational unit is relatively even, meaning that the computational tasks of each unit are more balanced. This reduces the extra time spent by computational units with fewer tasks waiting for those with more tasks, effectively improving the efficiency of federated learning training.

[0162] For example, please refer to Figure 5 This illustration shows a schematic diagram of concurrent gradient accumulation calculation provided by an embodiment of this application, which first segments and then summarizes the samples. First, based on the sample binning results of the first feature, the training samples of M bins are divided into T bins. The gradient accumulation values ​​(including the first accumulation value and the second accumulation value) of each of the T bins are calculated in parallel. The gradient accumulation values ​​of each of the T bins are summarized to obtain the ciphertext histogram of the first feature.

[0163] Step 430: The second module constructs a ciphertext histogram based on the first and second cumulative values ​​corresponding to each of the M bins, thereby obtaining the ciphertext histogram of the first feature.

[0164] Step 440: The second module obtains a ciphertext histogram object based on the ciphertext histograms of each of the N first features.

[0165] By using the above method, multiple computing units are used to calculate the cumulative values ​​of the first-order gradient and the cumulative values ​​of the second-order gradient of each bin in parallel, which effectively improves the performance and efficiency of federated learning training.

[0166] The following describes how to provide feedback on encrypted histogram objects.

[0167] In some embodiments, the encrypted histogram object is used to train the target decision tree, at least one training sample is divided into F sample subsets, each sample subset corresponds to a leaf node of the last layer of the target decision tree, the encrypted histogram object includes F encrypted histogram sub-objects, each encrypted histogram sub-object is used to split a leaf node of the last layer of the target decision tree, where F is a positive integer.

[0168] The target decision tree is the decision tree currently being trained. Training the target decision tree refers to the growth process of the target decision tree. F represents the number of leaf nodes to be split. In some embodiments, the target decision tree includes leaf nodes that have been split and leaf nodes to be split. In some embodiments, the leaf nodes of the last layer of the target decision tree are the leaf nodes to be split. The sample subset corresponding to a leaf node is the set of training samples that fall into that leaf node. It should be noted that there is no overlap between the sample subsets corresponding to different leaf nodes.

[0169] In some embodiments, for any one of the leaf nodes mentioned above, the ciphertext histogram sub-object corresponding to that leaf node is used to indicate the ciphertext histogram corresponding to the N first features constructed based on the sample subset corresponding to that leaf node. That is, for any leaf node to be split, the second module constructs the ciphertext histogram of the N first features corresponding to that leaf node based on the sample subset corresponding to that leaf node. It should be noted that the method for constructing the ciphertext histogram of the N first features corresponding to that leaf node is described in steps 330 and 410-440 above, and will not be repeated here.

[0170] In some embodiments, leaf nodes at the same level each correspond to a node information, and each node information is used to indicate a leaf node. In some embodiments, the node information can be implemented as a sequence number (SequenceID).

[0171] In some embodiments, the method of providing feedback on the encrypted histogram object can be either synchronous or asynchronous.

[0172] (1) Synchronous feedback

[0173] In some embodiments, the first module simultaneously sends F ciphertext histogram sub-objects to the second device.

[0174] In this scenario, the first device calculates all the encrypted histograms corresponding to the leaf nodes before simultaneously sending them to the second device. In some embodiments, leaf nodes at the same level share the same message topic. A message topic is a logical channel used for classifying and organizing messages.

[0175] (2) Asynchronous feedback

[0176] In some embodiments, the first module asynchronously sends F ciphertext histogram sub-objects to the second device.

[0177] In this scenario, after the first device calculates the encrypted histogram corresponding to a leaf node, it directly sends it to the second device, achieving asynchronous feedback of F encrypted histogram sub-objects. In some embodiments, leaf nodes at the same level share the same message topic.

[0178] For example, please refer to Figure 6 The diagram illustrates a target decision tree provided in an embodiment of this application. The target decision trees in the first device and the second device are completely identical. The last layer 60 of the target decision tree includes two leaf nodes, namely leaf node 2 and leaf node 3. Splitting operations on these two leaf nodes can be performed concurrently at the layer level and share the same message topic.

[0179] In some embodiments, the second device transmits the gradient serialization data corresponding to all training samples to the aforementioned message topic; the first device asynchronously obtains the gradient serialization data corresponding to the sample subset corresponding to a single leaf node from the aforementioned message topic.

[0180] In related technologies, Pulsar is used as a message middleware for communication. Its advantage is that it does not require the interconnection of networks between different institutions or enterprises. However, the creation of Pulsar topics and the synchronization of data sending and consumption are relatively time-consuming operations. If the implementation is not good during the interaction process, it will become a major performance bottleneck in the training process.

[0181] By employing the above method, the leaf nodes of the target decision tree are split concurrently at the layer level. This parallel processing coarsely and effectively accelerates the training process of the target decision tree and reduces the training time of federated learning. Concurrent interaction at the layer level, with layer merging performed during communication, significantly reduces the number of communication operations.

[0182] The data processing method on the first device side has been described above. The data processing method on the second device side will now be described. The steps of the data processing methods on the first and second devices correspond to each other. For details not shown in the embodiments corresponding to the data processing method on the second device side, please refer to the section on the data processing method on the first device side.

[0183] Please refer to Figure 7 The diagram illustrates a flowchart of a data processing method provided in another embodiment of this application. The execution entity for each step of this method can be a computer device; for example, the computer device could be... Figure 1 The second device 20 in the illustrated implementation environment. The method may include at least one of the following steps (710-760):

[0184] Step 710: The first module sends the gradient data to the second module. The gradient data includes the sample gradient corresponding to at least one training sample.

[0185] In some embodiments, the second device includes a first module and a second module, which are implemented using different programming languages.

[0186] The first module and the second module are two different functional modules in the second device used to implement the functions of the data provider. In some embodiments, the first module and the second module are used to implement different functions. In some embodiments, the computational efficiency of the second module is higher than that of the first module.

[0187] In some embodiments, the first module is used to implement business logic. In some embodiments, the first module is used to implement at least one of the following: data preprocessing, key management, business interaction, etc., and may also be used to implement other functions, which are not limited in this application embodiment.

[0188] In some embodiments, the second module is used to implement computationally intensive tasks. In some embodiments, the second module is used to implement at least one of the following: encryption computation, decryption computation, serialization operation, deserialization operation, gradient computation, etc., and may also be used to implement other functions, which are not limited in this application embodiment. Gradient computation refers to calculating the sample gradient corresponding to the training sample.

[0189] In some embodiments, the first module is implemented in Java, and the second module is implemented in C++ or C. The first module and the second module call each other through JNI.

[0190] In some embodiments, the first module sends the fifth information to the second module, and the fifth information is used to determine the storage location of the gradient data in the second device.

[0191] Step 720: The second module performs homomorphic encryption on the gradient data to obtain the gradient ciphertext object.

[0192] In some embodiments, the second module performs homomorphic encryption on the gradient data using a first homomorphic encryption algorithm to obtain a gradient ciphertext object. In some embodiments, the first homomorphic encryption algorithm can be any homomorphic encryption algorithm that supports addition among the homomorphic encryption algorithms described above.

[0193] Step 730: The second module performs a serialization operation on the gradient ciphertext object to obtain gradient serialized data, which is the serialized gradient ciphertext object.

[0194] In step 740, the second module sends the second information to the first module. The second information is used to determine the storage location of the gradient serialization data in the second device.

[0195] In some embodiments, the second information includes second pointer information of the gradient serialization data, which indicates the storage location of the gradient serialization data in the second device.

[0196] The second pointer information is a pointer used to locate the encrypted histogram object. In some embodiments, the type of the second pointer information includes at least one of the following: a memory pointer, a shared memory pointer, an inter-process communication handle, etc., and may also include other types, which are not limited in this application embodiment.

[0197] In some embodiments, the second information may also include at least one of the following: path information, index information, distributed storage address, memory address, URL, etc., and may also include other information, which are not limited in this application embodiment.

[0198] Step 750: The first module obtains the gradient serialization data from the storage location of the gradient serialization data in the second device based on the second information.

[0199] Since both the first module and the second module can access the storage system of the second device, after the first module receives the first information, it can obtain the encrypted histogram object from the storage system of the second device based on the first information.

[0200] In some embodiments, the first module and the second module share a second memory region in the second device, and the gradient serialization data is stored in the second memory region.

[0201] The second memory region is a region within the memory of the second device. In some embodiments, the second memory region may be created by the first module or by the second module, and this application embodiment does not limit this.

[0202] In some embodiments, the first module obtains gradient serialization data at a storage location in the second memory region based on the second information.

[0203] In some embodiments, the storage location indicated by the second information and the storage location indicated by the fifth information are both located within the second memory region.

[0204] Step 760: The first module sends the gradient serialization data to at least one first device.

[0205] In summary, the technical solution provided in this application embodiment allows modules implemented in different programming languages ​​to call each other during the process of encrypting gradient data of training samples. Second information is transmitted between different modules to indicate the storage location of gradient serialization data. Different modules can obtain gradient serialization data based on this second information, which reduces the amount of processing required by the device to transmit gradient serialization data, thereby effectively improving the processing efficiency of the device in training decision trees based on federated learning.

[0206] The process of determining the optimal split point is described below.

[0207] In some embodiments, please refer to Figure 8 The document illustrates a flowchart of determining the optimal split point according to an embodiment of this application. The process of determining the optimal split point includes at least one of the following steps (810-860):

[0208] Step 810: The first module receives at least one encrypted histogram object sent by the first device. The encrypted histogram object includes at least one encrypted histogram corresponding to the first device.

[0209] Each ciphertext histogram object corresponds to a first device, including at least one ciphertext histogram corresponding to the first device.

[0210] In step 820, the first module sends the third information to the second module, which indicates the storage location of the encrypted histogram object in the second device.

[0211] In some embodiments, the storage location indicated by the third information may belong to a second memory region or other memory regions, and this application embodiment does not limit this.

[0212] Step 830: The second module retrieves at least one encrypted histogram object sent by the device from the storage location of the encrypted histogram object in the second device, based on the third information.

[0213] In some embodiments, the first module stores at least one histogram serialization data sent by a first device in the storage location indicated by the third information. The histogram serialization data is the serialized data after ciphertext histogram serialization. In some embodiments, the second module performs a deserialization operation on the histogram serialization data sent by the at least one first device to obtain a ciphertext histogram object sent by the at least one first device.

[0214] Step 840: The second module performs homomorphic decryption on the ciphertext histogram object sent by at least one first device to obtain the gradient histogram object of at least one first device.

[0215] In some embodiments, the second module performs homomorphic decryption on the ciphertext histogram object sent by at least one first device using a first homomorphic encryption algorithm to obtain the gradient histogram object of at least one first device. The first homomorphic encryption algorithm is an algorithm used for homomorphic encryption of gradient data.

[0216] In some embodiments, before executing step 840, the second module performs a deserialization operation on the ciphertext histogram object sent by at least one first device to obtain object deserialization data of at least one first device. The object serialization data is the deserialized ciphertext histogram. The object deserialization data of at least one first device is then deserialized by the second module to obtain gradient histogram object of at least one first device.

[0217] Since the encrypted histogram object sent by the first device to the second device is a serialized encrypted histogram object, it is necessary to perform a deserialization operation on the serialized encrypted histogram object in order to finally obtain the encrypted histogram object.

[0218] Step 850: The second module sends at least one gradient histogram object of the first device to the first module.

[0219] In some embodiments, the second module sends sixth information to the first module, the sixth information being used to determine the storage location of at least one gradient histogram object of the first device in the second device.

[0220] Step 860: The first module determines node splitting information based on the gradient histogram object of the second device and at least one gradient histogram object of the first device. The node splitting information is used to indicate the splitting status of the leaf nodes of the target decision tree.

[0221] In some embodiments, node splitting information is used to indicate the sample subset of the left node and the sample subset of the right node of the split leaf node.

[0222] In some embodiments, the first module determines at least one candidate split point based on the bins corresponding to at least one feature; the first module calculates the gain information of each of the at least one candidate split point based on the gradient histogram object of the second device and the gradient histogram object of at least one first device, the gain information of the candidate split point being used to indicate the degree of decrease in the residual of the target decision tree based on the candidate split point; the first module determines the candidate split point with the largest gain information as the target split point. The at least one feature includes at least one first feature possessed by the first device and a second feature possessed by the second device. In some embodiments, the candidate split point can be the intersection of any two adjacent bins of any feature.

[0223] Gain information is the reduction in the objective function loss after splitting a leaf node, i.e., the gain brought by the split. In some embodiments, gain information is implemented as the Gini coefficient. That is, gain information is calculated using the Gini impurity formula.

[0224] In some embodiments, when the target split point belongs to the first device, the first module sends the target split point to the first device, the first device completes the splitting of the leaf nodes of the target decision tree, and the first device sends the node splitting information to other first devices and second devices.

[0225] In some embodiments, when the target split point belongs to the second device, the first module re-divides the sample subset corresponding to the leaf node based on the target split point to obtain the sample subset corresponding to the left node of the leaf node and the sample subset corresponding to the right node of the leaf node; the first module determines the node split information based on the sample subset corresponding to the left node of the leaf node and the sample subset corresponding to the right node of the leaf node.

[0226] In some embodiments, a gradient histogram object is used to train a target decision tree. The gradient histogram object includes at least one gradient histogram sub-object, which is used to train a leaf node of the last layer of the target decision tree. At least one training sample is divided into F sample subsets, each sample subset corresponding to a leaf node of the last layer of the target decision tree, where F is a positive integer.

[0227] In some embodiments, the target decision tree includes leaf nodes that have been split and leaf nodes to be split. In some embodiments, the leaf nodes of the last layer of the target decision tree are the leaf nodes to be split. The sample subset corresponding to a leaf node is the set of training samples that fall into that leaf node. It should be noted that there is no overlap between the sample subsets corresponding to different leaf nodes.

[0228] In some embodiments, for any one of the leaf nodes described above, at least one gradient histogram sub-object corresponding to that leaf node is used to indicate gradient histograms of different features constructed based on the sample subset corresponding to that leaf node. Each gradient histogram object includes F gradient histogram sub-objects, and each gradient histogram sub-object is used to split a leaf node.

[0229] In some embodiments, for any leaf node in the last layer of the target decision tree, the first module determines the node splitting information of the leaf node based on the gradient histogram sub-object corresponding to the leaf node. The node splitting information of the leaf node is used to indicate the distribution of the left and right nodes of the sample subset corresponding to the leaf node after the leaf node is split. The node splitting information includes the node splitting information of each of the F leaf nodes.

[0230] In some embodiments, for any leaf node in the last layer of the target decision tree, the number of gradient histogram sub-objects corresponding to that leaf node is greater than or equal to 1.

[0231] In some embodiments, the feedback node splits sub-information in a synchronous or asynchronous manner.

[0232] (1) Synchronous feedback

[0233] In some embodiments, the first module simultaneously sends the node splitting information of each of the F leaf nodes to the second device.

[0234] In this scenario, after the second device has determined all the node splitting information corresponding to the leaf nodes of the last layer, it simultaneously sends it to at least one first device. In some embodiments, leaf nodes at the same layer share the same message topic.

[0235] (2) Asynchronous feedback

[0236] In some embodiments, the first module asynchronously sends the node splitting information of each of the F leaf nodes to the second device.

[0237] In this scenario, once the second device has determined the node splitting information corresponding to a leaf node, it directly sends it to at least one first device, thus asynchronously feeding back the node splitting information of F leaf nodes. In some embodiments, leaf nodes at the same level share the same message topic.

[0238] In some embodiments, the first module sends gradient serialization data through the aforementioned message topic, the gradient serialization data including the gradient data of the aforementioned F sample subsets.

[0239] By using the above method, the leaf nodes of the target decision tree are split concurrently at the layer level. This parallel processing roughly and effectively accelerates the training process of the target decision tree and reduces the training time of federated learning.

[0240] The following describes the process of constructing the gradient histogram using the second device. The method by which the second device constructs the gradient histogram is similar to that of the first device in constructing the encrypted histogram. For details not shown below, please refer to the section on constructing the encrypted histogram using the first device.

[0241] In some embodiments, the second device has K second features of at least one training sample, where K is a positive integer; the second module constructs gradient histograms of each of the K second features based on the gradient data and the K second features corresponding to the at least one training sample, thereby obtaining a gradient histogram object of the second device.

[0242] K represents the number of second features possessed by the second device.

[0243] In some embodiments, the sample gradient corresponding to the training sample includes a first-order gradient and a second-order gradient. For any one of the K second features, the second module divides at least one training sample into L bins of the second feature to obtain the sample binning result of the second feature. The sample binning result is used to indicate the training sample corresponding to each of the L bins, where L is an integer greater than 1. Based on the sample binning result of the second feature, the second module allocates the L bins to P computing units. The P computing units calculate the first cumulative value and the second cumulative value corresponding to each of the L bins in parallel. The first cumulative value corresponding to the bin is the cumulative value of the first-order gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to the bin is the cumulative value of the second-order gradient of the training sample corresponding to the bin, where P is an integer greater than 1. Based on the first cumulative value and the second cumulative value corresponding to each of the L bins, the second module constructs a gradient histogram to obtain the gradient histogram of the second feature. Based on the gradient histograms of the K second features, the second module obtains the gradient histogram object of the second device, which is sent by the second module to the first module.

[0244] L represents the number of bins for this second feature. P represents the number of computing units used to compute the first and second cumulative values ​​of each of the L bins in parallel.

[0245] In some embodiments, the second module obtains G bins based on the sample binning results of the second feature. The G bins are obtained by dividing at least one training sample along the binning dimension, and each bin includes at least one training sample belonging to the same bin. The second module distributes the G bins to P computing units, and the P computing units calculate the first and second cumulative values ​​corresponding to each of the G bins in parallel. The first cumulative value corresponding to the bin is the cumulative value of the first gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to the bin is the cumulative value of the second gradient of the training sample corresponding to the bin. The second module calculates the first and second cumulative values ​​corresponding to each of the G bins based on the first and second cumulative values ​​corresponding to each of the G bins.

[0246] G represents the number of bin blocks obtained from L bins.

[0247] like Figure 9 As shown, it is a bar chart illustrating the average time taken to train a decision tree according to an embodiment of this application. Figure 9 The average training time (in minutes) for a single decision tree is shown for the benchmark method, the training method of federated GBDT based on communication layer merging and parallelization, the training method of federated GBDT based on OU homomorphic encryption, the training method of federated GBDT based on encrypted histogram scatter aggregation optimization, and the training method of federated GBDT with ciphertext memory sharing provided in this application.

[0248] Depend on Figure 9 As shown in Table 1 below, the training method of the encrypted memory-shared federated GBDT provided in this application has the shortest average training time and the highest training efficiency for training a single decision tree.

[0249] Table 1. Average time consumption of a single decision tree under different optimization methods.

[0250]

[0251] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0252] Please refer to Figure 10 This diagram illustrates a block diagram of a data processing apparatus according to an embodiment of this application. The apparatus has the functions described above, which can be implemented in hardware or by hardware executing corresponding software. The apparatus can be the first device 10 described above, or it can be disposed within the first device 10. Figure 10As shown, the device 1000 may include a first module 1010 and a second module 1020, which are implemented based on different programming languages.

[0253] The first module 1010 is used to receive gradient serialization data sent by the second device. The gradient serialization data is a serialized gradient ciphertext object, which is homomorphically encrypted gradient data. The gradient data includes sample gradients corresponding to at least one training sample.

[0254] The first module 1010 is also used to send the gradient serialization data to the second module.

[0255] The second module 1020 is used to construct a ciphertext gradient histogram for each of the N first features based on the gradient serialization data and the N first features corresponding to the at least one training sample, and obtain a ciphertext histogram object. The ciphertext histogram object includes the ciphertext histograms for each of the N first features. The ciphertext histogram of the first feature is used to indicate the distribution of the at least one training sample in at least one bin of the first feature, where N is a positive integer.

[0256] The second module 1020 is further configured to send first information to the first module, the first information being used to determine the storage location of the encrypted histogram object in the first device.

[0257] The first module 1010 is further configured to obtain the ciphertext histogram object from the storage location of the ciphertext histogram object in the first device based on the first information.

[0258] The first module 1010 is further configured to send the encrypted histogram object to the second device.

[0259] In some embodiments, the first module and the second module share a first memory region in the first device, and the ciphertext histogram object is stored in the first memory region; the first module 1010 is configured to obtain the ciphertext histogram object at the storage location in the first memory region according to the first information.

[0260] In some embodiments, the first information includes first pointer information of the ciphertext histogram object, the first pointer information being used to indicate the storage location of the ciphertext histogram object in the first device.

[0261] In some embodiments, the second module 1020 is configured to perform a deserialization operation on the gradient serialized data to obtain the gradient ciphertext object; and to construct ciphertext histograms for each of the N first features based on the gradient ciphertext object and the N first features corresponding to the at least one training sample to obtain the ciphertext histogram object.

[0262] In some embodiments, the sample gradient corresponding to the training sample includes a first-order gradient and a second-order gradient; the second module 1020 is configured to, for any one of the N first features, divide the at least one training sample into M bins of the first feature to obtain the sample binning result of the first feature, the sample binning result being used to indicate the training sample corresponding to each of the M bins, where M is an integer greater than 1; based on the sample binning result of the first feature, allocate the M bins to S computing units, and calculate the first cumulative value and the second cumulative value corresponding to each of the M bins in parallel through the S computing units, where the first cumulative value corresponding to the bin is the cumulative value of the first-order gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to the bin is the cumulative value of the second-order gradient of the training sample corresponding to the bin, where S is an integer greater than 1; construct a ciphertext histogram according to the first cumulative value and the second cumulative value corresponding to each of the M bins to obtain the ciphertext histogram of the first feature; and obtain the ciphertext histogram object based on the ciphertext histograms of the N first features.

[0263] In some embodiments, the second module 1020 is configured to obtain T bins based on the sample binning results of the first feature, wherein the T bins are obtained by dividing the at least one training sample along the binning dimension, and each bin includes at least one training sample belonging to the same bin; allocate the T bins to the S computing units, and calculate the first cumulative value and the second cumulative value corresponding to each of the T bins in parallel through the S computing units, wherein the first cumulative value corresponding to the bin is the cumulative value of the first gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to the bin is the cumulative value of the second gradient of the training sample corresponding to the bin; and calculate the first cumulative value and the second cumulative value corresponding to each of the M bins based on the first cumulative value and the second cumulative value corresponding to each of the T bins.

[0264] In some embodiments, the encrypted histogram object is used to train a target decision tree, the at least one training sample is divided into F sample subsets, each sample subset corresponds to a leaf node of the last layer of the target decision tree, the encrypted histogram object includes F encrypted histogram sub-objects, each encrypted histogram sub-object is used to split a leaf node of the last layer of the target decision tree, and F is a positive integer; the first module 1010 is used to send the F encrypted histogram sub-objects to the second device simultaneously; or, send the F encrypted histogram sub-objects to the second device asynchronously.

[0265] In some embodiments, the second module 1020 is configured to perform a serialization operation on the ciphertext histogram object to obtain a serialized ciphertext histogram object; the second module 1020 is configured to store the serialized ciphertext histogram in the storage location indicated by the first information, and the serialized ciphertext histogram is obtained by the first module from the storage location indicated by the first information and sent to the second device.

[0266] In some embodiments, the first module is implemented in Java, and the second module is implemented in C++ or C. The first module and the second module call each other through JNI.

[0267] In summary, the technical solution provided in this application embodiment allows modules implemented in different programming languages ​​to call each other during the construction of gradient histograms. By transmitting first information between different modules, the first information indicates the storage location of the constructed ciphertext histogram. Different modules can obtain the constructed ciphertext histogram based on this first information, reducing the processing load of transmitting ciphertext histograms by the device, thereby effectively improving the processing efficiency of the device in training decision trees based on federated learning.

[0268] Please refer to Figure 11 This illustrates a block diagram of a data processing apparatus according to another embodiment of this application. The apparatus has the functions described above, which can be implemented in hardware or by hardware executing corresponding software. The apparatus can be the second device 20 described above, or it can be disposed within the second device 20. For example... Figure 11 As shown, the device 1100 may include a first module 1110 and a second module 1120.

[0269] The first module 1110 is used to send gradient data to the second module, wherein the gradient data includes sample gradients corresponding to at least one training sample.

[0270] The second module 1120 is used to perform homomorphic encryption on the gradient data to obtain a gradient ciphertext object.

[0271] The second module 1120 is used to perform a serialization operation on the gradient ciphertext object to obtain gradient serialized data, wherein the gradient serialized data is the serialized gradient ciphertext object.

[0272] The second module 1120 is used to send second information to the first module, the second information being used to determine the storage location of the gradient serialization data in the second device.

[0273] The first module 1110 is used to obtain the gradient serialization data from the storage location of the gradient serialization data in the second device according to the second information.

[0274] The first module 1110 is used to send the gradient serialization data to at least one first device.

[0275] In some embodiments, the first module and the second module share a second memory region in the second device, and the gradient serialization data is stored in the second memory region; the first module 1110 is configured to obtain the gradient serialization data at the storage location in the second memory region according to the second information.

[0276] In some embodiments, the second information includes second pointer information of the gradient serialization data, which indicates the storage location of the gradient serialization data in the second device.

[0277] In some embodiments, the first module 1110 is configured to receive a ciphertext histogram object sent by the at least one first device, the ciphertext histogram object including at least one ciphertext histogram corresponding to the first device; send third information to the second module, the third information indicating the storage location of the ciphertext histogram object in the second device; the second module 1120 is configured to obtain the ciphertext histogram object sent by the at least one first device at the storage location of the ciphertext histogram object in the second device according to the third information; the second module 1120 is configured to perform homomorphic decryption on the ciphertext histogram object sent by the at least one first device to obtain the gradient histogram object of the at least one first device; the second module 1120 is configured to send the gradient histogram object of the at least one first device to the first module; the first module 1110 is configured to determine node splitting information based on the gradient histogram object of the second device and the gradient histogram object of the at least one first device, the node splitting information indicating the splitting status of the leaf nodes of the target decision tree.

[0278] In some embodiments, the second device has K second features of the at least one training sample, where K is a positive integer; the second module 1120 is used to construct gradient histograms of the K second features respectively based on the gradient data and the K second features corresponding to the at least one training sample, to obtain a gradient histogram object of the second device, wherein the gradient histogram object includes the gradient histograms of the K second features respectively.

[0279] In some embodiments, the sample gradient corresponding to the training sample includes a first-order gradient and a second-order gradient; the second module 1120 is configured to, for any one of the K second features, divide the at least one training sample into L bins of the second feature to obtain the sample binning result of the second feature, the sample binning result being used to indicate the training sample corresponding to each of the L bins, where L is an integer greater than 1; based on the sample binning result of the second feature, allocate the L bins to P computing units, and calculate the first cumulative value and the second cumulative value corresponding to each of the L bins in parallel through the P computing units, where the first cumulative value corresponding to the bin is the cumulative value of the first-order gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to the bin is the cumulative value of the second-order gradient of the training sample corresponding to the bin, where P is an integer greater than 1; construct a gradient histogram according to the first and second cumulative values ​​corresponding to each of the L bins to obtain the gradient histogram of the second feature; based on the gradient histograms of each of the K second features, obtain the gradient histogram object of the second device, and send the gradient histogram object of the second device to the first module by the second module.

[0280] In some embodiments, the second module 1120 is configured to obtain G bins based on the sample binning results of the second feature, wherein the G bins are obtained by dividing the at least one training sample along the binning dimension, and each bin includes at least one training sample belonging to the same bin; the second module 1120 is configured to allocate the G bins to the P computing units, and calculate the first cumulative value and the second cumulative value corresponding to each of the G bins in parallel through the P computing units, wherein the first cumulative value corresponding to the bin is the cumulative value of the first gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to the bin is the cumulative value of the second gradient of the training sample corresponding to the bin; the second module 1120 is configured to calculate the first cumulative value and the second cumulative value corresponding to each of the G bins based on the first cumulative value and the second cumulative value corresponding to each of the G bins.

[0281] In some embodiments, the gradient histogram object is used to train the target decision tree. The gradient histogram object includes at least one gradient histogram sub-object. The gradient histogram sub-object is used to train a leaf node of the last layer of the target decision tree. The at least one training sample is divided into F sample subsets, each sample subset corresponding to a leaf node of the last layer of the target decision tree, where F is a positive integer. The first module 1110 is used to determine the node splitting information of any leaf node of the last layer of the target decision tree based on the gradient histogram sub-object corresponding to the leaf node. The node splitting information of the leaf node is used to indicate the distribution of the left and right nodes of the sample subset corresponding to the leaf node after the leaf node is split. The node splitting information includes the node splitting information of each of the F leaf nodes.

[0282] In some embodiments, the first module 1110 is configured to simultaneously send the node splitting information of each of the F leaf nodes to the second device; or, asynchronously send the node splitting information of each of the F leaf nodes to the second device.

[0283] In some embodiments, the second module 1120 is configured to perform a deserialization operation on the ciphertext histogram object sent by the at least one first device to obtain object deserialization data of the at least one first device. The object serialization data is a deserialized ciphertext histogram. The object deserialization data of the at least one first device is used by the second module to perform homomorphic decryption to obtain the gradient histogram object of the at least one first device.

[0284] In summary, the technical solution provided in this application embodiment allows modules implemented in different programming languages ​​to call each other during the process of encrypting gradient data of training samples. Second information is transmitted between different modules to indicate the storage location of gradient serialization data. Different modules can obtain gradient serialization data based on this second information, which reduces the amount of processing required by the device to transmit gradient serialization data, thereby effectively improving the processing efficiency of the device in training decision trees based on federated learning.

[0285] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0286] Please refer to Figure 12 This diagram illustrates a structural block diagram of a computer device 1200 provided in one embodiment of this application. The computer device 1200 may be... Figure 1 The first device 10 in the implementation environment shown can also be Figure 1 The second device 20 in the illustrated implementation environment is used to implement the data processing method provided in the above embodiments. Specifically:

[0287] Typically, computer device 1200 includes a processor 1210 and a memory 1220.

[0288] Processor 1210 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1210 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1210 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1210 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1210 may also include an AI processor for handling computational operations related to machine learning.

[0289] The memory 1220 may include one or more computer-readable storage media, which may be non-transitory. The memory 1220 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1220 are used to store a computer program configured to be executed by one or more processors to implement the data processing method described above.

[0290] Those skilled in the art will understand that Figure 12 The structure shown does not constitute a limitation on the computer device 1200 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0291] In an exemplary embodiment, a computer-readable storage medium is also provided, wherein a computer program is stored in the storage medium, and the computer program, when executed by a processor, implements the above-described data processing method. Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0292] In an exemplary embodiment, a computer program product is also provided, the computer program product including a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, causing the computer device to perform the data processing method described above.

[0293] It should be noted that the collection and processing of relevant data (such as training samples) in this application should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0294] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0295] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A data processing method, characterized in that, The method is executed by a first device, the first device comprising a first module and a second module, the first module and the second module being implemented based on different programming languages, the method comprising: The first module receives gradient serialized data sent by the second device. The gradient serialized data is a serialized gradient ciphertext object, which is homomorphically encrypted gradient data. The gradient data includes sample gradients corresponding to at least one training sample. The first module sends the gradient serialized data to the second module; The second module constructs a ciphertext histogram for each of the N first features based on the gradient serialization data and the at least one training sample, resulting in a ciphertext histogram object. The ciphertext histogram object includes the ciphertext histograms for each of the N first features. The ciphertext histogram of the first feature is used to indicate the distribution of the at least one training sample in at least one bin of the first feature, where N is a positive integer. The second module sends the first information to the first module, and the first information is used to determine the storage location of the encrypted histogram object in the first device; The first module obtains the ciphertext histogram object from its storage location in the first device based on the first information. The first module sends the encrypted histogram object to the second device.

2. The method according to claim 1, characterized in that, The first module and the second module share a first memory region in the first device, and the ciphertext histogram object is stored in the first memory region; The step of obtaining the ciphertext histogram object from its storage location in the first device based on the first information by the first module includes: The first module obtains the ciphertext histogram object at the storage location in the first memory region based on the first information.

3. The method according to claim 1 or 2, characterized in that, The first information includes first pointer information of the encrypted histogram object, which indicates the storage location of the encrypted histogram object in the first device.

4. The method according to any one of claims 1 to 3, characterized in that, The step of constructing a ciphertext histogram for each of the N first features based on the gradient serialization data and the at least one training sample by the second module, thereby obtaining a ciphertext histogram object, includes: The second module performs a deserialization operation on the gradient serialized data to obtain the gradient ciphertext object; The second module constructs ciphertext histograms for each of the N first features based on the gradient ciphertext object and the N first features corresponding to the at least one training sample, thereby obtaining the ciphertext histogram object.

5. The method according to claim 4, characterized in that, The sample gradients corresponding to the training samples include first-order gradients and second-order gradients; The step of constructing ciphertext histograms for each of the N first features based on the gradient ciphertext object and the at least one training sample by the second module, thereby obtaining the ciphertext histogram object, includes: For any one of the N first features, the second module divides the at least one training sample into M bins of the first feature to obtain the sample binning result of the first feature. The sample binning result is used to indicate the training sample corresponding to each of the M bins, where M is an integer greater than 1. Based on the sample binning results of the first feature, the second module allocates the M bins to S computing units. The S computing units then calculate the first and second cumulative values ​​corresponding to each of the M bins in parallel. The first cumulative value corresponding to the bin is the cumulative value of the first gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to the bin is the cumulative value of the second gradient of the training sample corresponding to the bin. S is an integer greater than 1. The second module constructs a ciphertext histogram based on the first and second cumulative values ​​corresponding to each of the M bins, thereby obtaining the ciphertext histogram of the first feature; The second module obtains the ciphertext histogram object based on the ciphertext histograms of the N first features respectively.

6. The method according to claim 5, characterized in that, The process involves the second module allocating the M bins to S computing units based on the sample binning results of the first feature, and then using the S computing units to calculate the first and second cumulative values ​​corresponding to each of the M bins in parallel, including: Based on the sample binning results of the second module based on the first feature, T bins are obtained. The T bins are obtained by dividing the at least one training sample in the binning dimension. Each bin includes at least one training sample belonging to the same bin. The second module allocates the T bins to the S computing units, and the S computing units calculate in parallel the first and second cumulative values ​​corresponding to each of the T bins. The first cumulative value corresponding to the bin is the cumulative value of the first gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to the bin is the cumulative value of the second gradient of the training sample corresponding to the bin. The second module calculates the first and second cumulative values ​​corresponding to each of the M sub-boxes based on the first and second cumulative values ​​corresponding to each of the T sub-boxes.

7. The method according to any one of claims 1 to 6, characterized in that, The encrypted histogram object is used to train the target decision tree. The at least one training sample is divided into F sample subsets. Each sample subset corresponds to a leaf node in the last layer of the target decision tree. The encrypted histogram object includes F encrypted histogram sub-objects. Each encrypted histogram sub-object is used to split a leaf node in the last layer of the target decision tree. F is a positive integer. The step of sending the encrypted histogram object to the second device by the first module includes: The first module simultaneously sends the F ciphertext histogram sub-objects to the second device; or, The first module asynchronously sends the F encrypted histogram sub-objects to the second device.

8. The method according to any one of claims 1 to 7, characterized in that, Before the second module sends the first information to the first module, the method further includes: The second module performs a serialization operation on the ciphertext histogram object to obtain a serialized ciphertext histogram object; The second module stores the serialized ciphertext histogram in the storage location indicated by the first information, and the first module retrieves the serialized ciphertext histogram from the storage location indicated by the first information and sends it to the second device.

9. The method according to any one of claims 1 to 8, characterized in that, The first module is implemented in Java, and the second module is implemented in C++ or C. The first module and the second module call each other through the Java Native Interface (JNI).

10. A data processing method, characterized in that, The method is executed by a second device, the second device comprising a first module and a second module, the first module and the second module being implemented based on different programming languages, the method comprising: The first module sends gradient data to the second module, and the gradient data includes sample gradients corresponding to at least one training sample. The second module performs homomorphic encryption on the gradient data to obtain a gradient ciphertext object; The second module performs a serialization operation on the gradient ciphertext object to obtain gradient serialized data, which is the serialized gradient ciphertext object. The second module sends the second information to the first module, and the second information is used to determine the storage location of the gradient serialization data in the second device; The first module obtains the gradient serialization data from the storage location of the gradient serialization data in the second device based on the second information; The first module sends the gradient serialized data to at least one first device.

11. The method according to claim 10, characterized in that, The first module and the second module share a second memory region in the second device, and the gradient serialization data is stored in the second memory region; The step of obtaining the gradient serialization data from the storage location of the gradient serialization data in the second device by the first module based on the second information includes: The first module obtains the gradient serialization data at the storage location in the second memory region based on the second information.

12. The method according to claim 10 or 11, characterized in that, The second information includes a second pointer to the gradient serialized data, which indicates the storage location of the gradient serialized data in the second device.

13. The method according to any one of claims 10 to 12, characterized in that, The method further includes: The first module receives a ciphertext histogram object sent by the at least one first device, the ciphertext histogram object including at least one ciphertext histogram corresponding to the first device; The first module sends third information to the second module, the third information being used to indicate the storage location of the encrypted histogram object in the second device; The second module obtains the ciphertext histogram object sent by the at least one first device at the storage location of the ciphertext histogram object in the second device based on the third information. The second module performs homomorphic decryption on the ciphertext histogram object sent by the at least one first device to obtain the gradient histogram object of the at least one first device; The second module sends the gradient histogram object of the at least one first device to the first module; The first module determines node splitting information based on the gradient histogram object of the second device and the gradient histogram object of the at least one first device. The node splitting information is used to indicate the splitting status of the leaf nodes of the target decision tree.

14. The method according to claim 13, characterized in that, The second device possesses K second features of the at least one training sample, where K is a positive integer; the method further includes: The second module constructs gradient histograms for each of the K second features based on the gradient data and the at least one training sample, thereby obtaining a gradient histogram object for the second device. The gradient histogram object includes the gradient histograms for each of the K second features.

15. The method according to claim 14, characterized in that, The sample gradients corresponding to the training samples include first-order gradients and second-order gradients; The step of constructing gradient histograms for each of the K second features based on the gradient data and the at least one training sample, thereby obtaining a gradient histogram object for the second device, includes: For any one of the K second features, the second module divides the at least one training sample into L bins of the second feature to obtain the sample binning result of the second feature. The sample binning result is used to indicate the training sample corresponding to each of the L bins, where L is an integer greater than 1. Based on the sample binning results of the second module according to the second feature, the L bins are allocated to P computing units. The P computing units calculate the first cumulative value and the second cumulative value corresponding to each of the L bins in parallel. The first cumulative value corresponding to the bin is the cumulative value of the first gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to the bin is the cumulative value of the second gradient of the training sample corresponding to the bin. P is an integer greater than 1. The second module constructs a gradient histogram based on the first and second cumulative values ​​corresponding to each of the L bins, thereby obtaining the gradient histogram of the second feature; The second module obtains the gradient histogram object of the second device based on the gradient histograms of the K second features, and sends the gradient histogram object of the second device to the first module.

16. The method according to claim 15, characterized in that, Based on the sample binning results of the second module according to the second feature, the L bins are allocated to P computing units, and the first and second cumulative values ​​corresponding to each of the L bins are calculated in parallel by the P computing units, including: Based on the sample binning results of the second module based on the second feature, G bins are obtained. The G bins are obtained by dividing the at least one training sample in the binning dimension. Each bin includes at least one training sample belonging to the same bin. The second module distributes the G bins to the P computing units, and the P computing units calculate in parallel the first and second cumulative values ​​corresponding to each of the G bins. The first cumulative value corresponding to the bin is the cumulative value of the first gradient of the training sample corresponding to the bin, and the second cumulative value corresponding to the bin is the cumulative value of the second gradient of the training sample corresponding to the bin. The second module calculates the first and second cumulative values ​​corresponding to each of the G sub-boxes based on the first and second cumulative values ​​corresponding to each of the G sub-boxes.

17. The method according to any one of claims 13 to 16, characterized in that, The gradient histogram object is used to train the target decision tree. The gradient histogram object includes at least one gradient histogram sub-object. The gradient histogram sub-object is used to train a leaf node of the last layer of the target decision tree. The at least one training sample is divided into F sample subsets, and each sample subset corresponds to a leaf node of the last layer of the target decision tree, where F is a positive integer. The step of determining node splitting information by the first module based on the gradient histogram object of the second device and the gradient histogram object of at least one first device includes: For any leaf node in the last layer of the target decision tree, the first module determines the node splitting information of the leaf node based on the gradient histogram sub-object corresponding to the leaf node. The node splitting information of the leaf node is used to indicate the distribution of the left and right nodes of the sample subset corresponding to the leaf node after the leaf node is split. The node splitting information includes the node splitting sub-information of each of the F leaf nodes.

18. The method according to claim 17, characterized in that, The method further includes: The first module simultaneously sends the node splitting information of each of the F leaf nodes to the second device; or, The first module asynchronously sends the node splitting information of each of the F leaf nodes to the second device.

19. The method according to any one of claims 13 to 18, characterized in that, Before the second module performs homomorphic decryption on the ciphertext histogram object sent by the at least one first device to obtain the gradient histogram object of the at least one first device, the method further includes: The second module performs a deserialization operation on the ciphertext histogram object sent by the at least one first device to obtain the object deserialization data of the at least one first device. The object serialization data is the deserialized ciphertext histogram. The object deserialization data of the at least one first device is then processed by the second module through homomorphic decryption to obtain the gradient histogram object of the at least one first device.

Images