Method, device and electronic device for constructing multi-classification gradient boosting tree

Abstract

The present application provides a method and device for constructing a multi-classification gradient boosting tree, including: obtaining the predicted value of a training sample carrying a label; based on the predicted value and the corresponding label, determining a multi-dimensional vector for characterizing the gradient corresponding to the training sample, the number of dimensions of the multi-dimensional vector corresponding to the number of label categories of the multi-classification gradient boosting tree; performing privacy protection on the multi-dimensional vector, obtaining intermediate data, and sending the intermediate data to a second participant device; receiving splitting point data sent by the second participant device, the splitting point data being obtained by the second participant device constructing the corresponding splitting point of the multi-classification gradient boosting tree based on the intermediate data; determining the global splitting point of the multi-classification gradient boosting tree based on the splitting point data, and constructing the multi-classification gradient boosting tree based on the global splitting point. In this way, by performing multi-classification tasks through a multi-classification gradient boosting tree capable of multiple outputs, the overhead in the multi-classification task is effectively reduced and the training efficiency is improved.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to a method, device, electronic device, storage medium and computer program product for constructing a multi-classification gradient boosting tree. Background Art

[0002] In the field of federated learning of artificial intelligence, a model is trained jointly by data providers and data users, and then each data user makes corresponding predictions based on their own model. Under the general trend of gradually strengthening data privacy protection in various industries, federated learning, a technology that can establish machine learning by coordinating multi-party data while protecting data privacy, has become one of the keys to cooperation between enterprises / industries. Today, the federated ladder tree model: multi-classification gradient boosting tree, random forest, and classified multi-classification gradient boosting tree have been widely used in vertical federation scenarios and have become one of the commonly used and powerful algorithms in the fields of finance and risk control. In this field, classification tasks are used most frequently.

[0003] In related technologies, the vertical federated multi-classification gradient boosting tree adopts a multi-classification modeling strategy consistent with xgboost or lightgbm, that is, during the training process of the model, a binary classification tree is trained separately for the first-order and second-order derivatives of each class. However, when the number of label categories in the data increases, such as 10 or 20 categories, the training cost will become 10 or 20 times the original, resulting in high training cost and low training efficiency. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device, computer-readable storage medium and computer program product for constructing a multi-classification gradient boosting tree. It is not necessary to construct a multi-classification gradient boosting tree specifically for each label category. Instead, a multi-classification gradient boosting tree capable of multiple outputs is used to perform multi-classification tasks, thereby reducing the overhead in multi-classification tasks.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The embodiment of the present application provides a method for constructing a multi-classification gradient boosting tree, based on a federated learning system, wherein the federated learning system includes a first participant device having a label, and at least one second participant device, and the method is applied to the first participant device, including:

[0007] Get the predicted value of the training sample with label;

[0008] Based on the predicted values ​​and the corresponding labels, determining a multidimensional vector for characterizing the gradient corresponding to the training sample, wherein the number of dimensions of the multidimensional vector corresponds to the number of label categories of the multi-classification gradient boosting tree;

[0009] Performing privacy protection on the multidimensional vector to obtain intermediate data, and sending the intermediate data to a second participant device;

[0010] Receiving splitting point data sent by a second participant device, where the splitting point data is obtained by the second participant device constructing a splitting point corresponding to the multi-classification gradient boosting tree based on the intermediate data;

[0011] Based on the splitting point data, a global splitting point of the multi-classification gradient boosting tree is determined, and based on the global splitting point, the multi-classification gradient boosting tree is constructed.

[0012] The present application embodiment provides a device for constructing a multi-classification gradient boosting tree, comprising:

[0013] An acquisition module is used to obtain the predicted value of the training sample carrying the label;

[0014] A determination module, configured to determine, based on the predicted value and the corresponding label, a multidimensional vector for characterizing the gradient corresponding to the training sample, wherein the number of dimensions of the multidimensional vector corresponds to the number of label categories of the multi-classification gradient boosting tree;

[0015] A privacy protection module, used to perform privacy protection on the multidimensional vector, obtain intermediate data, and send the intermediate data to a second participant device;

[0016] A receiving module, configured to receive splitting point data sent by a second participant device, wherein the splitting point data is obtained by the second participant device constructing a splitting point corresponding to the multi-classification gradient boosting tree based on the intermediate data;

[0017] A construction module is used to determine a global splitting point of the multi-classification gradient boosting tree based on the splitting point data, and to construct the multi-classification gradient boosting tree based on the global splitting point.

[0018] In the above scheme, the determination module is also used to compare the predicted value with the corresponding label to obtain a comparison result; based on the comparison result, the first multidimensional vector and the second multidimensional vector of the training sample are determined; wherein the first multidimensional vector includes the first-order gradient of each label category of the training sample, and the second multidimensional vector includes the second-order gradient of each label category of the training sample.

[0019] In the above scheme, the determination module is also used to determine the compression control parameters of the multidimensional vector and the modeling parameters of the multi-classification gradient boosting tree; wherein the compression control parameters include: the maximum number of homomorphic integer digits, the fixed-point precision of floating-point numbers and the number of samples; based on the compression control parameters and the modeling parameters, the compression parameters are determined; based on the compression parameters, the multidimensional vector is compressed to obtain a compressed multidimensional vector; the privacy protection of the multidimensional vector to obtain intermediate data includes: privacy protection of the compressed multidimensional vector to obtain intermediate data.

[0020] In the above scheme, the compression parameters include: the placeholder of the gradient corresponding to each dimension in the multi-dimensional vector, the total placeholder of a label in the multi-classification gradient boosting tree, the number of labels accommodated, the number of ciphertexts, the number of unaccommodated labels, the placeholder of unaccommodated ciphertext packed integers, and the amount of ciphertext compression; the determination module is also used to determine the floating-point fixed-point accuracy of the multi-classification gradient boosting tree, and based on the floating-point fixed-point accuracy, the gradient corresponding to each dimension in the multi-dimensional vector is fixed-point processed to obtain the fixed-point integer corresponding to the gradient, and based on the number of samples and the fixed-point integer, the placeholder corresponding to the gradient is determined; wherein the gradient of each dimension in the multi-dimensional vector is for The invention relates to a method for calculating the gradient of each label category of the training sample; summing the placeholders of the gradients to determine the total placeholder of a label; performing integer division processing on the maximum number of homomorphic integer digits and the total placeholder of the label to determine the number of labels that can be accommodated; performing integer division processing on the number of label categories and the number of label capacity to determine the number of ciphertexts; performing remainder processing on the number of label categories and the number of label capacity to determine the number of unaccommodated labels; performing product processing on the number of unaccommodated labels and the total placeholder of the label to determine the integer placeholder that cannot be accommodated for ciphertext packing; performing integer division processing on the maximum number of homomorphic integer digits and the total placeholder of the label to determine the amount of ciphertext compression.

[0021] In the above scheme, the privacy protection module is also used to perform homomorphic encryption on the multi-dimensional vector to obtain intermediate data; determining the global splitting point of the multi-classification gradient boosting tree based on the splitting point data includes: decrypting the splitting point data to obtain decrypted splitting point data; and determining the global splitting point of the multi-classification gradient boosting tree based on the decrypted splitting point data.

[0022] In the above scheme, the splitting point data is that the second participant device constructs the splitting point of the multi-classification gradient boosting tree based on the intermediate data to obtain the splitting point construction result, and compresses the splitting point construction result to obtain; the construction module is also used to determine the global splitting point of the multi-classification gradient boosting tree based on the splitting point data, including: decompressing the splitting point data to obtain the splitting point construction result; combining the splitting point construction result and the training sample to determine the global splitting point of the multi-classification gradient boosting tree.

[0023] In the above scheme, the construction module is also used to traverse the splitting points of the multi-classification gradient boosting tree based on the training samples to determine the first splitting point; based on the training samples and the splitting point construction results, traverse the splitting points in the multi-classification gradient boosting tree to determine the second splitting point; compare the first splitting point and the second splitting point to determine the global splitting point of the multi-classification gradient boosting tree.

[0024] In the above scheme, the construction module is also used to traverse the splitting points of the multi-classification gradient boosting tree based on the label of the training sample and the multidimensional vector, and determine the first feature splitting gain histogram corresponding to the first participant device; wherein the first feature splitting gain histogram is used to determine the maximum gain splitting point in the first feature splitting gain histogram; based on the first feature splitting gain histogram, determine the gain value of the splitting point of the multi-classification gradient boosting tree, and select the splitting point with the largest gain value as the first splitting point; based on the label of the training sample and the splitting point construction result, traverse the splitting points of the multi-classification gradient boosting tree, and determine the second feature splitting gain histogram corresponding to the second participant device; wherein the second feature splitting gain histogram is used to determine the maximum gain splitting point in the second feature splitting gain histogram; based on the second feature splitting gain histogram, determine the gain value of the splitting point of the multi-classification gradient boosting tree, and select the splitting point with the largest gain value as the second splitting point.

[0025] In the above scheme, the construction module is also used to obtain the gain value of the first splitting point and the gain value of the second splitting point respectively; compare the gain value of the first splitting point and the gain value of the second splitting point, and select the splitting point corresponding to the larger gain value as the global splitting point of the multi-classification gradient boosting tree.

[0026] In the above scheme, the construction module is also used to obtain the feature value and feature code of the global splitting point; determine the features of the training sample corresponding to the feature code, and determine the feature splitting method in combination with the features and the feature value; and construct the multi-classification gradient boosting tree based on the feature splitting method.

[0027] An embodiment of the present application provides an electronic device, including:

[0028] A memory for storing executable instructions;

[0029] The processor is used to implement the method for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application when executing the executable instructions stored in the memory.

[0030] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the method for constructing a multi-classification gradient boosting tree provided in the embodiment of the present application.

[0031] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for constructing a multi-classification gradient boosting tree provided in the embodiment of the present application.

[0032] The embodiments of the present application have the following beneficial effects:

[0033] Compared with the multi-classification gradient boosting tree in the related art, which requires a multi-classification gradient boosting tree to be specially constructed for each class to perform the classification task, the present application represents the gradient information of multiple categories of the training sample in the form of a multi-dimensional vector based on the predicted value and corresponding label of the training sample, thereby processing the multi-dimensional vector representing the gradient information, and determining the corresponding splitting point data based on the processing result, thereby determining the global splitting point of the multi-classification gradient boosting tree, and then constructing a multi-classification gradient boosting tree. In this way, by performing multi-classification tasks through a multi-classification gradient boosting tree with multiple outputs, the overhead in the multi-classification task is effectively reduced and the training efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of an implementation scenario of a method for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application;

[0035] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0036] Figure 3 It is a flowchart of a method for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application;

[0037] Figure 4It is an optional flowchart of the multi-dimensional vector compression method provided in the embodiment of the present application;

[0038] Figure 5 It is an optional schematic diagram of the plaintext packaging process of the method for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application;

[0039] Figure 6 It is an optional schematic diagram of a process of traversing a multi-classification gradient boosting tree provided in an embodiment of the present application;

[0040] Figure 7 It is an optional schematic diagram of the ciphertext compression process of the method for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application;

[0041] Figure 8 It is an optional schematic diagram of the decompression process of the multi-classification gradient boosting tree provided in an embodiment of the present application;

[0042] Fig. 9 is an optional splitting point splitting schematic diagram of the splitting point provided in the embodiment of the present application;

[0043] Fig.10 This is an optional flowchart of a method for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application;

[0044] Fig.11 It is an optional structural diagram of a device for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0046] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0047] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0049] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0051] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0052] 1) Decision Tree is a machine learning method. It is specifically a tree structure, in which each internal node represents a judgment on an attribute, each branch represents the output of a judgment result, and finally each leaf node represents a classification result.

[0053] 2) Federated machine learning (Federated Learning), also known as federated learning, joint learning, and alliance learning. Federated machine learning is a machine learning framework that can effectively help multiple institutions use data and conduct machine learning modeling while meeting the requirements of user privacy protection, data security, and government regulations. As a distributed machine learning paradigm, federated learning can effectively solve the problem of data silos, allowing participants to jointly model without sharing data, which can technically break data silos and achieve collaborative intelligence.

[0054] Federated Learning (aka Federated Machine Learning) can be divided into three categories: Horizontal Federated Learning, Vertical Federated Learning, and Federated Transfer Learning.

[0055] Among them, horizontal federated learning is also called feature-aligned federated learning (Feature-Aligned Federated Learning), that is, the data features of participants in horizontal federated learning are aligned, which is suitable for the situation where the data features of participants overlap a lot, but the sample IDs (ID, Identity document) overlap less. Vertical federated learning is also called sample-aligned federated learning (Sample-Aligned Federated Learning), that is, the training samples of participants in vertical federated learning are aligned, which is suitable for the situation where the training sample IDs of participants overlap a lot, but the data features overlap less.

[0056] 3) Homomorphic encryption (HE) is a symmetric encryption algorithm proposed by Craig Gentry of IBM in 2009. The purpose of homomorphic encryption is to find an encryption algorithm that can perform addition and multiplication operations on ciphertext, so that the result of a certain operation on the encrypted ciphertext is exactly equal to the ciphertext obtained by performing the expected operation on the plaintext before encryption. Homomorphic encryption effectively ensures that the data processor can directly process the ciphertext of the data accordingly without knowing the plaintext information of the data it processes. This feature of homomorphic encryption ensures that the user's data and privacy can be properly secured. Therefore, homomorphic encryption is applied to many real-world scenarios to ensure data security.

[0057] If an encryption function satisfies both additive homomorphism and multiplicative homomorphism, it is called fully homomorphic encryption. This encryption function can be used to perform various encrypted operations (addition, subtraction, multiplication, division, polynomial evaluation, exponential, logarithmic, trigonometric functions, etc.).

[0058] The inventor found that there are some places with high overhead in the currently widely used vertical tree model: 1) The first participant device (dominant guest) needs to encrypt and transmit the gradient or label statistics, and the second participant device (participant host) needs to accept the homomorphic encrypted ciphertext, then perform calculations, and then return the split point, which contains the encrypted statistical results. The guest needs to further calculate the optimal split point based on this statistical result. In real scenarios, the time consumption of guest homomorphic encryption of labels, host calculation of ciphertext, and public network transmission is very high. Therefore, in practical applications, the encryption calculation, ciphertext calculation and communication time of the federated vertical tree will account for a large part; 2) The vertical federated multi-classification gradient boosting tree adopts a multi-classification modeling strategy consistent with xgboost / lightgbm, that is, the training will train a binary classification tree for the first-order and second-order derivatives of each class separately. However, when the number of classes in the data increases, such as 10 or 20 classes, the training cost will become 10 or 20 times the original, and this level of cost is unbearable. Therefore, there is a lot of room for optimization at these two points.

[0059] Based on this, the embodiments of the present application provide a method, device, electronic device, computer-readable storage medium and computer program product for constructing a multi-classification gradient boosting tree. It is not necessary to construct a multi-classification gradient boosting tree specifically for each label category. Instead, a multi-classification gradient boosting tree with multiple outputs is used to perform multi-classification tasks, thereby reducing the overhead in multi-classification tasks. At the same time, it can effectively reduce the amount of data transmission and processing during the construction of the multi-classification gradient boosting tree, thereby improving the construction efficiency of the multi-classification gradient boosting tree.

[0060] Based on the above explanations of the nouns and terms involved in the embodiments of the present application, the following describes the implementation scenario of the multi-classification gradient boosting tree construction method based on the federated learning system provided in the embodiments of the present application. Figure 1 , Figure 1 It is a schematic diagram of an implementation scenario of a multi-classification gradient boosting tree construction method based on a federated learning system provided in an embodiment of the present application. To support an exemplary application, a first participant device 200-1 is connected to a second participant device 200-2 via a network 300, wherein the first participant device 200-1 may be an active device storing user tags, such as a bank, and the second participant device 200-2 may be a passive device storing user feature data, such as a shopping mall or a supermarket. The first participant device 200-1 and the second participant device 200-2 assist each other in performing federated learning so that the first participant device 200-1 obtains a multi-classification gradient boosting tree. The network 300 may be a wide area network or a local area network, or a combination of the two, and data transmission is achieved using wireless or wired links.

[0061] The first participant device (including the first participant device 200-1) is used to obtain the predicted value of the training sample carrying the label; based on the predicted value and the corresponding label, determine the multidimensional vector used to characterize the gradient corresponding to the training sample, wherein the number of dimensions of the multidimensional vector corresponds to the number of label categories of the multi-classification gradient boosting tree; perform privacy protection on the multidimensional vector, obtain intermediate data, and send the intermediate data to the second participant device;

[0062] The second participant device (including the second participant device 200-2) is used to obtain the intermediate data sent by the first participant device; based on the intermediate data, construct the splitting point of the corresponding multi-classification gradient boosting tree to obtain the splitting point data; and send the splitting point data to the first participant device.

[0063] The first participant device (including the first participant device 200-1) is also used to receive the splitting point data sent by the second participant device; based on the splitting point data, determine the global splitting point of the multi-classification gradient boosting tree, and construct the multi-classification gradient boosting tree based on the global splitting point.

[0064] The multi-classification gradient boosting tree in the embodiment of the present application can be applied to risk control management, such as realizing multi-classification prediction of the user's credit level (such as excellent credit, average credit, and extremely poor credit) through the constructed multi-classification gradient boosting tree, and then executing risk control for the user (such as the amount of the loan) in combination with the prediction result; for another example, the multi-classification gradient boosting tree constructed can be used to predict and score the risk level of the project (such as high risk, medium risk, and low risk), and then executing risk control for the project (implement or not) in combination with the prediction result.

[0065] In actual applications, the first participant device 200-1 and the second participant device 200-2 can be independent physical servers, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The first participant device 200-1 and the second participant device 200-2 can also be smart phones, tablet computers, laptops, desktop computers, smart speakers, smart watches, etc., but are not limited to them. The first participant device 200-1 and the second participant device 200-2 can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0066] The hardware structure of the electronic device for implementing the method for constructing a multi-classification gradient boosting tree provided in the embodiment of the present application is described in detail below. The electronic device includes but is not limited to a server or a terminal. Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 2 The electronic device 200 shown includes: at least one processor 210, a memory 250, at least one network interface 220 and a user interface 230. The various components in the electronic device 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 240 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 240 is not described in detail. Figure 2 Various buses are labeled as bus system 240 .

[0067] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0068] The user interface 230 includes one or more output devices 231 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 230 also includes one or more input devices 232, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0069] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices that are physically remote from the processor 210.

[0070] The memory 250 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 250 described in the embodiments of the present application is intended to include any suitable type of memory.

[0071] In some embodiments, memory 250 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0072] Operating system 251, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0073] A network communication module 252, used to reach other computing devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB);

[0074] The input processing module 253 is used to detect one or more user inputs or interactions from one of the one or more input devices 232 and translate the detected inputs or interactions.

[0075] In some embodiments, the multi-classification gradient boosting tree construction device provided in the embodiments of the present application can be implemented in software. Figure 2 A construction device 254 of a multi-classification gradient boosting tree stored in a memory 250 is shown, which can be software in the form of a program and a plug-in, including the following software modules: an acquisition module 2541, a determination module 2542, a privacy protection module 2543, a receiving module 2544 and a construction module 2545. These modules are logical, and therefore can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0076] In other embodiments, the multi-classification gradient boosting tree construction device provided in the embodiments of the present application can be implemented in a combination of software and hardware. As an example, the multi-classification gradient boosting tree construction device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the multi-classification gradient boosting tree construction method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs) or other electronic components.

[0077] Based on the above description of the implementation scenario and electronic device of the method for constructing a multi-classification gradient boosting tree in the embodiment of the present application, the method for constructing a multi-classification gradient boosting tree provided in the embodiment of the present application is described below. Figure 3 , Figure 3 : is a flow chart of a method for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application. The method for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application includes:

[0078] Step 101: A first participant device obtains a prediction value of a training sample carrying a label.

[0079] Here, the training sample may be a user shared by the first participant device and the second participant device. The first participant device serves as a label holder and feature provider of the training sample, and the second participant device serves only as a feature provider of the training sample.

[0080] Exemplarily, when the multi-classification gradient boosting tree is applied to risk control management or risk control prediction, the label of the training sample can be a multi-classification label that identifies the user's credit level, a multi-classification label that identifies the user's loyalty level, or a multi-classification label used to evaluate the user's quality level. Here, the first participant is a platform that performs credit assessment on the user, for example, a financial loan platform (such as a bank), and the second participant can be a platform that holds the user's feature data, such as various types of consumer platforms (for example, shopping malls, supermarkets, online shopping platforms, game platforms, mobile application stores, etc.), or Internet companies, etc. Among them, the user's feature data may include but is not limited to the user's basic information, business data, behavior data, and device information. Basic information includes but is not limited to the user's name, gender, age, education, occupation, mobile phone number, ID number, address, income data (for example, annual salary, monthly salary), etc. Business data includes data under multiple business scenarios. For example, if the business scenario is loan business, business data includes loan data and repayment data, etc. Loan data includes loan amount, loan time and loan institution (for example, xx bank), etc. Repayment data includes repayment amount, repayment time, repayment institution and overdue repayment data (for example, number of overdue payments and number of overdue days), etc. For example, if the business scenario is consumption scenario, business data includes consumption purpose (for example, education consumption, shopping consumption, medical consumption, life consumption, etc.), consumption amount, consumption time, consumption institution (for example, shopping malls, supermarkets, convenience stores, online shopping stores, etc.), consumption method (for example, credit card consumption or savings card consumption, etc.) and the address of the user at the time of consumption, etc. Behavior data can be, for example, the payment institution commonly used by users (for example, WeChat payment, bank card payment, etc.), the payment method commonly used by users (for example, QR code payment method, card payment method) and operation frequency (for example, the frequency of payment operations within a period of time), etc. Device information includes the model of the device commonly used by users, the address of the device when the user behavior is implemented, etc.

[0081] It should be noted that, before using the training samples for model training, the embodiment of the present application also performs feature binning on the data of the training samples, for example, binning the age feature into less than 30 years old, 30-50 years old, and greater than 50 years old, etc. The model obtained by feature binning training is more interpretable.

[0082] Step 102: Based on the predicted values ​​and the corresponding labels, a multidimensional vector is determined for characterizing the gradient corresponding to the training sample, wherein the number of dimensions of the multidimensional vector corresponds to the number of label categories of the multi-classification gradient boosting tree.

[0083] In actual implementation, after obtaining the data labels corresponding to the feature binned data and the predicted values ​​of the samples, two multidimensional vectors of the gradients corresponding to the samples are constructed based on the data labels and the predicted values ​​of the samples. Here, the process of constructing two multidimensional vectors of the gradients corresponding to the samples is to compare the predicted values ​​with the corresponding labels to obtain the comparison results; based on the comparison results, the first multidimensional vector and the second multidimensional vector of the training samples are determined; wherein the first multidimensional vector includes the first-order gradient of each label category of the training samples, and the second multidimensional vector includes the second-order gradient of each label category of the training samples.

[0084] In some embodiments, the prediction value of the first generation of trees is directly assigned to 0, and then when constructing each generation of multi-classification gradient boosting trees, for each sample, the predicted value and label of the previous t-1 generation of trees are used to calculate the gradient, and two d-dimensional vectors will be obtained. The first d-dimensional vector g corresponds to the first-order gradient of each label category, and the second d-dimensional vector h corresponds to the second-order gradient of each label category. Exemplarily, a total of t generations of trees need to be constructed, where t is a positive integer greater than or equal to 1. Among them, the height of each multi-classification gradient boosting tree is the same, the first participating device counts the number of labels, and calculates the first-order gradient g and second-order gradient h corresponding to each label based on the local label and the predicted value of the previous t-1 generation of trees; based on the calculated first-order gradient and second-order gradient, the first d-dimensional vector g[g1, g2, ..., g d ] and the second d-dimensional vector h[h1,h2,…,h d ].

[0085] In actual implementation, after determining the two multidimensional vectors, the prediction value of the current node can be determined. In some embodiments, we use the i subscript to represent all samples in a leaf node, then the prediction value of the node is w*, where the prediction value of each dimension is calculated based on the two d-dimensional vectors according to the following formula (1). After obtaining the prediction values ​​of d dimensions, the prediction values ​​of d dimensions are compared and the largest prediction value is selected as the current prediction value.

[0086]

[0087] As an example, here we take a three-class multi-classification gradient boosting tree as an example, that is, d here is 3, then the first 3-dimensional vector g is [g1, g2, g3], the second 3-dimensional vector h is [h1, h2, h3], and the predicted values ​​of each dimension are:

[0088]

[0089] Then compare as well as The largest value is selected as the predicted value of the current node.

[0090] Step 103: Perform privacy protection on the multi-dimensional vector to obtain intermediate data, and send the intermediate data to the second participant device.

[0091] In actual implementation, after obtaining the multidimensional vector, the first participant device homomorphically encrypts the multidimensional vector, and sends the homomorphically encrypted multidimensional vector to the second participant device, so that the second participant device constructs the splitting points of the corresponding multi-classification gradient boosting tree based on the homomorphically encrypted multidimensional vector to obtain the splitting point data. In this way, privacy protection is achieved through the encryption operation of the multidimensional vector, and the security of the data transmission process is improved.

[0092] It should be noted that, in some embodiments, before performing privacy protection on the multidimensional vector, the multidimensional vector is also compressed.

[0093] Here, for the process of compressing multidimensional vectors, see Figure 4 , Figure 4 is an optional flow chart of the multidimensional vector compression method provided in the embodiment of the present application, based on Figure 3 Before step 102, the method for constructing a multi-classification gradient boosting tree provided in the embodiment of the present application further includes:

[0094] Step 201, determine the compression control parameters of the multi-dimensional vector and the modeling parameters of the multi-classification gradient boosting tree.

[0095] Here, the compression control parameters include: the maximum number of homomorphic integer digits, the fixed-point precision of floating-point numbers, and the number of samples, and the modeling parameters include the number of categories of the multi-classification gradient boosting tree, that is, the number of label categories.

[0096] Step 202: Determine compression parameters based on compression control parameters and modeling parameters.

[0097] Here, the compression parameters include the first-order gradient g placeholder corresponding to each label category in the first multidimensional vector, the second-order gradient h placeholder corresponding to each label category in the second multidimensional vector, the total placeholder of a label, the number of labels accommodated, the number of ciphertexts, the number of unaccommodated labels, the placeholder of unaccommodated ciphertext packed integers, and the ciphertext compression amount.

[0098] Next, the process of determining the first-order gradient g placeholder, the second-order gradient h placeholder, and the total placeholder of a label is described.

[0099] In some embodiments, the floating-point fixed-point accuracy of the multi-classification gradient boosting tree is first determined, and based on the floating-point fixed-point accuracy, the gradient corresponding to each dimension in the multi-dimensional vector is fixed-point processed to obtain the fixed-point integer corresponding to the gradient, and based on the number of samples and the fixed-point integer, the occupancy of the corresponding gradient is determined, that is, the first-order gradient g occupancy and the second-order gradient h occupancy; wherein the gradient of each dimension in the multi-dimensional vector is the gradient of each label category of the corresponding training sample; then, the gradient occupancy is summed, that is, the first-order gradient g occupancy and the second-order gradient h occupancy are summed to determine the total occupancy of a label.

[0100] Exemplarily, the first participant device reads the compression control parameters through the compression planning module, determines the maximum number of homomorphic integer digits, the fixed-point precision of floating-point numbers, and the number of samples, and then determines the first-order gradient g and the second-order gradient h corresponding to the sample based on the sample gradient, so as to perform fixed-point processing on the first-order gradient g and the second-order gradient h, and obtain the fixed-point integers corresponding to the first-order gradient g and the second-order gradient h, that is, the first-order gradient g integer = int(g*2^ (定点化精度) ), second-order gradient h integer = int(h*2^ (定点化精度) ), and the second-order gradient h itself is between [0, 1], so there is no need to do fixed-point conversion. Then consider the maximum value of the sum of the integer part of the first-order gradient g and the integer part of the second-order gradient h to ensure that the number of bits will not overflow. The number of samples is multiplied by the fixed-point integer to determine the corresponding sample gradient position, that is, the first-order gradient g position = the number of bits (int (g_max*2^ (定点化精度) )*number of samples), second-order gradient h occupies = number of bits (int(h_max*2^ (定点化精度) )*number of samples), and then based on the first-order gradient g occupancy and the second-order gradient h occupancy, determine the total occupancy of a label, which is the sum of the first-order gradient g occupancy and the second-order gradient h occupancy.

[0101] Here, if the total number of occupancy is less than the maximum number of homomorphic integer digits, the algorithm can continue, otherwise the algorithm stops and informs that the number of digits is insufficient.

[0102] It should be noted that, because the first-order gradient g may be negative, if the integer after plaintext packaging is negative, the ciphertext compression by addition and multiplication cannot obtain the correct result. Therefore, before the first-order gradient g is fixed-point processed, in order to correctly execute the ciphertext compression operation of the gradient boosting multi-classification gradient boosting tree, the first-order gradient g should be offset to a positive number. In some embodiments, the first-order gradient g in the classification task is between [-1, 1], so it is necessary to first add 1 to g to offset it to between [0, 2].

[0103] As an example, here we take 53 precision bits, 1000000 samples, and 2046 as the maximum number of homomorphic integer bits. The value of g after offset is 1.2312, then the fixed-point conversion is g integer = int (1.2312*2^ 53 )=11089663722437110, then calculate that g occupancy=bit length(18014398509481984000000)=74 bits, h occupancy=bit length(9007199254740992000000)=73 bits, then calculate that the total occupancy of a label=73+74=147.

[0104] Next, the process of determining the number of tags and the number of ciphertexts is described.

[0105] The number of labels here is the number of ciphertexts required to pack all classes of plaintexts.

[0106] In some embodiments, the number of labels that can be accommodated here, that is, the number of labels that a ciphertext can accommodate, is the result of integer division of the integer bits of the homomorphic ciphertext and the total occupancy of a label, and the number of ciphertexts is the result of integer division of the number of label categories and the number of labels that can be accommodated and then rounded up.

[0107] Continuing with the above example, the integer number of homomorphic ciphertext bits here is 2047, the total number of bits occupied by a label is 147, and the number of label categories is, for example, 16, then the number of labels accommodated = 2047 / / 147 = 13, and the number of ciphertexts = 16 / / 13 = 2.

[0108] Next, the number of unaccommodated tags, the number of unaccommodated ciphertext packing integers, and the amount of ciphertext compression are explained.

[0109] In some embodiments, the number of unaccommodated tags here is the result of dividing the number of tag categories and the number of tags accommodated, the unaccommodated ciphertext packing integer placeholders is the product of the number of unaccommodated tags and the total placeholders of a tag, and the ciphertext compression amount is the result of integer division of the homomorphic ciphertext integer places and the unaccommodated ciphertext packing integer placeholders.

[0110] Continuing with the above example, the integer number of homomorphic ciphertext bits here is 2047, the total number of bits occupied by a label is 147, and the number of label categories is 16. The number of unaccommodated labels = 16%13=3, the number of unaccommodated ciphertext packing integer bits = 3*147=441, and the ciphertext compression amount = 2047 / / 441=4.

[0111] After obtaining the compression parameters including the first-order gradient g placeholder, the second-order gradient h placeholder, the total placeholder of a label, the number of labels that can be accommodated, the number of ciphertexts, the number of unaccommodated labels, the placeholder of unaccommodated ciphertext packed integers, and the ciphertext compression amount, the first participant device sends these compression parameters to the second participant device.

[0112] Continuing with the above example, the first participant device sends the g position 74, the h position 73, the total position of a label 147, the number of label categories 13, the number of ciphertexts 2, the number of unaccommodated labels 3, the unaccommodated ciphertext packaged integer position 441, and the ciphertext compression amount 4 to the second participant device.

[0113] Step 203: compress the multidimensional vector based on the compression parameter to obtain a compressed multidimensional vector.

[0114] In actual implementation, the first participant device calls the plaintext compression module to compress the multidimensional vector. The process is divided into two parts. The first part is to compress the first multidimensional vector g[g1, g2, ..., g d ] and the second multidimensional vector h[h1, h2, ..., h d ] is compressed to obtain the vector combination g_h[g1h1, g2h2, …, g d h d ]; The second part is to compress the vector combination g_h of each sample again to obtain a multi-dimensional vector combination.

[0115] Next, the process of calling the plaintext compression module on the first participant device to compress the two multi-dimensional vectors g and h to obtain the vector combination g_h corresponding to each sample is described.

[0116] In some embodiments, for each dimension of the multidimensional vector g of each sample, 1 is first added to offset it to a positive number, and then the g of each dimension of the multidimensional vector g and the h of each dimension of the multidimensional vector h are fixed-point converted to obtain fixed-point integers corresponding to g and h of each dimension, and g is shifted left by the bit length of the fixed-point integer corresponding to h to obtain a displacement integer, and the displacement integer is added to h to obtain a gradient combination gh corresponding to each dimension, so that based on the gradient combination gh corresponding to each dimension, after determining the gradient combination of each dimension, a packed vector combination g_h is obtained.

[0117] Continuing with the above example, refer to Figure 5 , Figure 5This is an optional schematic diagram of the plaintext packing process of the method for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application. Here, g is fixed-pointed to an integer with a bit length of 74 bits, h is fixed-pointed to an integer with a bit length of 73 bits, g is shifted left by 73 bits, and h with a bit length of 73 after fixed-pointing is added to obtain a gradient combination gh corresponding to each dimension with a bit length of 147, thereby determining that each dimension after packing has a vector combination g_h of 147 bits.

[0118] Next, the plaintext compression module is called on the first participant device to compress the vector combination g_h of each sample again to obtain the process of multi-dimensional vector combination.

[0119] It should be noted that the dimension of the multi-dimensional vector combination here is related to the number of ciphertexts. The process of compressing the vector combination g_h also needs to be divided into two parts, namely, compressing the data of some dimensions in the vector combination g_h whose dimensions are not greater than the number of labels accommodated, and compressing the data of some dimensions whose dimensions are greater than the number of labels accommodated, that is, the data of some dimensions that do not accommodate the number of labels.

[0120] Next, the process of compressing data of some dimensions whose number of dimensions is not greater than the number of labels accommodated in the vector combination g_h is described.

[0121] Here, the number of dimensions to be compressed is determined according to the number of label accommodation in the compression parameter, that is, the gradient combinations of the dimensions corresponding to the number of label accommodation are concatenated in bit length, that is, compressed into a large integer. In some embodiments, assuming that it is a d-dimensional vector combination g_h, the large integer = {(the first gradient combination gh*2^ 总占位 )+the second gradient combination gh}*2^ 总占位 ...+the dth gradient combination gh.

[0122] Continuing with the above example, refer to Figure 5 , where the number of label classes is 13, the 13 gradient combinations are compressed into a large integer 1 with a bit length of 1911 (i.e. 13*147) bits, specifically, the large integer 1 = {(the first gradient combination gh*2^ 147 )+the second gradient combination gh}*2^ 147 ...+thirteenth gradient combination gh.

[0123] Next, the process of compressing data of some dimensions whose number of dimensions is greater than the number of labels that can be accommodated, that is, some dimensions that cannot accommodate the number of labels, is described.

[0124] Here, according to the number of unaccommodated labels in the compression parameter, it is determined that the gradient combination of the dimension corresponding to the number of unaccommodated labels in the vector combination needs to be concatenated in bit length, that is, compressed into a large integer, and the bit length of the large integer is the placeholder for the unaccommodated ciphertext packed integer. The compression process here is the same as the above compression process, which will not be described in detail here.

[0125] Continuing with the above example, refer to Figure 5 , where the last number of labels is 3, the three gradient combinations are compressed into a large integer 2 with a bit length of 441 (i.e. 3*147) bits, specifically, the large integer 2 = {(the eleventh gradient combination gh*2^ 147 )+the twelfth gradient combination gh}*2^ 147 +Thirteenth gradient combination gh.

[0126] Here, based on two compression processes, a combination of multidimensional vectors consisting of two large integers is determined.

[0127] It should be noted that when the number of ciphertexts is greater than 2, the process of compressing the data of some dimensions in the vector combination g_h whose number of dimensions is not greater than the number of label capacity based on the number of label capacity and the total label space is the same. For example, when the number of ciphertexts is 3, the multidimensional vector combination is a three-dimensional vector combination, and the process of determining the data of the first dimension of the three-dimensional vector combination is the same as the process of determining the data of the second dimension.

[0128] In this way, by compressing the transmitted data during the interaction between the first participant device and the second participant device, the amount of data sent from the first participant device to the second participant is reduced, the overhead of data transmission is reduced, and at the same time, the amount of data encryption in subsequent homomorphic encryption is reduced, thereby improving data processing efficiency.

[0129] In actual implementation, after obtaining the multidimensional vector combination, the multidimensional vector combination is homomorphically encrypted to obtain an encrypted multidimensional vector combination and send it to the second participant device.

[0130] Continue to refer to Figure 5 , here the large integer 1 and the large integer 2 are homomorphically encrypted respectively, and an encrypted multi-dimensional vector combination consisting of ciphertext 1 and ciphertext 2 is obtained.

[0131] Step 104: Receive splitting point data sent by the second participant device.

[0132] It should be noted that since the intermediate data can be an uncompressed multidimensional vector or a compressed multidimensional vector, the splitting point data here can be obtained by the second participant device constructing the splitting point of the corresponding multi-classification gradient boosting tree based on the encrypted multidimensional vector, or can be obtained by constructing the splitting point of the corresponding multi-classification gradient boosting tree based on the encrypted multidimensional vector combination.

[0133] Next, the situation in which the splitting point data is obtained by the second participant device constructing the splitting point of the corresponding multi-classification gradient boosting tree based on the encrypted multi-dimensional vector combination is explained.

[0134] In some embodiments, based on Figure 3 Step 104 can also be implemented in the following manner: the first participant device receives the splitting point data sent by the second participant device, the splitting point data is, the second participant device constructs the splitting point of the multi-classification gradient boosting tree based on the encrypted multi-dimensional vector combination to obtain the splitting point construction result, and compresses the splitting point construction result.

[0135] In actual implementation, after receiving the encrypted multidimensional vector combination sent by the first participant device, the second participant device traverses the splitting points of the multi-classification gradient boosting tree based on the encrypted multidimensional vector combination. For example, see Figure 6 , Figure 6 This is an optional schematic diagram of the process of traversing the multi-classification gradient boosting tree provided in the embodiment of the present application. Assuming the sample data is (24 years old, credit 710, income 25,000), the second participant device traverses according to the sample data Figure 6 After the multi-classification gradient boosting tree is shown, the encrypted split point construction result marked as A1 is obtained.

[0136] Here, the encrypted split point construction result is that the second participant device traverses the split points based on the local features and the received encrypted multidimensional vector combination, and obtains the second encrypted multidimensional vector combination for each split point, and the second encrypted multidimensional vector combination is in the same form as the received encrypted multidimensional vector

[0137] It should be noted that after the second participant device obtains the splitting point construction result based on the encrypted multi-dimensional vector combination, the second participant device can send the splitting point construction result to the first participant device, or can also call the ciphertext compression module to compress the encrypted splitting point construction result based on the received compression parameter to obtain a large encrypted integer.

[0138] Next, the process of calling the ciphertext compression module on the second participant device and compressing the encrypted split point construction result based on the received compression parameters to obtain a large encrypted integer is described.

[0139] Here, it should be noted that, for the process of determining the encrypted multidimensional vector combination, when compressing the data of the partial dimensions whose dimensionality is not greater than the number of labels that can be accommodated in the vector combination g_h, the compression space is fully used (the packed gh occupies all the integer space), but the data of the partial dimensions whose dimensionality is greater than the number of labels that can be accommodated may still have room for compression, therefore, the second encrypted multidimensional vector combination having the same form as the encrypted multidimensional vector is divided into compressible ciphertext (i.e., the data of the partial dimensions whose dimensionality is not greater than the number of labels that can be accommodated) and incompressible ciphertext (i.e., the data of the partial dimensions whose dimensionality is greater than the number of labels that can be accommodated). Therefore, the compression processing process of the encrypted split point construction result, i.e., the second encrypted multidimensional vector combination, by the second participant device, is only to compress the compressible ciphertext part of the second encrypted multidimensional vector combination, and only perform ordinary statistical processing on the incompressible ciphertext part of the second encrypted multidimensional vector combination.

[0140] It should be noted that, since there may be multiple dimensions in the encrypted multi-dimensional vector combination except the last dimension, there may also be multiple corresponding incompressible ciphertexts in the encrypted multi-dimensional vector combination.

[0141] Next, the process of compressing the compressible ciphertext of the second encrypted multi-dimensional vector combination is described.

[0142] In some embodiments, the compression parameters include the unaccommodated ciphertext packing integer position and the ciphertext compression amount. For each split point, the unaccommodated ciphertext is taken out to form a ciphertext list of length n. Assuming that the ciphertext compression amount is k and the unaccommodated ciphertext packing integer position is p, k ciphertexts are taken out from n in a loop without replacement. If there are less than k ciphertexts, all the remaining ciphertexts are taken out. If n ciphertexts have been processed, the first ciphertext taken out in each cycle is used as the first summary result. Starting from the second ciphertext to the kth ciphertext, the final summary result = ((k-1)th summary result * 2^ p )+kth ciphertext, save the final summary result, if n ciphertexts have been processed, jump out of the loop; after processing all the summary results, multiple final summary results are obtained, and then the multiple final summary results are made into a compressed ciphertext list, and then the compressed ciphertext list is saved.

[0143] Here is the reference Figure 7 , Figure 7 It is an optional schematic diagram of the ciphertext compression process of the method for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application. After obtaining the second encrypted multidimensional vector combination, the ciphertext of the last dimension of the second encrypted multidimensional vector combination, i.e., ciphertext 4, is compressed.

[0144] Continuing with the above example, the ciphertext compression amount here is 4, and the number of places not accommodating the ciphertext packing integer is 441. Assuming that there are 40 second encrypted multidimensional vector combinations, that is, there are 40 compressible parts, that is, 40 ciphertexts 4, then the 40 ciphertexts 4 are compressed into one ciphertext compression result, that is, the compression process is repeated 10 times, and 10 final summary results are calculated. For the calculation process of each final summary result, specifically, the final summary result = {(first ciphertext * 2^ 441 )+second ciphertext}*2^ 441 ...+ the fourth ciphertext. Here, after obtaining 10 final summary results, the 10 final summary results are made into a compressed ciphertext list and saved.

[0145] Next, the process of performing ordinary statistical processing on the incompressible ciphertext of the second encrypted multi-dimensional vector combination is described.

[0146] In some embodiments, for each split point, after the last ciphertext is taken out, the ciphertexts that have not been taken out are organized into an uncompressed ciphertext list and saved.

[0147] Continue to refer to Figure 7 , the ciphertext that has not been taken out here is ciphertext 3, so the uncompressed ciphertext list is a list consisting of ciphertext 3.

[0148] It should be noted that, since there may be multiple corresponding incompressible ciphertexts in the second encrypted multi-dimensional vector combination, there may also be multiple corresponding uncompressed ciphertext lists.

[0149] In actual implementation, after obtaining the uncompressed ciphertext list and the compressed ciphertext list, the second participant device sends the compressed ciphertext list, the uncompressed ciphertext list and the statistical number of samples to the first participant device.

[0150] Step 105, based on the splitting point data, determine the global splitting point of the multi-classification gradient boosting tree, and construct the multi-classification gradient boosting tree based on the global splitting point.

[0151] In actual implementation, since the splitting point data may be compressed by the second participant device, or may not be compressed by the second participant, the first participant device determines the global splitting point of the multi-classification gradient boosting tree based on the splitting point data into two cases.

[0152] In some embodiments, when the splitting point data is not compressed by the second participant, the first participant device decrypts the splitting point data to obtain the decrypted splitting point construction result; combines the splitting point construction result and the training sample to determine the global splitting point of the multi-classification gradient boosting tree.

[0153] In other embodiments, when the splitting point data is compressed by the second participant, the first participant device first decompresses the splitting point data to obtain a splitting point construction result; and combines the splitting point construction result and the training sample to determine the global splitting point of the multi-classification gradient boosting tree.

[0154] Next, a process is described in which, when the splitting point data is compressed by the second participant, the first participant device decompresses the splitting point data to obtain a splitting point construction result.

[0155] It should be noted that, since the second participant device only compresses the compressible ciphertext in the second encrypted multidimensional vector, the decompression process of the first participant device also only decompresses the compressed compressible ciphertext in the second encrypted multidimensional vector.

[0156] Here, before the first participant device performs decompression processing, it is necessary to use a decompression module to decrypt the compressed ciphertext list and the uncompressed ciphertext list to obtain a compressed plaintext list and a plaintext list.

[0157] Here, firstly, the process of the first participant device decrypting the uncompressed ciphertext list to obtain the plaintext list is explained.

[0158] In some embodiments, the first participant device decrypts all ciphertexts in the uncompressed ciphertext list to obtain multiple incompressible plaintexts and saves them to obtain a plaintext list.

[0159] It should be noted that the incompressible plaintext here is the result of decrypting the incompressible ciphertext in the encrypted multidimensional vector combination, and the number of incompressible plaintext here is the same as the number of the second encrypted multidimensional vector combination.

[0160] Continuing with the above example, see Figure 8 , Figure 8 This is an optional schematic diagram of the decompression process of the multi-classification gradient boosting tree provided in an embodiment of the present application. Here, there are 40 second encrypted multidimensional vector combinations. Then, when the uncompressed ciphertext list composed of the ciphertext 3 is decrypted, 40 large integers 1 with a bit length of 1911 bits will be obtained, and the 40 large integers 1 are saved to obtain a plaintext list.

[0161] It should be noted that, since there may be multiple corresponding uncompressed ciphertext lists, there may also be multiple corresponding plaintext lists obtained by decryption.

[0162] Next, the process of decrypting the compressed ciphertext list to obtain the compressed plaintext list is described.

[0163] In some embodiments, the first participant device decrypts all ciphertexts in the compressed ciphertext list to obtain multiple compressed plaintexts and save them to obtain a compressed plaintext list.

[0164] It should be noted that the compressed plaintext here is the result of compressing the compressible ciphertext in the encrypted multi-dimensional vector combination and then decrypting it. The number of compressed plaintexts here is the same as the number of the final summary results.

[0165] Continuing with the above example, see Figure 8 , where the ciphertext compression amount is 4, there are a total of 40 second encrypted multidimensional vector combinations, and the final summary result is 10. Then, decrypting the compressed ciphertext list composed of ciphertext 4 will obtain 10 integers with a bit length of 1764 bits, and then save the 10 integers to obtain a compressed plaintext list.

[0166] Next, the process of decompression processing on the first participant's device is explained.

[0167] Here, the first participant device performing decompression processing means that the first participant device performs decompression processing on the compressed plaintext list.

[0168] In some embodiments, for i=0; i<compression number; i++;

[0169] Mask = 2^ 未容纳密文的打包整数占位 -1;

[0170] Statistical result = plain text & mask;

[0171] The plaintext shifted right occupies a total of bits;

[0172] Save the statistical results to the decompressed list;

[0173] Here, at each traversal, the following processing is performed to determine mask = 2^ 未容纳密文的打包整数占位 -1, then determine the statistical result = plaintext & mask, where the plaintext is the compressed plaintext obtained by decryption, and then move the plaintext to the right by the length of the packed integer that does not accommodate the ciphertext to obtain the statistical result. Here, the statistical result is the large integer of the last dimension in the second encrypted multidimensional vector combination, that is, the compressible plaintext in the second encrypted multidimensional vector combination, and save the statistical result to the decompression list.

[0174] Continuing with the above example, refer to Figure 8 , we decompose the integer formed by concatenating four 441-bit integers to obtain four 441-bit large integers 2. Since there are 10 1764-bit integers here, we finally get 40 441-bit large integers 2.

[0175] It should be noted that the 40 441-bit large integers 2 here are the large integers 2 of the last dimension in the second encrypted multi-dimensional vector combination corresponding to each splitting point.

[0176] It should be noted that since the first participant device may have compressed the multi-dimensional vector, after obtaining the decompression list and the plaintext list, when the first participant device has not compressed the multi-dimensional vector, the first participant device directly combines the splitting point construction results and training samples in the decompression list and the plaintext list to determine the global splitting point of the multi-classification gradient boosting tree; when the first participant device compresses the multi-dimensional vector, the first participant device also needs to decompress the plaintext in the decompression list and the plaintext list to obtain the splitting point construction result at the second participant device, and then combines the splitting point construction result and the training samples to determine the global splitting point of the multi-classification gradient boosting tree.

[0177] Next, after obtaining the decompressed list and the plaintext list, the first participant device first restores the split point construction result based on the decompressed list and the plaintext list, that is, restores the plaintext correspondence between the decompressed list and the plaintext list into a multi-dimensional vector combination, and then decompresses the multi-dimensional vector combination.

[0178] In some embodiments, for i=0; i<number of ciphertexts; i++;

[0179] If i! = number of ciphertexts - 1;

[0180] According to the total space occupied by a label, the plaintext is unpacked into gh packets as many as the label can hold, and then each gh packet is unpacked into g and h to obtain the g and h statistics of the label;

[0181] If i = number of ciphertexts - 1;

[0182] According to the total space occupied by a label, the plaintext is unpacked into gh packets that do not contain the number of labels, and then each gh packet is unpacked into g and h to obtain the g and h statistics of the label;

[0183] Here, at each traversal, the following processing is performed: if i is not equal to the number of ciphertexts - 1, then according to the total space occupied by a label, the incompressible plaintext is decompressed into the number of gradient combinations gh that the label can accommodate, and then each gradient combination gh is decompressed to obtain the g, h statistical results of the label;

[0184] If i = the number of ciphertexts - 1, then according to the total space occupied by a label, the compressible plaintext is decompressed into gradient combinations gh that do not accommodate the number of labels, and then each gradient combination gh is decompressed to obtain the g and h statistical results of the label; here, the gradient combination gh corresponds to each dimension in the multidimensional vector combination respectively; the g statistical result is the sum of the first-order gradients corresponding to each dimension in the multidimensional vector combination respectively, and the h statistical result is the sum of the second-order gradients corresponding to each dimension in the multidimensional vector combination respectively.

[0185] Continuing with the above example, refer to Figure 8 In the process of decompressing the incompressible plaintext, the large integer 1 with a bit length of 1911 bits, which is composed of 13 147-bit integers, is disassembled to obtain 13 147-bit integers (i.e., gradient combination gh). Since there are 40 large integers here, 520 (i.e., 13*40) 147-bit integers are finally obtained. It should be noted that the 520 147-bit integers here are the gradient combinations gh of each dimension in the multidimensional vector combination corresponding to each splitting point, and then each 147-bit gradient combination is decompressed into g statistics with a bit length of 74 and h statistics with a bit length of 73.

[0186] Continuing with the above example, refer to Figure 8 In the process of compressing the compressible plaintext, the large integer 2 with a bit length of 441 bits, which is composed of three 147-bit integers, is disassembled to obtain three 147-bit integers (i.e., gradient combination gh). Since there are 40 large integers here, 120 (i.e., 3*40) 147-bit integers are finally obtained. It should be noted that the 120 147-bit integers here are the gradient combination gh of each dimension in the multidimensional vector combination corresponding to each split point, and then each 147-bit gradient combination is decompressed into a g statistical result with a bit length of 74 and an h statistical result with a bit length of 73.

[0187] Next, the process of decompressing each plaintext into the gradient combination gh in the above disassembly process is explained.

[0188] In some embodiments, for i=0; i<number of tags to accommodate; i++;

[0189] Gradient combination gh = big integer & 2^ 一个标签的总占位 -1, get a gradient combination gh;

[0190] Shift the large integer to the right by the total space occupied by the label;

[0191] Save the gradient combination gh;

[0192] Here, at each traversal, the following processing is performed to determine the gradient combination gh = large integer & 2^ 一个标签的总占位-1, then move the large integer to the right by the total number of places occupied by the label and save the resulting gradient combination gh.

[0193] Continuing with the above example, refer to Figure 8 In the process of decompressing each incompressible plaintext into the gradient combination gh, each large integer 1 with a bit length of 1911 bits is disassembled to obtain 13 147-bit integers (i.e., gradient combinations gh). Since there are 40 large integers here, 520 gradient combinations gh are finally obtained.

[0194] Continuing with the above example, refer to Figure 8 In the process of decompressing each compressible plaintext into the gradient combination gh, the large integer 2 with a bit length of 441 bits is disassembled to obtain 3 147-bit integers (i.e., the gradient combination gh). Since there are 40 large integers here, 10 gradient combinations gh are finally obtained.

[0195] Next, the process of decompressing each gradient combination in the above decompression process to obtain the g and h statistical results of the label is described.

[0196] Here, multiple gradient combinations gh corresponding to each dimension are converted to floating point, and two offset multidimensional vectors g and h are obtained, respectively corresponding to the sum of each dimension. In some embodiments, for each gradient combination, the gradient combination is shifted right by h bits to obtain an integer statistical result of g, and then the statistical result &(2^ h占位 -1), get the integer statistics of h, and then divide the statistics of g by 2^ 精度位 , get the statistical result of g, and finally divide the statistical result of h by 2^ 精度位 , and the statistical result of h is obtained. In this way, the statistical results of g and h are obtained, that is, the sum of the first-order gradients corresponding to each dimension in the multidimensional vector combination after offset and the sum of the second-order gradients corresponding to each dimension in the multidimensional vector combination.

[0197] Continuing with the above example, for a total of 640 (i.e., 120+520) gh, floating point conversion is performed on 640 147-bit integers. In some embodiments, for each 147-bit integer, shift right by 73 bits to obtain a 74-bit integer statistical result of g, and the 147-bit integer &2^ 73 -1, get the 73-bit integer statistical result of h, then divide the 74-bit integer statistical result of g by 2^ 53 , the statistical result of g is the sum of the first-order gradients of each dimension in the corresponding multidimensional vector combination after the shift, and the 73-bit integer statistical result of h is divided by 2^ 53 , the statistical result of h is the sum of the second-order gradients of each dimension in the multi-dimensional vector combination.

[0198] It should be noted that, since the first-order gradient g is offset, it is necessary to restore the sum of the offset first-order gradient based on the number of received samples. In some embodiments, the number of samples of the split point construction result is determined, and the statistical result of g is subtracted by (1*the number of samples of the split point statistical result), so that the offset is offset.

[0199] In some embodiments, after obtaining the final splitting point construction result, the global splitting point of the multi-classification gradient boosting tree is determined in combination with the final splitting point construction result and the training samples.

[0200] Here, in combination with the final splitting point construction result, i.e., the training sample, the process of determining the global splitting point of the multi-classification gradient boosting tree can be that the first participant device traverses the multi-classification gradient boosting tree based on the training sample to determine the first splitting point, and at the same time, the first participant device traverses the splitting points in the multi-classification gradient boosting tree based on the training sample and the splitting point construction result to determine the second splitting point; here, the first splitting point is the optimal splitting point at the first participant device, and the second splitting point is the optimal splitting point at the second participant device; the first splitting point and the second splitting point are compared to determine the global splitting point of the multi-classification gradient boosting tree.

[0201] In actual implementation, based on the training samples, the splitting points of the multi-classification gradient boosting tree are traversed to determine the first splitting point. The process can be based on the labels and multidimensional vectors of the training samples, and the splitting points of the multi-classification gradient boosting tree are traversed to determine the first feature splitting gain histogram corresponding to the first participating device; based on the first feature splitting gain histogram, the gain value of the splitting point of the multi-classification gradient boosting tree is determined, and the splitting point with the largest gain value is selected as the first splitting point.

[0202] In actual implementation, based on the training samples and the splitting point construction results, the splitting points in the multi-classification gradient boosting tree are traversed to determine the second splitting point. The process can be based on the labels of the training samples and the splitting point construction results, and the splitting points of the multi-classification gradient boosting tree are traversed to determine the second feature splitting gain histogram corresponding to the second participant device; based on the second feature splitting gain histogram, the gain value of the splitting point of the multi-classification gradient boosting tree is determined, and the splitting point with the largest gain value is selected as the second splitting point.

[0203] Here, the process of determining the maximum gain splitting point in the first feature splitting gain histogram and the second feature splitting gain histogram may be based on a preset gain formula to determine the maximum gain splitting point in the first feature splitting gain histogram and the second feature splitting gain histogram.

[0204] The preset gain formula here is:

[0205]

[0206]

[0207] Here, in formula (2) and To traverse the current split point, split the node according to a certain eigenvalue, divide it into two, and obtain the left son and the right son. Then calculate the gains of the left son and the right son based on formula (3). The g in formula (3) is j and h j It is the first-order gradient and the second-order gradient of the dimension corresponding to the maximum predicted value selected in step 102.

[0208] In actual implementation, after determining the first splitting point and the second splitting point, the first splitting point and the second splitting point are compared to determine the global splitting point of the multi-classification gradient boosting tree, which can be to obtain the gain value of the first splitting point and the gain value of the second splitting point respectively; compare the gain value of the first splitting point and the gain value of the second splitting point, and select the splitting point corresponding to the larger gain value as the global splitting point of the multi-classification gradient boosting tree.

[0209] It should be noted that the gain value of the first splitting point and the gain value of the second splitting point obtained here are the gain values ​​of the corresponding first splitting point and the second splitting point obtained in the process of determining the first splitting point and the second splitting point.

[0210] In some embodiments, after determining the global splitting point, the process of constructing a multi-classification gradient boosting tree based on the global splitting point can be by obtaining the feature value and feature coding of the global splitting point; determining the features of the training samples corresponding to the feature coding, and determining the feature splitting method in combination with the features and feature values; and constructing a multi-classification gradient boosting tree based on the feature splitting method.

[0211] It should be noted that when the global splitting point belongs to the first participant device, the splitting point code corresponding to the global splitting point is obtained, and the feature value and feature code of the global splitting point are determined based on the splitting point code; at the same time, an empty message is sent to the second participant device.

[0212] When the global splitting point belongs to the second participant device, a splitting point code corresponding to the global splitting point is sent to the second participant device, and a feature value and a feature code returned by the second participant device based on the splitting point code are received; and at the same time, it is recorded that the splitting point belongs to the second participant device.

[0213] For example, see Fig. 9 , Fig. 9This is an optional splitting point splitting diagram provided by the embodiment of the present application. Here, the sample data falling on the splitting point includes the corresponding sample data of samples 2, 5, 7, 8 and 10. The feature code corresponding to the global splitting point is 3, and the feature value = 0.4. Here, when the global splitting point belongs to the second participant device, the first participant device sends the global splitting point in the form of a number to the second participant device. The second participant device obtains the feature code and feature value corresponding to the global splitting point based on the number, and sends the corresponding feature code and feature value to the first participant device. Fig. 9 As shown, after splitting the split point, the left child node corresponds to sample 2, sample 5 and sample 10, and the right child node corresponds to sample 7 and sample 8.

[0214] In actual implementation, if the multi-classification gradient boosting tree where the split point is located does not meet the stop splitting condition, the splitting continues until the stop splitting condition is met, and the construction of the multi-classification gradient boosting tree of this generation is completed. Here, the output of the previous generation tree is used to update the current prediction value, and the first participant device continues to calculate the loss function to determine whether the loss function converges. If not, continue to build a new multi-classification gradient boosting tree until the loss function converges, or determine whether the number of multi-classification gradient boosting trees reaches a preset number. If not, continue to build a new multi-classification gradient boosting tree until the preset number is reached. When the loss function converges or the multi-classification gradient boosting tree reaches the preset number, the construction of the multi-classification gradient boosting tree is completed.

[0215] It should be noted that the stop splitting condition may be that the depth of the multi-classification gradient boosting tree reaches a preset depth, the gain value of the multi-classification gradient boosting tree reaches a preset gain value, or the number of samples does not meet the requirement for continued splitting.

[0216] In this way, when the first participant device and the second participant device have a large amount of data calculation and transmission during the data transmission process, the sample gradient is compressed in plain text and then encrypted to ensure that the original sample gradient can satisfy the addition and subtraction operations under compression, thereby reducing the encryption and decryption amount of the first participant device and the calculation amount of the second participant device.

[0217] By using the above-mentioned embodiment of the present application, based on the predicted value and corresponding label of the training sample, the gradient information of multiple categories of the training sample is represented in the form of a multi-dimensional vector, thereby processing the multi-dimensional vector representing the gradient information, and determining the corresponding splitting point data based on the processing result, thereby determining the global splitting point of the multi-classification gradient boosting tree, and then constructing a multi-classification gradient boosting tree. In this way, by performing multi-classification tasks through a multi-classification gradient boosting tree capable of multiple outputs, the overhead in the multi-classification task is effectively reduced and the training efficiency is improved.

[0218] Next, taking the application scenario of the multi-classification gradient boosting tree as an example, the construction method of the multi-classification gradient boosting tree provided in the embodiment of the present application is introduced. Fig.10 This is an optional flow chart of the method for constructing a multi-classification gradient boosting tree provided in the embodiment of the present application, see Fig.10 The method for constructing a multi-classification gradient boosting tree provided in the embodiment of the present application is implemented by a first participant device (leading party guest) and a second participant device (participant host) in collaboration. The first participant device and the second participant device can be a server or a terminal. Fig.10 The method for constructing a multi-classification gradient boosting tree provided in the embodiment of the present application includes:

[0219] In step 301, the first participant device and the second participant device respectively complete local initialization, bin local features and start to jointly construct a multi-classification gradient boosting tree.

[0220] Here, there are t generations of trees to be constructed, where t is a positive integer greater than or equal to 1. Among them, the height of each multi-classification gradient boosting tree is the same.

[0221] Step 302: The first participant device obtains two multi-dimensional vectors g and h according to the local label and the prediction value of the previous t-1 generation tree.

[0222] Step 303: call the compression planning module, calculate the compression parameters and send them to the second participant device.

[0223] Step 304: Call the plaintext compression module to compress the two multi-dimensional vectors g and h to obtain a vector combination g_h corresponding to each sample.

[0224] Step 305: compress the vector combination g_h of each sample again to obtain a multi-dimensional vector combination.

[0225] Here, the multidimensional vector is obtained by compressing the vector combination g_h according to the number of label classes, the number of ciphertexts, the last number of labels, and the last ciphertext packed integer placeholder.

[0226] Step 306: homomorphically encrypt the multidimensional vector combination to obtain an encrypted multidimensional vector combination and send it to the second participant device.

[0227] Step 307: the first participant device and the second participant device initialize the root node, and all samples fall on the root node and are added to the node list.

[0228] Step 308: Take out a node from the node list. At the current split node, the first participant device uses the local features and the two multidimensional vectors g and h to determine the optimal split point of the dominant party.

[0229] Here, the current split point is traversed, and the node is split according to a certain eigenvalue, and it is divided into two to obtain the left son and the right son. The gain calculation formula according to the eigenvalue split is:

[0230]

[0231]

[0232] Here, g in formula (5) j and h j It is the first-order gradient and the second-order gradient of the dimension corresponding to the maximum predicted value selected in step 302.

[0233] In actual implementation, the first participant device determines the optimal splitting point of the dominant party by comparing the gains calculated by splitting according to different eigenvalues.

[0234] Step 309: The second participant device constructs a splitting point based on the received encrypted multi-dimensional vector combination to obtain an encrypted splitting point construction result.

[0235] Here, the second participant device takes out a node from the node list, calculates a histogram using the received encrypted multidimensional vector combination at the current split node, and constructs a split point from the histogram to obtain an encrypted split point construction result.

[0236] In step 310, the second participant device calls the ciphertext compression module, compresses the compressible part of the encrypted multidimensional vector combination in the construction result based on the received compression parameters and saves it as a compressed ciphertext list, and saves the incompressible part of the encrypted multidimensional vector combination in the construction result as an uncompressed ciphertext list, and counts the number of samples in each splitting point.

[0237] Step 311: Send the compressed ciphertext list, the uncompressed ciphertext list, and the statistical number of samples to the first participant device.

[0238] Step 312: The first participant device uses a decompression module to decrypt the compressed ciphertext list and the uncompressed ciphertext list to obtain a compressed plaintext list and a plaintext list.

[0239] Step 313: Use a decompression module to decompress the compressed plaintext list, obtain compressible plaintext and save it to the decompression list.

[0240] Step 314, using a decompression module to decompose the compressible plaintext in the decompression list and the incompressible plaintext in the plaintext list, and obtain the sum of two multi-dimensional vectors g and h of each sample corresponding to each dimension respectively.

[0241] Step 315: Based on the sum of each dimension of the two multi-dimensional vectors g and h, the gains of multiple splitting points are calculated to obtain the optimal splitting point of the participants.

[0242] The calculation process here refers to step 308. Based on the process of step 308, in actual implementation, the first participant device determines the optimal splitting point of the participant by comparing the gains calculated by splitting according to different eigenvalues.

[0243] Step 316: Based on the optimal splitting points of the participants, the optimal splitting points of the dominant party are integrated to obtain the global optimal splitting point.

[0244] Here, by comparing the gains of the optimal splitting point of the participating party with the optimal splitting point of the dominant party, the splitting point with the larger gain is selected as the global optimal splitting point.

[0245] Here, if the split point belongs to the first participant device, an empty message is sent to the second participant device to determine the feature value and feature code of the split point, and at the same time record that the node belongs to the first participant device;

[0246] If the split point belongs to the second participant device, the split point code is sent to the second participant device, the feature value and feature code sent by the second participant device are received, and the node is recorded as belonging to the second participant device.

[0247] Step 317, determine whether the stop splitting condition is met, if so, return to step 309, if not, execute step 318.

[0248] It should be noted that the splitting condition here is that the maximum tree height or gain value has not been reached, and the number of samples is sufficient to continue splitting.

[0249] In actual implementation, if the splitting conditions are met, the current node is split based on the feature value and feature coding, the leaf node is added to the node list, and step 309 is returned to repeat until the current tree is built. If not, the establishment of the multi-classification gradient boosting tree is stopped and step 318 is executed.

[0250] Step 318, using the output of the previous generation tree to update the current prediction value.

[0251] Step 319, checking whether the stopping condition is met, if not, the first participant device and the second participant device continue to jointly construct the multi-classification gradient boosting tree, and return to step 302, if satisfied, execute step 320.

[0252] Here, the stopping condition includes whether the loss function of the multi-classification gradient boosting tree converges or whether the maximum number of trees is reached.

[0253] Step 320, completing the construction of the multi-classification gradient boosting tree.

[0254] By using the above-mentioned embodiment of the present application, based on the predicted value and corresponding label of the training sample, the gradient information of multiple categories of the training sample is represented in the form of a multi-dimensional vector, thereby processing the multi-dimensional vector representing the gradient information, and determining the corresponding splitting point data based on the processing result, thereby determining the global splitting point of the multi-classification gradient boosting tree, and then constructing a multi-classification gradient boosting tree. In this way, by performing multi-classification tasks through a multi-classification gradient boosting tree capable of multiple outputs, the overhead in the multi-classification task is effectively reduced and the training efficiency is improved.

[0255] The following is a description of the construction device 254 of the multi-classification gradient boosting tree provided in the embodiment of the present application. Fig.11 , Fig.11 is a schematic diagram of the structure of a multi-classification gradient boosting tree construction device 254 provided in an embodiment of the present application. The multi-classification gradient boosting tree construction device 254 provided in an embodiment of the present application includes:

[0256] The acquisition module 2541 is used to obtain the predicted value of the training sample carrying the label.

[0257] The determination module 2542 is used to determine a multidimensional vector for characterizing the gradient corresponding to the training sample based on the predicted value and the corresponding label, and the number of dimensions of the multidimensional vector corresponds to the number of label categories of the multi-classification gradient boosting tree.

[0258] The privacy protection module 2543 is used to perform privacy protection on the multi-dimensional vector, obtain intermediate data, and send the intermediate data to the second participant device.

[0259] The receiving module 2544 is used to receive the splitting point data sent by the second participant device, where the splitting point data is obtained by the second participant device constructing the splitting point corresponding to the multi-classification gradient boosting tree based on the intermediate data.

[0260] The construction module 2545 is used to determine the global splitting point of the multi-classification gradient boosting tree based on the splitting point data, and construct the multi-classification gradient boosting tree based on the global splitting point.

[0261] In some embodiments, the determination module 2542 is also used to compare the predicted value with the corresponding label to obtain a comparison result; based on the comparison result, determine the first multidimensional vector and the second multidimensional vector of the training sample; wherein the first multidimensional vector includes the first-order gradient of each label category of the training sample, and the second multidimensional vector includes the second-order gradient of each label category of the training sample.

[0262] In some embodiments, the determination module 2542 is also used to determine the compression control parameters of the multidimensional vector and the modeling parameters of the multi-classification gradient boosting tree; wherein the compression control parameters include: the maximum number of homomorphic integer digits, the fixed-point precision of floating-point numbers, and the number of samples; based on the compression control parameters and the modeling parameters, the compression parameters are determined; based on the compression parameters, the multidimensional vector is compressed to obtain a compressed multidimensional vector; the privacy protection of the multidimensional vector to obtain intermediate data includes: privacy protection of the compressed multidimensional vector to obtain intermediate data.

[0263] In some embodiments, the compression parameters include: the placeholder of the gradient corresponding to each dimension in the multidimensional vector, the total placeholder of a label in the multi-classification gradient boosting tree, the number of labels accommodated, the number of ciphertexts, the number of unaccommodated labels, the placeholder of unaccommodated ciphertext packed integers, and the amount of ciphertext compression; the determination module 2542 is also used to determine the floating-point fixed-point accuracy of the multi-classification gradient boosting tree, and based on the floating-point fixed-point accuracy, the gradient corresponding to each dimension in the multidimensional vector is fixed-point processed to obtain the fixed-point integer corresponding to the gradient, and based on the number of samples and the fixed-point integer, the placeholder corresponding to the gradient is determined; wherein, the gradient of each dimension in the multidimensional vector The degree is the gradient of each label category corresponding to the training sample; the placeholders of the gradients are summed to determine the total placeholder of a label; the maximum number of homomorphic integer digits and the total placeholder of the label are divided by integers to determine the number of labels that can be accommodated; the number of label categories and the number of label capacity are divided by integers to determine the number of ciphertexts; the number of label categories and the number of label capacity are divided by remainder to determine the number of unaccommodated labels; the number of unaccommodated labels is multiplied by the total placeholder of the label to determine the integer placeholder that cannot be accommodated for ciphertext packing; the maximum number of homomorphic integer digits and the total placeholder of the label are divided by integers to determine the amount of ciphertext compression.

[0264] In some embodiments, the privacy protection module 2543 is also used to homomorphically encrypt the multi-dimensional vector to obtain intermediate data; determining the global splitting point of the multi-classification gradient boosting tree based on the splitting point data includes: decrypting the splitting point data to obtain decrypted splitting point data; and determining the global splitting point of the multi-classification gradient boosting tree based on the decrypted splitting point data.

[0265] In some embodiments, the splitting point data is that the second participant device constructs the splitting point of the multi-classification gradient boosting tree based on the intermediate data to obtain the splitting point construction result, and compresses the splitting point construction result; the construction module 2545 is also used to determine the global splitting point of the multi-classification gradient boosting tree based on the splitting point data, including: decompressing the splitting point data to obtain the splitting point construction result; combining the splitting point construction result and the training sample to determine the global splitting point of the multi-classification gradient boosting tree.

[0266] In some embodiments, the construction module 2545 is also used to traverse the splitting points of the multi-classification gradient boosting tree based on the training samples to determine a first splitting point; based on the training samples and the splitting point construction results, traverse the splitting points in the multi-classification gradient boosting tree to determine a second splitting point; and compare the first splitting point and the second splitting point to determine a global splitting point of the multi-classification gradient boosting tree.

[0267] In some embodiments, the construction module 2545 is also used to traverse the splitting points of the multi-classification gradient boosting tree based on the label of the training sample and the multidimensional vector, and determine the first feature splitting gain histogram corresponding to the first participant device; wherein the first feature splitting gain histogram is used to determine the maximum gain splitting point in the first feature splitting gain histogram; based on the first feature splitting gain histogram, determine the gain value of the splitting point of the multi-classification gradient boosting tree, and select the splitting point with the largest gain value as the first splitting point; based on the label of the training sample and the splitting point construction result, traverse the splitting points of the multi-classification gradient boosting tree, and determine the second feature splitting gain histogram corresponding to the second participant device; wherein the second feature splitting gain histogram is used to determine the maximum gain splitting point in the second feature splitting gain histogram; based on the second feature splitting gain histogram, determine the gain value of the splitting point of the multi-classification gradient boosting tree, and select the splitting point with the largest gain value as the second splitting point.

[0268] In some embodiments, the construction module 2545 is also used to obtain the gain value of the first splitting point and the gain value of the second splitting point respectively; compare the gain value of the first splitting point and the gain value of the second splitting point, and select the splitting point corresponding to the larger gain value as the global splitting point of the multi-classification gradient boosting tree.

[0269] In some embodiments, the construction module 2545 is also used to obtain the feature value and feature code of the global splitting point; determine the features of the training sample corresponding to the feature code, and determine the feature splitting method in combination with the features and the feature value; and construct the multi-classification gradient boosting tree based on the feature splitting method.

[0270] By using the above-mentioned embodiment of the present application, based on the predicted value and corresponding label of the training sample, the gradient information of multiple categories of the training sample is represented in the form of a multi-dimensional vector, thereby processing the multi-dimensional vector representing the gradient information, and determining the corresponding splitting point data based on the processing result, thereby determining the global splitting point of the multi-classification gradient boosting tree, and then constructing a multi-classification gradient boosting tree. In this way, by performing multi-classification tasks through a multi-classification gradient boosting tree capable of multiple outputs, the overhead in the multi-classification task is effectively reduced and the training efficiency is improved.

[0271] The present application also provides an electronic device, the electronic device comprising:

[0272] A memory for storing executable instructions;

[0273] The processor is used to implement the method for constructing a multi-classification gradient boosting tree provided in an embodiment of the present application when executing the executable instructions stored in the memory.

[0274] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for constructing a multi-classification gradient boosting tree provided in the embodiment of the present application.

[0275] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method for constructing a multi-classification gradient boosting tree provided in the embodiment of the present application.

[0276] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0277] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0278] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0279] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0280] In summary, through the embodiments of the present application, a multi-classification gradient boosting tree capable of multiple outputs can be used to perform multi-classification tasks, which effectively reduces the overhead in multi-classification tasks and improves training efficiency.

[0281] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for constructing a multi-classification gradient boosting tree, applied in the field of financial risk control, characterized in that: Based on a federated learning system, the federated learning system includes a first participant device with a label, and at least one second participant device, the method is applied to the first participant device, wherein the first participant is a bank, and the second participant is a platform holding user feature data, and the second participant includes a shopping mall, a supermarket, an online shopping platform, a game platform, a mobile application store, and / or an Internet company, wherein the user feature data includes the user's basic information, business data, behavior data and device information, including: Obtaining a prediction value of a training sample carrying a label, wherein the label of the training sample is a multi-classification label identifying a user's credit level, and / or a multi-classification label identifying a user's loyalty level, and / or a multi-classification label for evaluating a user's quality level; Based on the predicted values ​​and the corresponding labels, determining a multidimensional vector for characterizing the gradient corresponding to the training sample, wherein the number of dimensions of the multidimensional vector corresponds to the number of label categories of the multi-classification gradient boosting tree; Performing privacy protection on the multidimensional vector to obtain intermediate data, and sending the intermediate data to the second participant device, wherein performing privacy protection on the multidimensional vector to obtain the intermediate data includes: Performing homomorphic encryption on the multidimensional vector to obtain intermediate data; The step of determining the global splitting point of the multi-classification gradient boosting tree based on the splitting point data includes: Decrypting the splitting point data to obtain decrypted splitting point data; Determining a global splitting point of the multi-classification gradient boosting tree based on the decrypted splitting point data; Receiving splitting point data sent by a second participant device, where the splitting point data is obtained by the second participant device constructing a splitting point corresponding to the multi-classification gradient boosting tree based on the intermediate data; Determine a global splitting point of the multi-classification gradient boosting tree based on the splitting point data, and construct the multi-classification gradient boosting tree based on the global splitting point; The splitting point data is obtained by the second participant device constructing the splitting point of the multi-classification gradient boosting tree based on the intermediate data, and compressing the splitting point construction result; Decompressing the splitting point data to obtain the splitting point construction result; The global splitting point of the multi-classification gradient boosting tree is determined based on the splitting point construction result and the training samples.

2. The method according to claim 1, characterized in that The step of determining, based on the predicted value and the corresponding label, a multidimensional vector for characterizing the gradient corresponding to the training sample comprises: Comparing the predicted value with the corresponding label to obtain a comparison result; Based on the comparison result, determining a first multidimensional vector and a second multidimensional vector of the training sample; The first multidimensional vector includes the first-order gradient of each label category of the training sample, and the second multidimensional vector includes the second-order gradient of each label category of the training sample.

3. The method according to claim 1, characterized in that Before performing privacy protection on the multidimensional vector to obtain intermediate data, the method further includes: Determining compression control parameters of the multidimensional vector and modeling parameters of the multi-classification gradient boosting tree; The compression control parameters include: the maximum number of homomorphic integer digits, floating point precision and number of samples; Determining a compression parameter based on the compression control parameter and the modeling parameter; Based on the compression parameter, compressing the multidimensional vector to obtain a compressed multidimensional vector; The step of performing privacy protection on the multidimensional vector to obtain intermediate data includes: Privacy protection is performed on the compressed multi-dimensional vector to obtain intermediate data.

4. The method according to claim 3, characterized in that: The compression parameters include: the placeholder of the gradient corresponding to each dimension in the multidimensional vector, the total placeholder of a label in the multi-classification gradient boosting tree, the number of labels accommodated, the number of ciphertexts, the number of unaccommodated labels, the placeholder of unaccommodated ciphertext packed integers, and the ciphertext compression amount; The step of determining the compression parameter based on the compression control parameter and the bit length of the data of each dimension in the multidimensional vector comprises: Determine the floating-point fixed-point accuracy of the multi-classification gradient boosting tree, and based on the floating-point fixed-point accuracy, perform fixed-point processing on the gradient corresponding to each dimension in the multi-dimensional vector to obtain a fixed-point integer corresponding to the gradient, and determine the placeholder corresponding to the gradient based on the number of samples and the fixed-point integer; wherein the gradient of each dimension in the multi-dimensional vector is the gradient corresponding to each label category of the training sample; Summing up the position occupied by the gradient to determine the total position occupied by a label; Perform integer division processing on the maximum number of homomorphic integers and the total number of occupancy of the tags to determine the number of tags that can be accommodated; Perform integer division processing on the number of the tag categories and the number of tags that can be accommodated to determine the number of ciphertexts; Divide the number of the label categories by the number of labels that can be accommodated and take the remainder to determine the number of labels that cannot be accommodated; The number of unaccommodated labels is multiplied by the total number of labels to determine the number of unaccommodated ciphertext packaged integers; An integer division process is performed on the homomorphic maximum integer digit and the total occupancy of the tag to determine the ciphertext compression amount.

5. The method according to claim 1, characterized in that The step of combining the split point construction result and the training sample to determine the global split point of the multi-classification gradient boosting tree includes: Based on the training sample, traverse the splitting points of the multi-classification gradient boosting tree to determine a first splitting point; Based on the training samples and the split point construction result, traverse the split points in the multi-classification gradient boosting tree to determine a second split point; The first splitting point and the second splitting point are compared to determine a global splitting point of the multi-classification gradient boosting tree.

6. The method according to claim 5, characterized in that The step of traversing the splitting points of the multi-classification gradient boosting tree based on the training sample to determine a first splitting point includes: Based on the labels of the training samples and the multidimensional vector, traversing the splitting points of the multi-classification gradient boosting tree, and determining a first feature splitting gain histogram corresponding to the first participant device; Based on the first feature splitting gain histogram, determine the gain value of the splitting point of the multi-classification gradient boosting tree, and select the splitting point with the largest gain value as the first splitting point; The traversing the splitting points in the multi-classification gradient boosting tree based on the training sample and the splitting point construction result to determine the second splitting point includes: Based on the labels of the training samples and the splitting point construction results, traverse the splitting points of the multi-classification gradient boosting tree to determine a second feature splitting gain histogram corresponding to the second participant device; Based on the second feature splitting gain histogram, the gain value of the splitting point of the multi-classification gradient boosting tree is determined, and the splitting point with the largest gain value is selected as the second splitting point.

7. The method according to claim 5, characterized in that The comparing the first splitting point and the second splitting point to determine the global splitting point of the multi-classification gradient boosting tree includes: respectively obtaining a gain value of the first splitting point and a gain value of the second splitting point; Compare the gain value of the first splitting point and the gain value of the second splitting point, and select the splitting point corresponding to the larger gain value as the global splitting point of the multi-classification gradient boosting tree.

8. The method according to claim 1, characterized in that The step of constructing the multi-classification gradient boosting tree based on the global split point includes: Obtaining the feature value and feature code of the global splitting point; Determine the features of the training sample corresponding to the feature code, and determine the feature splitting method in combination with the features and the feature values; Based on the feature splitting method, the multi-classification gradient boosting tree is constructed.

9. A device for constructing a multi-classification gradient boosting tree, applied in the field of financial risk control, characterized in that: The device comprises: An acquisition module, used to acquire the predicted value of a training sample carrying a label, wherein the label of the training sample is a multi-classification label identifying the user's credit level, and / or a multi-classification label identifying the user's loyalty level, and / or a multi-classification label for evaluating the user's quality level; A determination module, configured to determine, based on the predicted value and the corresponding label, a multidimensional vector for characterizing the gradient corresponding to the training sample, wherein the number of dimensions of the multidimensional vector corresponds to the number of label categories of the multi-classification gradient boosting tree; A privacy protection module is used to perform privacy protection on the multidimensional vector, obtain intermediate data, and send the intermediate data to a second participant device, the second participant is a platform holding the user's feature data, and the second participant includes a shopping mall, a supermarket, an online shopping platform, a game platform, a mobile application store, and / or an Internet company, wherein the user's feature data includes the user's basic information, business data, behavior data and device information, and the privacy protection module is specifically used to perform homomorphic encryption on the multidimensional vector to obtain intermediate data; based on the splitting point data, determining the global splitting point of the multi-classification gradient boosting tree includes: decrypting the splitting point data to obtain the decrypted splitting point data; based on the decrypted splitting point data, determining the global splitting point of the multi-classification gradient boosting tree; A receiving module, configured to receive splitting point data sent by a second participant device, wherein the splitting point data is obtained by the second participant device constructing a splitting point corresponding to the multi-classification gradient boosting tree based on the intermediate data; A construction module, configured to determine a global splitting point of the multi-classification gradient boosting tree based on the splitting point data, and construct the multi-classification gradient boosting tree based on the global splitting point; The apparatus further includes: the splitting point data is obtained by constructing the splitting point of the multi-classification gradient boosting tree based on the intermediate data by the second participant device, and compressing the splitting point construction result; Decompressing the splitting point data to obtain the splitting point construction result; The global splitting point of the multi-classification gradient boosting tree is determined based on the splitting point construction result and the training samples.

10. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable instructions; A processor, configured to implement the method according to any one of claims 1 to 8 when executing the executable instructions stored in the memory.

11. A computer-readable storage medium, characterized in that: Executable instructions are stored, and when executed by a processor, the method described in any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images