Decision model training method, device and federated learning system based on federated learning

Abstract

The present disclosure provides a decision model training method, device, federated learning system, electronic device, computer-readable storage medium and computer program product based on federated learning, which relates to the field of artificial intelligence, and in particular to the field of federated learning. The scheme is: a first participant sorts part of the sample data based on one attribute value of at least one attribute value of part of the sample data in multiple sample data; divides the sorted sample data into multiple data groups; for each data group: generates a sample identification ID code based on a sample identification ID set composed of at least one sample data; sends the sample identification ID code to a second participant; receives a count of each label value in at least one label value corresponding to the data group; determines the splitting gain corresponding to the data group based on the count; determines the local splitting node included in the decision model based on the splitting gain corresponding to each data group.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of federated learning technology, and specifically to a decision model training method, device, federated learning system, electronic device, computer-readable storage medium and computer program product based on federated learning. Background Art

[0002] With the development of computer technology and the advancement of artificial intelligence technology, Federated Learning has gradually become a hot topic in the field of artificial intelligence. Federated Learning refers to the calculation process in which multiple data holders can perform model training and obtain the final model without leaving the original data. Therefore, the training task of the machine learning model can be completed through multi-party collaboration.

[0003] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0004] The present disclosure provides a decision model training method, device, federated learning system, electronic device, computer-readable storage medium and computer program product based on federated learning.

[0005] According to one aspect of the present disclosure, a method for training a decision model based on federated learning is provided, and the method is applied to a first participant holding multiple sample data in a federated learning system, and the federated learning system also includes a second participant in communication connection with the first participant. The method includes: sorting part of the sample data based on one attribute value of at least one attribute value of part of the sample data in the multiple sample data; dividing the sorted sample data into multiple data groups; for each data group: generating a sample identification ID code based on a sample identification ID set composed of a sample identification ID of at least one sample data in the data group; sending the sample identification ID code to the second participant; receiving a count of each label value in at least one label value corresponding to the data group from the second participant; and determining a splitting gain corresponding to the data group based on the count, the splitting gain indicating the heterogeneity of the classification result expected to be obtained after the decision model performs node splitting at the data group. The method also includes determining the local splitting node included in the decision model based on the splitting gain corresponding to each data group, thereby training the decision model.

[0006] According to another aspect of the present disclosure, a method for training a decision model based on federated learning is provided. The method is applied to a second participant holding multiple label data in a federated learning system, and the federated learning system also includes a first participant in communication connection with the second participant. The method includes: receiving multiple sample identification ID codes from the first participant, each of the multiple sample identification ID codes is generated by the first participant based on a sample identification ID set consisting of at least one sample data divided into a corresponding data group from the multiple sample data held; for each sample identification ID code: querying the label value of at least one label data associated with the sample identification ID code from the multiple label data; based on the label value of the associated at least one label data, determining the count of each label value in the at least one label value corresponding to the corresponding data group; and sending the count to the first participant, so that the first participant determines the local splitting node included in the decision model based on the count, thereby training the decision model.

[0007] According to another aspect of the present disclosure, a decision model training device based on federated learning is provided. The device is applied to a first participant holding multiple sample data in a federated learning system, and the federated learning system also includes a second participant connected to the first participant in communication. The device includes: a sorting unit, a grouping unit, an encoding unit, an encoding sending unit, a label value count receiving unit, a splitting gain determining unit, and a training unit. The sorting unit is configured to sort the partial sample data based on one attribute value of at least one attribute value of the partial sample data in the multiple sample data; the grouping unit is configured to divide the sorted sample data into multiple data groups; for each data group: the encoding unit is configured to generate a sample identification ID code based on a sample identification ID set composed of at least one sample data in the data group; the encoding sending unit is configured to send the sample identification ID code to the second participant; the label value count receiving unit is configured to receive the count of each label value in at least one label value corresponding to the data group from the second participant; and the splitting gain determining unit is configured to determine the splitting gain corresponding to the data group based on the count, and the splitting gain indicates the heterogeneity of the classification result expected to be obtained after the decision model performs node splitting at the data group. In addition, the training unit is configured to determine the local splitting nodes included in the decision model based on the splitting gain corresponding to each data group, so as to train the decision model.

[0008] According to another aspect of the present disclosure, a decision model training device based on federated learning is provided. The device is applied to a second participant holding multiple label data in a federated learning system, and the federated learning system also includes a first participant connected to the second participant in communication. The device includes: a coding receiving unit, a label value query unit, a label value counting unit, and a label value count sending unit. The coding receiving unit is configured to receive multiple sample identification ID codes from the first participant, each of which is generated by the first participant based on a sample identification ID set consisting of at least one sample data divided into a corresponding data group from the multiple sample data held; for each sample identification ID code: the label value query unit is configured to query the label value of at least one label data associated with the sample identification ID code from the multiple label data; the label value counting unit is configured to determine the count of each label value in at least one label value corresponding to the corresponding data group based on the label value of the associated at least one label data; and the label value count sending unit is configured to send the count to the first participant, so that the first participant determines the local split node included in the decision model based on the count, thereby training the decision model.

[0009] According to another aspect of the present disclosure, a federated learning system is provided, comprising a decision model training device based on federated learning for a first participant holding multiple sample data in the federated learning system as described above, and a decision model training device based on federated learning for a second participant holding multiple label data in the federated learning system as described above.

[0010] According to another aspect of the present disclosure, an electronic device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned decision model training method based on federated learning.

[0011] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above-mentioned decision model training method based on federated learning.

[0012] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the above-mentioned decision model training method based on federated learning when executed by a processor.

[0013] According to one or more embodiments of the present disclosure, the security of data in a federated learning process may be improved.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0016] Figure 1 A schematic diagram showing an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0017] Figure 2 A flowchart of a decision model training method based on federated learning according to an embodiment of the present disclosure is shown;

[0018] Figure 3 A schematic diagram showing a partial process of a decision model training method based on federated learning according to an embodiment of the present disclosure;

[0019] Figure 4 A flowchart showing a partial process of a decision model training method based on federated learning according to an embodiment of the present disclosure is shown;

[0020] Figure 5 A schematic diagram showing a partial process of a decision model training method based on federated learning according to an embodiment of the present disclosure;

[0021] Figure 6 A flowchart of a decision model training method based on federated learning according to an embodiment of the present disclosure is shown;

[0022] Figure 7 A flowchart showing a partial process of a decision model training method based on federated learning according to an embodiment of the present disclosure is shown;

[0023] Figure 8 A flowchart showing a partial process of a decision model training method based on federated learning according to an embodiment of the present disclosure is shown;

[0024] Fig. 9 A schematic diagram showing a partial process of a decision model training method based on federated learning according to an embodiment of the present disclosure;

[0025] Fig.10 A structural block diagram of a decision model training device based on federated learning applied to a first participant in a federated learning system according to an embodiment of the present disclosure is shown;

[0026] Fig.11 A structural block diagram of a decision model training device based on federated learning applied to a second participant in a federated learning system according to an embodiment of the present disclosure is shown;

[0027] Fig.12 A process diagram showing model training using a decision model training method based on federated learning according to an embodiment of the present disclosure; and

[0028] Fig.13 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0029] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0030] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0031] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0032] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0033] Federated learning (also known as federated machine learning or joint learning) is a machine learning framework that can effectively help multiple data holders use data and conduct machine learning modeling while meeting the requirements of user privacy protection, data security, and government regulations. According to the data distribution, federated learning can be divided into horizontal federated learning and vertical federated learning. Horizontal federated learning is also called feature-aligned federated learning, that is, the data features of the participants in horizontal federated learning are aligned, which is suitable for the situation where the data features of the participants overlap more, but the sample identifiers (ID) overlap less. Vertical federated learning is also called sample-aligned federated learning, that is, the training samples of the participants in vertical federated learning are aligned, which is suitable for the situation where the training sample IDs of the participants overlap more, but the data features overlap less.

[0034] In related technologies, the implementation method of federated learning may require the use of an intermediate coordination node (or coordinator, intermediate coordination party), which participates in the computing tasks of data aggregation and data distribution, which increases the risk of data privacy leakage of the participants. In addition, in practical applications, it is difficult to find such an intermediate coordination party that is trusted by other participants.

[0035] In addition, in related technologies, federated learning, especially the implementation method of decision models based on federated learning, is more likely to leak label data. Participants with label data can encrypt label data and send it to other participants. In classification tasks, due to the limited types of label data values, there are certain statistical regularities in the encrypted label data. Participants with sample data can easily infer the original plaintext label data based on the ciphertext data. This poses a challenge to both participants with label data and user privacy.

[0036] In addition, in related technologies, the overall computational efficiency of federated learning is low. In order to maximize the privacy of the label data and sample data of the participants, related technologies use a large amount of homomorphic encryption technology, which has complex computational logic and high communication overhead, and cannot be well applied to large-scale data sets.

[0037] In view of this, the present disclosure proposes a decision model training method based on federated learning. The leakage of label data is reduced or avoided. Thus, the security of data (especially label data) in federated learning is improved. In addition, according to the decision model training method of the embodiment of the present disclosure, the communication volume between two parties (or multiple parties) can be greatly reduced, thereby improving the efficiency of federated learning and reducing the computing pressure of computing resources.

[0038] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0039] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0040] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

[0041] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications capable of executing a decision model training method based on federated learning (for example, a decision model training method applied to a first participant holding multiple sample data in a federated learning system according to an embodiment of the present disclosure or a decision model training method applied to a second participant holding multiple label data in a federated learning system).

[0042] In some embodiments, server 120 may also provide other services or software applications that may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0043] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0044] The client devices 101, 102, 103, 104, 105 and / or 106 may also run one or more services or software applications that are capable of executing a decision model training method based on federated learning (e.g., a decision model training method applied to a first participant holding multiple sample data in a federated learning system or a decision model training method applied to a second participant holding multiple label data in a federated learning system according to an embodiment of the present disclosure). The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Although Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure may support any number of client devices.

[0045] Client devices 101, 102, 103, 104, 105 and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. Client devices are capable of executing various different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.

[0046] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0047] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0048] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0049] In some embodiments, the server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from users of the client devices 101, 102, 103, 104, 105, and / or 106. The server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of the client devices 101, 102, 103, 104, 105, and / or 106. In some embodiments, the server 120 may include one or more applications, such as applications for services such as target detection and recognition, signal conversion, etc. based on data such as images, videos, voices, texts, digital signals, etc., to process task requests such as voice interaction, text classification, image recognition, or key point detection received from the client devices 101, 102, 103, 104, 105, and / or 106. The server may train a neural network model using training samples according to a specific deep learning task, and may test each subnetwork in a supernetwork module of the neural network model, and determine the structure and parameters of the neural network model for performing the deep learning task according to the test results of each subnetwork. Various data can be used as training sample data for deep learning tasks, such as image data, audio data, video data or text data, etc. After the training of the neural network model is completed, the server 120 can also automatically search for the optimal model structure through model search technology to perform the corresponding task.

[0050] In some embodiments, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0051] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The databases 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The databases 130 may be of different types. In some embodiments, the databases used by the server 120 may be, for example, relational databases. One or more of these databases may store, update, and retrieve data to and from the databases in response to commands.

[0052] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0053] Figure 1 The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0054] Figure 2 A flowchart of a decision model training method 200 based on federated learning according to an embodiment of the present disclosure is shown.

[0055] The method 200 is applied to a first participant holding a plurality of sample data in a federated learning system, and the federated learning system further includes a second participant in communication connection with the first participant.

[0056] According to some embodiments, each sample data among the multiple sample data held by the first participant includes a sample identification ID and at least one attribute value, and the second participant holds multiple label data, and the multiple label data are respectively associated with corresponding sample data among the multiple sample data.

[0057] like Figure 2 As shown, the method 200 includes:

[0058] Step S210: sorting part of the sample data based on one attribute value of at least one attribute value of part of the sample data among the plurality of sample data;

[0059] Step S220, dividing the sorted sample data into a plurality of data groups;

[0060] For each data packet, the method 200 further includes:

[0061] Step S230, generating a sample identification ID code based on a sample identification ID set formed by a sample identification ID of at least one sample data in the data group;

[0062] Step S240, sending the sample identification ID code to the second participant;

[0063] Step S250, receiving from the second participant a count of each label value in at least one label value corresponding to the data packet; and

[0064] Step S260: Determine a splitting gain corresponding to the data group based on the count, where the splitting gain indicates the heterogeneity of the classification result expected to be obtained after the decision model performs node splitting at the data group.

[0065] In addition, the method 200 further includes step S270, determining the local splitting nodes included in the decision model based on the splitting gain corresponding to each data group, thereby training the decision model.

[0066] According to method 200, the first participant holding the sample data groups the sorted sample data, and then for each data group, sends the sample identification ID code corresponding to the group to the second participant, and receives the count of each label value in at least one label value corresponding to the data group from the second participant, and trains the decision model based on the label value technology. It can be seen that the first participant does not send the full amount of sample data it holds to the second participant holding the label data, and does not send the attribute value of the label data to the second participant, thereby avoiding the leakage of the attribute information of the sample data held by the first participant.

[0067] In addition, the first participant receives the count of each label value corresponding to each group from the second participant, but does not receive the label data itself, thereby reducing or avoiding the leakage of the label data of the second participant. Especially for training decision models or classification models, since the first participant does not receive the label data itself or the encrypted value of the label data, it is difficult for the first participant to reversely infer the original plaintext label data from the count of the label value. The count of label values ​​can be used to train decision models, thereby reducing or avoiding the leakage of the label data of the second participant on the basis of ensuring the model training effect. This improves the security of data (especially label data) in federated learning.

[0068] The federated learning system may be a vertical federated learning system. The sample identification IDs of the multiple sample data held by the first participant of the federated learning system may be unique numbers of the sample data, and the sample identification IDs of the label data held by the second participant may be numbers corresponding to the sample identification IDs of the multiple sample data held by the first participant. It should be understood that the second participant may also hold sample data corresponding to the label data. For the sake of simplicity in describing the scheme, the sample data of the second participant is not involved in the following description, and only the label data held by it is described.

[0069] The attributes of the sample data may be the characteristics of the sample data. For example, the sample data may include the user's age, education level, consumption amount, etc. Among them, age, education level, and consumption amount may be the attributes (or characteristics) of the sample data in this article; and the specific amounts of age, education level, and consumption amount may be the attribute values ​​(or characteristic values) of the sample data in this article. It should be understood that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good customs.

[0070] In some examples, in step S210, sorting the partial sample data based on one attribute value of at least one attribute value of the partial sample data in the plurality of sample data may include: sorting the partial sample data in ascending order or descending order according to one attribute value of the sample data. The sorted data has a certain regularity, which is convenient for grouping the sample data. For example, the sample data may be sorted in ascending order of age according to the age of the sample data. For another example, the sample data may be sorted in descending order of education level according to the education level of the sample data.

[0071] In some examples, in step S220, dividing the sorted sample data into a plurality of data groups may include performing average grouping or chi-square grouping on the sorted sample data. For example, 10,000 sample data items sorted in ascending order of age are distributed into 10 evenly divided data groups, wherein each data group has a numerical interval of 10 years old, that is, the 10 groups may be 1-10 years old, 11-20 years old, ... 91-100 years old, respectively.

[0072] By grouping the data, on the one hand, it is possible to avoid subsequent communication with the second party based on the full sample of the first party, thereby improving data security. On the other hand, it is also possible to reduce the amount of data communication, thereby increasing the speed of model training and reducing the computing pressure of computing resources. In addition, through data grouping, some continuous attribute values ​​(feature values) can be preprocessed to discretize the continuous attribute values, so that model training can be performed based on the discretized attribute values. Moreover, the discretized attributes can also perform feature crossover, thereby reducing the occurrence of overfitting of the machine learning model.

[0073] It should be understood that different data grouping rules may be selected according to different attributes of the sample data, which is not limited here.

[0074] According to some embodiments, the sample identification ID code may include a sample identification ID code array having multiple storage bits, and the above-mentioned step S230 may include: for each sample data in the data group, according to the first mapping rule, mapping the sample identification ID of the sample data to some storage bits in the multiple storage bits.

[0075] refer to Figure 3 , Figure 3 FIG. 2 is a schematic diagram showing a partial process of a decision model training method 200 based on federated learning according to an embodiment of the present disclosure. Figure 3 As shown, the sample identification ID code may include a sample identification ID code array 310 having 12 storage bits. Here, the storage bit may be a digital bit or a binary bit in the computer science field.

[0076] For example, for a sample data, the sample identification ID ( Figure 3 A) in the sample ID encoding array 310 are respectively mapped to the first, third, and seventh storage bits of the 12 storage bits of the sample ID encoding array 310. For another sample data, the sample ID ( Figure 3 B) in the figure is respectively mapped to the 5th, 8th and 10th storage bits of the 12 storage bits of the sample identification ID encoding array 310.

[0077] In one example, the storage bits may be binary bits, and multiple different hash functions may be used to process the sample identification ID of each sample data to obtain multiple different hash values, and the positions in the ID encoding array 310 pointed to by the hash values ​​(for example, the 1st, 3rd, 7th storage bits or the 5th, 8th, 10th storage bits) are set to 1, and the remaining storage bits may be set to 0 respectively.

[0078] Thus, the sample identification ID of each sample data in the data group can be mapped to some storage bits in the multiple storage bits of the coding array 310, and the coding array 310 is used as the sample identification ID code, which contains the information related to the sample identification ID of each sample data in the data group. When the first participant sends the sample identification ID code, the amount of data sent is small, thereby improving the efficiency of information communication and reducing the communication burden and memory consumption. In addition, the sample identification ID is properly encrypted, which can also improve the security of the sample identification ID code during transmission.

[0079] Figure 4 A flowchart of a partial process of a decision model training method 200 based on federated learning according to an embodiment of the present disclosure is shown.

[0080] like Figure 4 As shown, according to some embodiments, the step of mapping the sample identification ID of the sample data to some storage bits in the plurality of storage bits according to the first mapping rule may include:

[0081] Step S431, dividing the sample identification ID of the sample data into multiple sub-parts;

[0082] Step S432: encrypt each sub-part to obtain an encrypted string; and

[0083] Step S433: Map the encrypted Chen character string corresponding to each sub-part to some storage bits among the multiple storage bits.

[0084] refer to Figure 5 , Figure 5FIG. 2 is a schematic diagram showing a partial process of a decision model training method 200 based on federated learning according to an embodiment of the present disclosure. Figure 5 As shown, the sample identification ID code may include a sample identification ID code array 510 having 12 storage bits.

[0085] For example, for a sample data, the sample identification ID ( Figure 5 C) is divided into multiple sub-parts; each sub-part is encrypted to obtain encrypted strings C1, C2, and C3; the encrypted strings C1, C2, and C3 corresponding to each sub-part are mapped to the 1st, 3rd, and 7th storage bits of the 12 storage bits of the sample ID encoding array 510. For another sample data, the sample ID ( Figure 5 D) is divided into multiple sub-parts; each sub-part is encrypted to obtain encrypted strings D1, D2, and D3; the encrypted strings D1, D2, and D3 corresponding to each sub-part are mapped to the 5th, 8th, and 10th storage bits of the 12 storage bits of the sample identification ID encoding array 510 respectively.

[0086] In one example, the storage bit can be a digital bit in the field of computer science, such as a digital bit that can store a string. In one example, each sub-part can be encrypted using an algorithm such as HD5, SHA256, etc. to obtain an encrypted string. Figure 5 As shown, the positions in the ID encoding array 510 (e.g., the 1st, 3rd, 7th storage bits or the 5th, 8th, 10th storage bits) are respectively set to corresponding encrypted strings (e.g., C1, C2, C3, D1, D2, D3), and the remaining storage bits can be respectively set to N (null value).

[0087] The encrypted string further improves the data security of the sample identification ID and greatly reduces the possibility of data being overwritten during the encoding process, thereby improving the performance of the trained model.

[0088] According to some embodiments, the above step S260, based on the count, determining the splitting gain corresponding to the data packet may include:

[0089] Based on the count, determining the proportion of the number of each label value in at least one label value; and

[0090] The split gain is determined based on the proportion corresponding to each label value.

[0091] The above process will be described below with reference to the example in Table 1.

[0092] Table 1 shows sample identification IDs (1 to 10) of 10 samples in a data group, and label values ​​corresponding to each sample at the second participant (including three labels Label 1, Label 2, and Label 3).

[0093] Table 1

[0094]

[0095] Table 1 shows sample identification IDs (1 to 10) of 10 samples in a data group, and label values ​​corresponding to each sample at the second participant (including three labels Label 1, Label 2, and Label 3).

[0096] First, in step S250, the first participant receives from the second participant a count of each label value in at least one label value corresponding to the data group, the count including: 6 Label 1, 3 Label 2 and 1 Label 3. The first participant can then determine the splitting gain based on the proportion corresponding to each label value.

[0097] In one example, the split gain can be represented by the Gini index. The Gini index can indicate the heterogeneity (or purity) of the classification results expected to be obtained after the decision model performs node splitting at the data grouping. For the Gini index, the higher the sample purity, the smaller the Gini index. The Gini index can be calculated using the following equation:

[0098]

[0099] Wherein, j represents the split node, that is, the number of types of label values ​​corresponding to the sample. For example, for the above example of Label 1, Label 2, and Label 3, j = 3. j Represents the proportion of each type in the split node sample. For example, for the above example, p j They are 0.6, 0.3 and 0.1 respectively.

[0100] Further, according to the above equation, the Gini index is:

[0101] Gini=1-(0.6 2 +0.3 2 +0.1 2 )=0.54

[0102] Therefore, the corresponding splitting gain can be determined to be 0.54.

[0103] According to some embodiments, in step S270, based on the splitting gain corresponding to each data group, the local splitting nodes included in the decision model are determined, so that training the decision model may include: from the splitting nodes corresponding to each data group, selecting the splitting node with the smallest splitting gain as the local splitting node, wherein the smallest splitting gain indicates that the heterogeneity of the classification results expected to be obtained after the decision model performs node splitting at the data group is the largest.

[0104] Therefore, by selecting the splitting node with the largest heterogeneity (highest purity) as the local splitting node at the data grouping, the purity of the classification result expected to be obtained at the node can be the highest, and the classification effect can be better.

[0105] According to some embodiments, the method 200 may further include:

[0106] Sending the splitting attribute corresponding to the local splitting node to the second participant; and

[0107] In response to receiving the indication information from the second participant, the split attribute and the split threshold corresponding to the local split node are recorded.

[0108] Therefore, at the first participant, the splitting attributes and splitting thresholds of the model splitting nodes are recorded, while at the holder of the label data (i.e., the second participant), the splitting attributes and splitting thresholds of the model splitting nodes do not need to be recorded. As a result, the model structure in the federated learning system is simplified, thereby reducing the amount of data communication in the training process and the subsequent prediction process using the model, and can further reduce the computing pressure of computing resources.

[0109] According to some embodiments, the first party may include a plurality of computing units, and the method 200 may further include:

[0110] Divide the plurality of sample data into a plurality of data sets, wherein the sample data with the same attribute category are divided into the same data set, and each data set includes sample data with at least one attribute category. The sample data with the same attribute category in any one of the data sets is the part of the sample data in the plurality of sample data; and

[0111] Each data set is assigned to a corresponding one of the plurality of computing units so that each computing unit trains the decision model in parallel.

[0112] Thus, by dividing the sample data with the same attribute category into the same data set (i.e., dividing the sample data vertically according to the attribute category of the data), and assigning each data set to a corresponding one of the multiple computing units, the first participant can assign the heavy model training tasks to multiple computing units, and the decision model can be trained in parallel by multiple computing units, further improving the efficiency of model training. It is worth noting that the applicant has found that this parallel training method is particularly suitable for model training scenarios with large sample data volumes and more attributes of sample data (such as training scenarios of random forest models).

[0113] In one example, a computer process may be assigned to each computing unit, and each process independently completes the model training task of a computing unit.

[0114] In one example, the data set may be partitioned first, and then steps S210 - S270 in the method 200 may be respectively performed in each of the partitioned data sets.

[0115] Figure 6 A flowchart of a decision model training method 600 based on federated learning according to an embodiment of the present disclosure is shown;

[0116] The method 600 is applied to a second participant holding a plurality of label data in a federated learning system, and the federated learning system further includes a first participant in communication connection with the second participant.

[0117] According to some embodiments, each of the multiple label data held by the second participant includes a sample identification ID and a label value, and the first participant holds multiple sample data, and the multiple sample data are respectively associated with corresponding label data in the multiple label data.

[0118] like Figure 6 As shown, the method 600 includes:

[0119] Step S610: receiving a plurality of sample identification ID codes from a first participant, each of the plurality of sample identification ID codes being generated by the first participant based on a sample identification ID set consisting of at least one sample data divided into a corresponding data group among a plurality of sample data held by the first participant;

[0120] For each sample identification ID code, method 600 further includes:

[0121] Step S620: querying a label value of at least one label data associated with the sample identification ID code from multiple label data;

[0122] Step S630: determining the count of each label value in at least one label value corresponding to the corresponding data group based on the label value of at least one associated label data; and

[0123] Step S640: Send the count to the first participant, so that the first participant determines the local splitting nodes included in the decision model based on the count, thereby training the decision model.

[0124] According to method 600, by receiving a sample identification ID code from a first participant holding sample data, a second participant queries the tag value of at least one tag data associated with the sample identification ID code from the tag data held by it, and returns the count of each tag value to the first participant. Thus, on the one hand, the second participant does not obtain the full amount of sample data or attribute information of the sample data held by the first participant, which can avoid leakage of attribute information of the sample data held by the first participant.

[0125] On the other hand, the second participant sends the count of each label value corresponding to each group to the first participant, instead of sending the label data itself, thereby reducing or avoiding the leakage of the label data of the second participant. Especially for training decision models or classification models, since the second participant does not send the label data itself or the encrypted value of the label data, it is difficult for the first participant to reversely infer the original plaintext label data from the count of the label value. The count of label values ​​can be used to train decision models, so on the basis of ensuring the model training effect, the leakage of the label data of the second participant is reduced or avoided. This improves the security of data (especially label data) in federated learning.

[0126] refer to Figure 7 , Figure 7 A flowchart of a partial process of a decision model training method 600 based on federated learning according to an embodiment of the present disclosure is shown. According to some embodiments, the sample identification ID code may include a sample identification ID code array having multiple storage bits, and some of the multiple storage bits store the sample identification ID of the sample data mapped thereto by the first participant according to the first mapping rule. And the above step S620, querying the label value of at least one label data associated with the sample identification ID code from the multiple label data may include:

[0127] Step S721: according to a second mapping rule associated with the first mapping rule, parse the sample identification ID code to obtain a sample identification ID set corresponding to the sample identification ID code; and

[0128] Step S722: query the label value of the label data associated with each sample identification ID in the sample identification ID set from the multiple label data.

[0129] Since the sample identification ID code contains information related to the sample identification ID of each sample data in the corresponding data group, when the second participant receives the sample identification ID code, the amount of data to be processed is small, thereby improving the efficiency of information communication and reducing the communication burden and memory consumption. In addition, the sample identification ID has been properly encrypted, which can also improve the security of the sample identification ID code during transmission.

[0130] According to some embodiments, some of the multiple storage bits may store an encrypted string mapped therein by the first participant, where the encrypted string is obtained by encrypting multiple sub-parts obtained by dividing the sample identification ID of the sample data.

[0131] The encrypted string further improves the data security of the sample identification ID and greatly reduces the possibility of data being overwritten during the encoding process, thereby improving the performance of the trained model.

[0132] Figure 8 FIG. 6 is a flowchart showing a partial process of a decision model training method 600 based on federated learning according to an embodiment of the present disclosure. Figure 8 As shown, according to some embodiments, the above step S721 may include:

[0133] Step S821, in a selection vector having the same number of storage bits as the sample identification ID encoding array, map the sample identification ID of each tag data to part of the storage bits respectively; and

[0134] Step S822: Obtain a sample identification ID set corresponding to the sample identification ID code from the sample identification ID code array by determining the intersection of the selection vector and the sample identification ID code array.

[0135] For example, the second party may provide a sample ID encoding array (eg Figure 3 The sample identification ID encoding array 310 shown or for example Figure 5 In the sample identification ID encoding array 510 shown in the figure, the sample identification ID of each tag data is mapped to a part of the storage bits in the selection vector having the same number (e.g., 12). Then, by determining the intersection of the selection vector and the sample identification ID encoding array, the sample identification ID set corresponding to the sample identification ID code is obtained from the sample identification ID encoding array.

[0136] This can further ensure that the second participant will not obtain the full sample data of the first participant, but only query the corresponding label data for the multiple sample data corresponding to the sample ID encoding array, thereby further ensuring the security of the data. In addition, this method of finding the intersection significantly improves the correctness compared to related technologies.

[0137] The following will refer to Fig. 9 The above steps S821 and S822 are further explained. Fig. 9 A schematic diagram of a partial process of a decision model training method 600 based on federated learning according to an embodiment of the present disclosure is shown.

[0138] like Fig. 9 As shown, the second participant, for example, receives the sample identification ID encoding array 510 from the first participant. The second participant can map the sample identification ID (e.g., CZ) of each tag data to a portion of the storage bits in the selection vector 910 having the same number of storage bits as the sample identification ID encoding array 510.

[0139] Here, for the sake of brevity, only C and Z are shown, while several sample identification IDs between C and Z are not shown.

[0140] In one example, for example, the storage bits of the selection vector 910 may be binary bits. Multiple different hash functions (the same as the hash function used by the first participant) may be used to process the sample identification ID of each tag data to obtain multiple different hash values, and the positions in the selection vector 910 pointed to by the hash values ​​(for example, the 1st, 3rd, and 7th storage bits corresponding to C) are set to 1, and the remaining storage bits may be set to 0 respectively.

[0141] Furthermore, the intersection of the selection vector 910 and the sample identification ID code array 510 may be determined, and a sample identification ID set 920 corresponding to the sample identification ID code may be obtained from the sample identification ID code array 510 .

[0142] According to some embodiments, the federated learning system may further include a third party in communication connection with the second party, the third party holding a plurality of sample data, the plurality of sample data being associated with corresponding label data in the plurality of label data held by the second party. And the method 600 may further include:

[0143] Receiving a split attribute corresponding to a first local split node from a first participant;

[0144] receiving a split attribute corresponding to the second local split node from a third party;

[0145] From the first local splitting node and the second local splitting node, a node with the smallest splitting gain is selected as the global splitting node. The smallest splitting gain indicates that the heterogeneity of the classification results expected to be obtained by the decision model after performing node splitting at the data group is the largest; and

[0146] Record the split properties of the global split node and the party holding the global split node.

[0147] Thus, the splitting attributes of the global splitting node of the model and the participant holding the global splitting node are recorded at the second participant, while the structure of the entire model (such as the structure of the decision tree) does not need to be recorded at the holder of the sample data (i.e., the first participant). Thus, the model structure in the federated learning system is simplified, thereby reducing the amount of data communication in the training process and the subsequent prediction process using the model, and can further reduce the computing pressure of computing resources.

[0148] According to some embodiments, method 600 may further include: sending an instruction message to the first participant so that the first participant records the splitting attribute and splitting threshold corresponding to the first local splitting node. Thus, at the first participant, the splitting attribute and splitting threshold of the model splitting node are recorded, while at the holder of the label data (i.e., the second participant), there is no need to record the splitting attribute and splitting threshold of the model splitting node. Thus, the model structure in the federated learning system is simplified, thereby reducing the amount of data communication in the training process and the subsequent prediction process using the model, and can further reduce the computing pressure of computing resources.

[0149] According to an embodiment of the present disclosure, a decision model training device 1000 based on federated learning is also provided. Fig.10 A structural block diagram of a decision model training device 1000 based on federated learning applied to a first participant in a federated learning system according to an embodiment of the present disclosure is shown.

[0150] The device 1000 is applied to a first participant holding a plurality of sample data in a federated learning system, and the federated learning system also includes a second participant in communication connection with the first participant.

[0151] According to some embodiments, each sample data among the multiple sample data held by the first participant includes a sample identification ID and at least one attribute value, and the second participant holds multiple label data, and the multiple label data are respectively associated with corresponding sample data among the multiple sample data.

[0152] like Fig.10 As shown, the device 1000 includes: a sorting unit 1010, a grouping unit 1020, an encoding unit 1030, a coding sending unit 1040, a label value count receiving unit 1050, a splitting gain determining unit 1060 and a training unit 1070.

[0153] The sorting unit 1010 is configured to sort the partial sample data based on one attribute value of at least one attribute value of the partial sample data in the plurality of sample data; the grouping unit 1020 is configured to divide the sorted sample data into a plurality of data groups; for each data group: the encoding unit 1030 is configured to generate a sample identification ID code based on a sample identification ID set formed by the sample identification ID of at least one sample data in the data group; the encoding sending unit 1040 is configured to send the sample identification ID code to the second participant; the label value count receiving unit 1050 is configured to receive the count of each label value in at least one label value corresponding to the data group from the second participant; and the splitting gain determining unit 1060 is configured to determine the splitting gain corresponding to the data group based on the count, and the splitting gain indicates the heterogeneity of the classification result expected to be obtained after the decision model performs node splitting at the data group. In addition, the training unit 1070 is configured to determine the local splitting nodes included in the decision model based on the splitting gain corresponding to each data group, thereby training the decision model.

[0154] According to an embodiment of the present disclosure, a decision model training device 1100 based on federated learning is also provided. Fig.11 A structural block diagram of a decision model training device 1100 based on federated learning applied to a second participant in a federated learning system according to an embodiment of the present disclosure is shown.

[0155] The device 1100 is applied to a second participant holding multiple label data in a federated learning system, and the federated learning system also includes a first participant in communication connection with the second participant.

[0156] According to some embodiments, each of the multiple label data held by the second participant includes a sample identification ID and a label value, and the first participant holds multiple sample data, and the multiple sample data are respectively associated with corresponding label data in the multiple label data.

[0157] like Fig.11 As shown, the apparatus 1100 includes: an encoding receiving unit 1110 , a tag value query unit 1120 , a tag value counting unit 1130 and a tag value count sending unit 1140 .

[0158] The code receiving unit 1110 is configured to receive multiple sample identification ID codes from the first participant, each of the multiple sample identification ID codes is generated by the first participant based on a sample identification ID set consisting of the sample identification ID of at least one sample data divided into a corresponding data group from multiple sample data held by the first participant; for each sample identification ID code: the label value query unit 1120 is configured to query the label value of at least one label data associated with the sample identification ID code from multiple label data; the label value counting unit 1130 is configured to determine the count of each label value in at least one label value corresponding to the corresponding data group based on the label value of the associated at least one label data; and the label value count sending unit 1140 is configured to send the count to the first participant, so that the first participant determines the local splitting node included in the decision model based on the count, thereby training the decision model.

[0159] According to another aspect of the present disclosure, a federated learning system is provided, including a decision model training device 1000 based on federated learning for a first participant holding multiple sample data in the federated learning system as described above, and a decision model training device 1100 based on federated learning for a second participant holding multiple label data in the federated learning system as described above.

[0160] The following will be combined Fig.12 The decision model training method according to the embodiment of the present disclosure is further described.

[0161] Fig.12 A process diagram of model training using a decision model training method based on federated learning according to an embodiment of the present disclosure is shown.

[0162] like Fig.12 As shown, the first participant 1210 in the federated learning system holds multiple sample data, and the second participant 1220 holds multiple label data.

[0163] In step S1201, the first participant 1210 sorts part of the sample data based on one attribute value of at least one attribute value of part of the sample data in a plurality of sample data.

[0164] In step S1202, the first participant 1210 divides the sorted sample data into a plurality of data groups.

[0165] In step S1203, the first participant 1210 generates a sample identification ID code based on a sample identification ID set consisting of a sample identification ID of at least one sample data in the data group.

[0166] In step S1204 , the first party 1210 sends the sample identification ID code to the second party 1220 .

[0167] In step S1205, the second participant 1220 searches for a tag value of at least one tag data associated with the sample identification ID code from the multiple tag data.

[0168] In step S1206, the second participant 1220 determines the count of each tag value in at least one tag value corresponding to the corresponding data group based on the tag value of the associated at least one tag data.

[0169] In step S1207, the second party 1220 sends the count to the first party.

[0170] In step S1208, the first participant 1210 determines a splitting gain corresponding to the data packet based on the count.

[0171] In step S1209, the first participant 1210 determines the local splitting nodes included in the decision model based on the splitting gain corresponding to each data group, thereby training the decision model.

[0172] According to an embodiment of the present disclosure, an electronic device, a readable storage medium and a computer program product are also provided.

[0173] refer to Fig.13 , a block diagram of an electronic device 1300 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0174] like Fig.13As shown, the device 1300 includes a computing unit 1301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 132 or a computer program loaded from a storage unit 1308 into a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the device 1300 can also be stored. The computing unit 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0175] Multiple components in the device 1300 are connected to the I / O interface 1305, including: an input unit 1306, an output unit 1307, a storage unit 1308, and a communication unit 1309. The input unit 1306 can be any type of device that can input information to the device 1300. The input unit 1306 can receive input digital or character information and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone and / or a remote control. The output unit 1307 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator and / or a printer. The storage unit 1308 can include but is not limited to a disk, an optical disk. The communication unit 1309 allows the device 1300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as Bluetooth TM devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0176] The computing unit 1301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1301 performs the various methods and processes described above. For example, in some embodiments, the method described in the embodiments of the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1308. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1300 via ROM 1302 and / or communication unit 1309. When the computer program is loaded into RAM 1303 and executed by the computing unit 1301, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 1301 may be configured to perform the method according to the embodiments of the present disclosure in any other appropriate manner (e.g., by means of firmware).

[0177] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0178] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0179] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0180] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0181] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0182] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0183] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0184] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A decision model training method based on federated learning, applied to a first participant in a federated learning system holding multiple sample data, the federated learning system also including a second participant in communication connection with the first participant, the method comprising: sorting the part of the sample data based on one attribute value of at least one attribute value of the part of the sample data among the plurality of sample data; Dividing the sorted sample data into a plurality of data groups; For each data group: Generate a sample identification ID code based on a sample identification ID set formed by a sample identification ID of at least one sample data in the data group; Sending the sample identification ID code to the second participant; receiving from the second participant a count of each label value in at least one label value corresponding to the data packet; as well as Based on the count, determining a split gain corresponding to the data group, the split gain indicating the heterogeneity of the classification result expected to be obtained after the decision model performs node splitting at the data group; as well as Based on the splitting gain corresponding to each data group, the local splitting nodes included in the decision model are determined, so as to train the decision model.

2. The method according to claim 1, wherein: The sample identification ID code includes a sample identification ID code array having a plurality of storage bits, and wherein the sample identification ID set composed of the sample identification ID of at least one sample data in the data group is generated, and the sample identification ID code is generated comprising: For each sample data in this data group: According to the first mapping rule, the sample identification ID of the sample data is respectively mapped to some storage bits in the plurality of storage bits.

3. The method according to claim 2, wherein: Mapping the sample identification ID of the sample data to some storage bits in the plurality of storage bits respectively according to the first mapping rule comprises: Dividing the sample identification ID of the sample data into multiple sub-parts; encrypting each subpart to obtain an encrypted string; and The encrypted string corresponding to each sub-part is respectively mapped to the part of the storage bits in the plurality of storage bits.

4. The method according to claim 1, wherein: The determining, based on the count, a splitting gain corresponding to the data packet comprises: Based on the count, determining the proportion of each label value in the at least one label value; and The splitting gain is determined based on the ratio corresponding to each label value.

5. The method according to any one of claims 1 to 4, wherein: The determining of the local splitting nodes included in the decision model based on the splitting gain corresponding to each data group comprises: From the splitting nodes corresponding to each data group, a splitting node with the smallest splitting gain is selected as the local splitting node, wherein the smallest splitting gain indicates that the heterogeneity of the classification results expected to be obtained by the decision model after performing node splitting at the data group is the largest.

6. The method according to claim 5, further comprising: Sending the splitting attribute corresponding to the local splitting node to the second participant; as well as In response to receiving the indication information from the second participant, the split attribute and the split threshold corresponding to the local split node are recorded.

7. The method according to any one of claims 1 to 4, wherein: The first party includes a plurality of computing units, and the method further includes: Dividing the plurality of sample data into a plurality of data sets, wherein the sample data with the same attribute category are divided into the same data set, and each data set includes sample data with at least one attribute category, wherein the sample data with the same attribute category in any one of the data sets are the part of the sample data in the plurality of sample data; and Each data set is assigned to a corresponding one of the plurality of computing units, so that each computing unit trains the decision model in parallel.

8. A decision model training method based on federated learning, applied to a second participant in a federated learning system that holds multiple label data, the federated learning system also including a first participant in communication connection with the second participant, the method comprising: receiving a plurality of sample identification ID codes from the first participant, each of the plurality of sample identification ID codes being generated by the first participant based on a sample identification ID set consisting of at least one sample data grouped into a corresponding data group from a plurality of sample data held by the first participant; as well as For each sample ID code: Querying a label value of at least one label data associated with the sample identification ID code from the plurality of label data; Determine, based on the tag value of the associated at least one tag data, a count of each tag value in at least one tag value corresponding to the corresponding data group; as well as The count is sent to the first participant, so that the first participant determines the local split nodes included in the decision model based on the count, thereby training the decision model.

9. The method according to claim 8, wherein: The sample identification ID code includes a sample identification ID code array having a plurality of storage bits, and some of the plurality of storage bits store the sample identification ID of the sample data mapped thereto by the first participant according to the first mapping rule, and wherein querying a label value of at least one label data associated with the sample identification ID code from the plurality of label data includes: According to a second mapping rule associated with the first mapping rule, parsing the sample identification ID code to obtain a sample identification ID set corresponding to the sample identification ID code; and The label value of the label data associated with each sample identification ID in the sample identification ID set is queried from the plurality of label data.

10. The method according to claim 9, wherein: Some of the multiple storage bits store encrypted strings mapped therein by the first participating direction, and the encrypted strings are obtained by encrypting multiple sub-parts obtained by dividing the sample identification ID of the sample data.

11. The method according to claim 9 or 10, wherein: According to a second mapping rule associated with the first mapping rule, a sample identification ID set corresponding to the sample identification ID code is obtained by parsing the sample identification ID code, including: In a selection vector having the same number of storage bits as the sample identification ID encoding array, mapping the sample identification ID of each tag data to a portion of the storage bits respectively; and By determining the intersection of the selection vector and the sample identification ID code array, a sample identification ID set corresponding to the sample identification ID code is obtained from the sample identification ID code array.

12. The method according to any one of claims 8 to 10, wherein: The federated learning system further includes a third party in communication connection with the second party, the third party holding a plurality of sample data, the plurality of sample data being associated with corresponding label data among the plurality of label data held by the second party, and the method further includes: receiving, from the first participant, a split attribute corresponding to a first local split node; receiving a split attribute corresponding to the second local split node from the third party; Selecting a node with the smallest splitting gain from the first local splitting node and the second local splitting node as a global splitting node, wherein the smallest splitting gain indicates that the heterogeneity of the classification results expected to be obtained after the decision model performs node splitting at the data group is the largest; and The splitting attributes of the global splitting node and the party holding the global splitting node are recorded.

13. The method according to claim 12, further comprising: Sending instruction information to the first participant so that the first participant records the split attribute and the split threshold corresponding to the first local split node.

14. A decision model training device based on federated learning, applied to a first participant in a federated learning system holding multiple sample data, the federated learning system also includes a second participant in communication connection with the first participant, the device comprising: A sorting unit, a grouping unit, a coding unit, a coding sending unit, a label value counting receiving unit, a splitting gain determining unit and a training unit, wherein: The sorting unit is configured to sort the partial sample data based on one attribute value of at least one attribute value of the partial sample data among the plurality of sample data; The grouping unit is configured to divide the sorted sample data into a plurality of data groups; For each data group: The encoding unit is configured to generate a sample identification ID code based on a sample identification ID set formed by a sample identification ID of at least one sample data in the data group; The code sending unit is configured to send the sample identification ID code to the second participant; The label value count receiving unit is configured to receive, from the second participant, a count of each label value of at least one label value corresponding to the data packet; and The splitting gain determining unit is configured to determine a splitting gain corresponding to the data group based on the count, wherein the splitting gain indicates the heterogeneity of the classification result expected to be obtained after the decision model performs node splitting at the data group; and The training unit is configured to determine the local splitting nodes included in the decision model based on the splitting gain corresponding to each data group, so as to train the decision model.

15. A decision model training device based on federated learning, applied to a second participant in a federated learning system holding multiple label data, the federated learning system also includes a first participant in communication connection with the second participant, the device comprising: A coding receiving unit, a tag value query unit, a tag value counting unit and a tag value counting sending unit, wherein: The code receiving unit is configured to receive a plurality of sample identification ID codes from the first participant, each of the plurality of sample identification ID codes being generated by the first participant based on a sample identification ID set consisting of at least one sample data divided into a corresponding data group among a plurality of sample data held by the first participant; For each sample ID code: The tag value query unit is configured to query the tag value of at least one tag data associated with the sample identification ID code from the multiple tag data; The label value counting unit is configured to determine the count of each label value in the at least one label value corresponding to the corresponding data group based on the label value of the associated at least one label data; and The label value count sending unit is configured to send the count to the first participant, so that the first participant determines the local splitting nodes included in the decision model based on the count, thereby training the decision model.

16. A federated learning system, comprising: The decision model training device based on federated learning according to claim 14; as well as A decision model training device based on federated learning according to claim 15.

17. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 13.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

19. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images