Grouping method, device and system and storage medium

Abstract

The invention discloses a grouping method, device and system and a storage medium, and belongs to the field of communication. The method comprises the following steps: receiving n data sets from n clients; clustering the n data sets to obtain information of a plurality of first clusters; sending information of a plurality of first clusters to the management node; a plurality of first global clustering centers sent by the management node are received, the plurality of first global clustering centers are obtained by the management node based on information of a first clusters, the information of the a first clusters comes from part or all of the access nodes, a is an integer greater than 1, and a is an integer greater than 1; the plurality of first global clustering centers are used for indicating a plurality of groups; and determining the group to which each client belongs based on part or all of the plurality of first global clustering centers. The grouping efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular to a grouping method, device, system and storage medium. Background Art

[0002] The federated learning system is a distributed machine learning system for training models. The federated learning system includes a server and multiple clients, each of which locally includes training data and an artificial intelligence (AI) model. For each client, the client uses local training data to train the local AI model and sends a first data set to the server, the first data set including multiple network parameters of the trained AI model.

[0003] The server receives a first data set sent by each client, and divides the multiple clients into multiple groups based on the first data set sent by each client. For each group, the network parameters included in the first data set sent by each client included in the group are aggregated to obtain a group model corresponding to the group, and a second data set is sent to each client in the group, the second data set including multiple network parameters of the group model. The clients in the group adjust the local AI model based on the second data set.

[0004] The total data volume of the first data sets sent by the multiple clients is often very large, which results in a large amount of calculation for the server to perform grouping operations based on the first data sets sent by each client, thereby reducing grouping efficiency. Summary of the invention

[0005] The present application provides a grouping method, device, system and storage medium to improve grouping efficiency. The technical solution is as follows:

[0006] In a first aspect, the present application provides a federated learning system, the federated learning system comprising a management node, a plurality of access nodes, and a plurality of clients connected to each access node, the plurality of access nodes including a first access node, the first client being one of n clients connected to the first access node, and n being an integer greater than 1. The first client is used to send a data set to the first access node, the n clients connected to the first access node including the first client, and the data set is used to describe a first artificial intelligence AI model in the first client. The first access node is used to cluster the data sets from the n clients to obtain information of a plurality of first clusters, and to send information of a plurality of first clusters to the management node. The management node is used to obtain a plurality of first global cluster centers based on information of a first clusters, the information of a first clusters comes from part or all of the plurality of access nodes, a is an integer greater than 1, and the plurality of first global cluster centers are used to indicate a plurality of groups of clients. The management node is also used to send the plurality of first global cluster centers to part or all of the plurality of access nodes. The first access node is also used to obtain the group to which each of the n clients belongs based on part or all of the plurality of first global cluster centers.

[0007] Since the federated learning system includes multiple access nodes, each access node receives a data set sent by a respective accessed client, and each access node performs clustering operations on the received data set in parallel to obtain information of multiple first clusters. The first access node sends information of multiple first clusters to the management node. The amount of data in the information of the first cluster is small, and the management node quickly obtains multiple first global cluster centers based on the received information of the first cluster. The first access node determines the group to which each client belongs based on the multiple first global cluster centers, thereby improving the grouping efficiency.

[0008] In a possible implementation, the n data sets are first data sets from n clients. The first access node is used to perform sparse processing on the n first data sets to obtain n second data sets, the n first data sets correspond to the n second data sets, and for each first data set, the data volume of the first data set is less than the data volume of the second data set corresponding to the first data set, and clustering operations are performed on the n second data sets to obtain information of the multiple first clusters. This can reduce the amount of data that needs to be calculated for the clustering operation and improve the efficiency of the clustering operation.

[0009] In another possible implementation, the n data sets are second data sets from n clients. The first client is used to perform sparse processing on the first data set to obtain the second data set, the first data set is used to describe the first AI model, the data volume of the second data set is less than the data volume of the first data set, and the second data set is sent to the first access node. In this way, the network resources required for transmitting the second data set are reduced, and the amount of data for clustering operations on the second data set is reduced.

[0010] In another possible implementation, the first data set includes multiple first data for describing the first AI model, and the second data set includes m first data and the serial numbers of the m first data in the first data set, where m is an integer greater than 1, and the m first data are the first data in the first data set that meet the first condition. The first access node is used to perform clustering operations based on the m first data and the serial numbers of the m first data included in each of the n second data sets to obtain information of multiple first clusters. Among them, the m first data that meet the first condition are data with significant features in the first data set, and clustering operations are performed based on the m first data and the serial numbers of the m first data included in each of the n second data sets to avoid affecting the accuracy of clustering.

[0011] In another possible implementation, the first access node is used to cluster n data sets based on multiple first global cluster centers to obtain multiple second clusters, the multiple first global cluster centers correspond to the multiple second clusters, and for each second cluster, the second cluster includes a partial data set in the n data sets, the partial data set and the first global cluster center corresponding to the second cluster meet the first similarity condition, and the client corresponding to each data set included in the second cluster is determined as a group. In this way, the grouping of each client can be accurately determined through the multiple first global cluster centers.

[0012] In another possible implementation, the information of the first cluster includes at least one data set in the first cluster, the at least one data set and the center of the first cluster satisfy the second similarity condition, and the management node is used to perform clustering operations on the data sets included in the information of the a first cluster to obtain multiple first global clusters, and the multiple first global cluster centers include the centers of the multiple first global clusters. Thereby, the number of data sets included in the information of the first cluster can be reduced, and the amount of data in the information of the first cluster can be reduced.

[0013] In another possible implementation, the information of the first cluster includes the center of the first cluster, and the management node is used to obtain multiple first global cluster centers based on the weights corresponding to the a first clusters and the centers of the a first clusters. The center of the first cluster included in the information of the first cluster reduces the data volume of the information of the first cluster.

[0014] In another possible implementation, the multiple first global cluster centers do not meet the convergence condition. The first access node is used to cluster n data sets based on the multiple first global cluster centers to obtain multiple third clusters. The first access node is also used to send information of multiple third clusters to the management node. The management node is also used to obtain multiple second global cluster centers based on the information of b third clusters, the information of b third clusters comes from part or all of the multiple access nodes, b is an integer greater than 1, and the multiple second global cluster centers are sent to part or all of the multiple access nodes. The first access node is also used to obtain the group to which each client belongs based on the multiple second global cluster centers when the multiple second global cluster centers meet the convergence condition. In this way, multiple global cluster centers can be obtained cyclically until multiple global cluster centers that meet the convergence condition are obtained, and the grouping accuracy is improved by multiple global cluster centers that meet the convergence condition.

[0015] In another possible implementation, the n clients include multiple second clients belonging to a first group, and the multiple groups include the first group. The first access point is also used to receive multiple first data sets, which are data sets corresponding to the multiple second clients, and the multiple first data sets are used to describe the first AI model in the multiple second clients, and send a third data set to the management node, and the third data set is obtained based on the multiple first data sets. The management node is also used to obtain the group model corresponding to the first group based on multiple third data sets, and the multiple third data sets come from part or all of the multiple access nodes, and send a fourth data set to the multiple access nodes, and the fourth data set is used to describe the group model corresponding to the first group. The first access node is also used to send a fourth data set to the multiple second clients. In this way, a distributed method is used to obtain the group model corresponding to the first group, thereby improving the efficiency of obtaining the group model.

[0016] In a second aspect, the present application provides a grouping method, which is applied to a first access node, and the first access node is any one of the multiple access nodes included in the federated learning system. The federated learning system also includes a management node and n clients connected to the first access node, where n is an integer greater than 1. In the method, n data sets are received from the n clients. The n data sets are clustered to obtain information of multiple first clusters. The information of multiple first clusters is sent to the management node. Multiple first global cluster centers sent by the management node are received, and the multiple first global cluster centers are obtained by the management node based on the information of a first clusters, and the information of a first clusters comes from part or all of the multiple access nodes, a is an integer greater than 1, and the multiple first global cluster centers are used to indicate multiple groups of clients. The group to which each client belongs is obtained based on part or all of the multiple first global cluster centers.

[0017] Since the federated learning system includes multiple access nodes, each access node receives a data set sent by a respective accessed client, and each access node performs clustering operations on the received data set in parallel to obtain information of multiple first clusters. The first access node sends information of multiple first clusters to the management node. The amount of data in the information of the first cluster is small, and the management node quickly obtains multiple first global cluster centers based on the received information of the first cluster. The first access node determines the group to which each client belongs based on the multiple first global cluster centers, thereby improving the grouping efficiency.

[0018] In a possible implementation, the n data sets are clustered based on the multiple first global cluster centers to obtain multiple second clusters, the multiple first global cluster centers correspond to the multiple second clusters, and for each second cluster, the second cluster includes a partial data set in the n data sets, and the partial data set and the first global cluster center corresponding to the second cluster meet the first similarity condition. The client corresponding to each data set included in the second cluster is determined as a group. In this way, the grouping of each client can be accurately determined by the multiple first global cluster centers.

[0019] In another possible implementation, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies the second similarity condition with the center of the first cluster, thereby reducing the number of data sets included in the information of the first cluster and reducing the amount of data in the information of the first cluster.

[0020] In another possible implementation, the information of the first cluster includes the center of the first cluster, and the multiple first global cluster centers do not meet the convergence condition. Based on the multiple first global cluster centers, the n data set is clustered to obtain multiple third clusters. The information of the multiple third clusters is sent to the management node. The multiple second global cluster centers sent by the management node are received, and the multiple second global cluster centers are used to indicate the multiple groups. When the multiple second global cluster centers meet the convergence condition, the group to which each client belongs is obtained based on the multiple second global cluster centers. In this way, multiple global cluster centers can be obtained cyclically until multiple global cluster centers that meet the convergence condition are obtained, and the grouping accuracy is improved by the multiple global cluster centers that meet the convergence condition.

[0021] In another possible implementation, the n data sets are n second data sets from n clients. For each client, the second data set corresponding to the client is obtained by the client performing sparse processing on the first data set. The first data set is used to describe the first AI model in the client, and the data volume of the second data set is smaller than that of the first data set. This reduces the network resources required to transmit the second data set and reduces the data volume for clustering operations on the second data set.

[0022] In another possible implementation, the first data set includes multiple first data for describing the first AI model obtained by the client performing model training, and the second data set includes m first data and the sequence numbers of the m first data in the first data set, where m is an integer greater than 1, and the m first data are the first data in the first data set that meet the first condition. A clustering operation is performed based on the m first data and the sequence numbers of the m first data included in each of the n second data sets to obtain information of the multiple first clusters.

[0023] Among them, the m first data that meet the first condition are data with significant features in the first data set, and clustering operations are performed based on the m first data included in each of the n second data sets and the serial numbers of the m first data to avoid affecting the accuracy of clustering.

[0024] In another possible implementation, the n data sets are n first data sets from n clients, and for each client, the first data set corresponding to the client is used to describe the first AI model in the client. The n first data sets are sparsely processed to obtain n second data sets, and the n first data sets correspond to the n second data sets. For each first data set, the amount of data in the first data set is less than the amount of data in the second data set corresponding to the first data set. Clustering operations are performed on the n second data sets to obtain information of multiple first clusters. This can reduce the amount of data that needs to be calculated for clustering operations and improve the efficiency of clustering operations.

[0025] In another possible implementation, multiple first data sets sent by multiple clients in a first group are received, the n clients include multiple clients, the multiple first data sets are used to describe the first AI model in the multiple clients, and the multiple groups include the first group. A third data set is sent to the management node, and the third data set is obtained based on the multiple first data sets. A fourth data set is received from the management node, and the fourth data set is used to describe the group model corresponding to the first group. The fourth data set is sent to each client in the first group. In this way, a distributed method is used to obtain the group model corresponding to the first group, thereby improving the efficiency of obtaining the group model.

[0026] In a third aspect, the present application provides a grouping method, which is applied to a management node in a federated learning system, wherein the federated learning system includes multiple access nodes and multiple clients connected to each access node, wherein the multiple access nodes include a first access node. In the method, information of a first cluster is received, wherein the information of a first cluster comes from part or all of the multiple access nodes, a is an integer greater than 1, and the information of the first cluster sent by the first access node is obtained by clustering n data sets, n is an integer greater than 1, and the n data sets come from n clients connected to the first access node. Based on the information of a first cluster, multiple first global cluster centers are obtained, and the multiple first global cluster centers are used to indicate multiple groups of clients. The multiple first global cluster centers are sent to part or all of the multiple access nodes, and the first access node is used to obtain the group to which each of the n clients belongs based on part or all of the multiple first global cluster centers.

[0027] Since the federated learning system includes multiple access nodes, each access node receives a data set sent by a respective accessed client, and each access node performs clustering operations on the received data set in parallel to obtain information of multiple first clusters. The management node receives information of multiple first clusters sent by each access node. The amount of data of the first cluster information is relatively small, and the management node quickly obtains multiple first global cluster centers based on the information sent by some or all of the multiple access nodes, and sends the multiple first global cluster centers to some or all of the multiple access nodes. The first access node determines the group to which each client belongs based on the multiple first global cluster centers, thereby improving the grouping efficiency.

[0028] In one possible implementation, for each first cluster received, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set and the center of the first cluster satisfy the second similarity condition. A clustering operation is performed on the data sets included in the information of a first cluster to obtain multiple first global clusters, and the multiple first global cluster centers include the centers of the multiple first global clusters. Since the information of the first cluster includes at least one data set in the first cluster, the number of data sets included in the information of the first cluster can be reduced, and the amount of data of the information of the first cluster can be reduced, that is, the amount of data of the clustering operation can be reduced, and the efficiency of the clustering operation can be improved.

[0029] In another possible implementation, the information of the first cluster includes the center of the first cluster. Based on the weights corresponding to the a first clusters and the centers of the a first clusters, multiple first global cluster centers are obtained. Since the information of the first cluster includes the center of the first cluster, the amount of data of the information of the first cluster can be reduced, that is, the amount of data that needs to be calculated when obtaining the global cluster center is reduced.

[0030] In another possible implementation, the multiple first global cluster centers do not meet the convergence condition. Receive information of b third clusters, the information of the b third clusters comes from some or all of the multiple access nodes, and the multiple third clusters sent by the first access node are obtained by clustering the n data sets by the first access node based on the multiple first global cluster centers. Based on the information of the b third clusters, obtain multiple second global cluster centers. Send the multiple second global cluster centers to some or all of the multiple access nodes. In this way, multiple global cluster centers can be obtained cyclically until multiple global cluster centers that meet the convergence condition are obtained, and the grouping accuracy is improved by multiple global cluster centers that meet the convergence condition.

[0031] In another possible implementation, multiple third data sets are received, the multiple third data sets are from some or all of the multiple access nodes, and the third data set sent by the first access node is used to describe the first AI module in multiple clients belonging to the first group, the n clients include the multiple clients, and the multiple groups include the first group. The group model corresponding to the first group is obtained based on the multiple third data sets. A fourth data set is sent to some or all of the multiple access nodes, and the fourth data set is used to describe the group model. In this way, the group model corresponding to the first group is obtained in a distributed manner, thereby improving the efficiency of obtaining the group model.

[0032] In a fourth aspect, the present application provides a grouping method, the method being applied to a client, the client including training data. In the method, model training is performed based on the training data to obtain a first artificial intelligence AI model. The first data set is sparsely processed to obtain a second data set, the first data set is used to describe the first AI model, and the amount of data in the second data set is less than the amount of data in the first data set. The second data set is sent, and the second data set is used to request to determine the group to which the client belongs.

[0033] Since the second data set is obtained by performing sparse processing on the first data set, the data volume of the second data set is smaller than that of the first data set. Thus, when the second data set of each client is used for grouping, the data volume of the grouping operation is reduced, thereby improving the grouping efficiency.

[0034] In one possible implementation, the first data set includes multiple first data for describing the first AI model. Select m first data satisfying the first condition from the first data set, where m is an integer greater than or equal to 1. Obtain the second data set based on the m first data. Among them, the m first data satisfying the first condition are data with significant features in the first data set, and the second data set is obtained based on the m first data. The second data set can be used instead of the first data set for grouping to avoid affecting the accuracy of the grouping.

[0035] In another possible implementation, the first condition includes that the m first data are the m data with the largest absolute value in the first data set, or the m first data are the m data with values ​​in the first data set that are within a first range, so as to ensure that the selected m first data are data with significant features in the first data set.

[0036] In another possible implementation, the second data set includes m first data and the serial numbers of the m first data in the first data set; or, the second data set includes m first bits and xm second bits, x is the number of first data included in the first data set, x is an integer greater than m, the m first bits correspond to the m first data, the xm second bits correspond to the xm first data, and the xm first data are the data in the first data set other than the m first data.

[0037] In another possible implementation, m first data are quantized to obtain m second data, the m first data correspond to the m second data, the bit width of each first data is greater than the bit width of the second data corresponding to each first data, and the second data set includes the m second data and the sequence number of the first data corresponding to each second data in the first data set. Thus, the amount of data in the second data set can be reduced.

[0038] In another possible implementation, the type of the first data set is a matrix or a vector. A fifth data set is obtained, and the dimension of the fifth data set is smaller than the dimension of the first data set; and at least one operation is performed on the fifth data set and the first data set to obtain a second data set. In this way, sparseness is achieved by reducing the dimension, thereby improving the flexibility of sparseness.

[0039] In another possible implementation, the first data set is obtained based on network parameters of the second AI model and / or network parameters of the first AI model, and the first AI model is obtained by training the second AI model based on the training data.

[0040] In another possible implementation, a first data set is sent, a fourth data set is received, and the fourth data set is used to describe a group model corresponding to the group, where the group model is obtained based on the first data set sent by each client in the group, and a network parameter of the first AI model is adjusted based on the fourth data set.

[0041] In a fifth aspect, the present application provides a grouping device, which is used to execute the method in the second aspect or any possible implementation of the second aspect. Specifically, the device includes a unit for executing the method in the second aspect or any possible implementation of the second aspect.

[0042] In a sixth aspect, the present application provides a grouping device, which is used to execute the method in the third aspect or any possible implementation of the third aspect. Specifically, the device includes a unit for executing the method in the third aspect or any possible implementation of the third aspect.

[0043] In a seventh aspect, the present application provides a grouping device for executing the method in the fourth aspect or any possible implementation of the fourth aspect. Specifically, the device includes a unit for executing the method in the fourth aspect or any possible implementation of the fourth aspect.

[0044] In an eighth aspect, the present application provides a grouping device comprising at least one processor and a memory, wherein the at least one processor is used to couple with the memory, read and execute instructions in the memory, so as to implement the method in the second aspect or any possible implementation manner of the second aspect.

[0045] In a ninth aspect, the present application provides a grouping device comprising at least one processor and a memory, wherein the at least one processor is used to couple with the memory, read and execute instructions in the memory, so as to implement the method in the third aspect or any possible implementation manner of the third aspect.

[0046] In the tenth aspect, the present application provides a grouping device, comprising at least one processor and a memory, wherein the at least one processor is used to couple with the memory, read and execute instructions in the memory to implement the method in the fourth aspect or any possible implementation of the fourth aspect.

[0047] In the eleventh aspect, the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium, and the computer program is loaded by a processor to implement the method of the above-mentioned second aspect, third aspect, fourth aspect, any possible implementation of the second aspect, any possible implementation of the third aspect, or any possible implementation of the fourth aspect.

[0048] In the twelfth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program is loaded by a processor to execute the method of the second aspect, the third aspect, the fourth aspect, any possible implementation of the second aspect, any possible implementation of the third aspect, or any possible implementation of the fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a structural diagram of a federated learning system provided in an embodiment of the present application;

[0050] Figure 2 is a structural diagram of another federated learning system provided in an embodiment of the present application;

[0051] Figure 3 is a structural diagram of another federated learning system provided in an embodiment of the present application;

[0052] Figure 4It is a flow chart of a grouping method provided in an embodiment of the present application;

[0053] Figure 5 is a flow chart of another grouping method provided in an embodiment of the present application;

[0054] Figure 6 is a flow chart of another grouping method provided in an embodiment of the present application;

[0055] Figure 7 is a flow chart of another grouping method provided in an embodiment of the present application;

[0056] Figure 8 is a flow chart of another grouping method provided in an embodiment of the present application;

[0057] Fig. 9 is a flow chart of another grouping method provided in an embodiment of the present application;

[0058] Fig.10 It is a schematic diagram of the structure of a grouping device provided in an embodiment of the present application;

[0059] Fig.11 is a schematic diagram of another structure of a grouping device provided in an embodiment of the present application;

[0060] Fig.12 is a schematic diagram of another structure of a grouping device provided in an embodiment of the present application;

[0061] Fig.13 It is a schematic diagram of a device structure provided in an embodiment of the present application;

[0062] Fig.14 is another schematic diagram of a device structure provided in an embodiment of the present application;

[0063] Fig.15 is another schematic diagram of a device structure provided in an embodiment of the present application;

[0064] Fig.16 It is a structural diagram of another federated learning system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0066] As the application of artificial intelligence technology continues to become more popular, the privacy and security of users' personal data has received increasing attention. Artificial intelligence technology requires training models. The system used to train the model includes a server and multiple clients, and each client locally includes training data. For each client, the local training data of the client may be the user's personal data, including the user's privacy. Each client sends the local training data to the server, and the server trains the model based on the training data sent by each client, and sends the trained model to each client. However, the training data sent by each client may be leaked, resulting in the leakage of the user's personal privacy.

[0067] For example, if you need to train an information recommendation model, the local training data of each client includes the records of user browsing information. The local training data of the client can reflect the user's browsing preferences. For example, some users may like to browse entertainment information, some users may like to browse current affairs information, and some users may like to browse financial information. If each client sends local training data to the server, it is possible to leak the training data, and others can infer the user's preferences through the training data, resulting in the leakage of personal privacy.

[0068] In order to protect personal privacy and improve the accuracy of training AI models. For each client, the client performs model training based on local training data to obtain an AI model, and sends a data set to the server, which includes multiple network parameters of the AI ​​model. The server performs clustering operations based on the data set sent by each client to obtain multiple clusters, and the multiple clusters correspond one-to-one to multiple groups, and the clients corresponding to each data set included in the same cluster are divided into a group. For each group, the group model corresponding to the group is obtained based on the data set sent by each client in the group, and multiple network parameters of the group model are sent to each client in the group. Optionally, the multiple network parameters of the group model are obtained by weighted averaging the data set sent by each client. Each client in the group adjusts the local AI model based on the multiple network parameters of the group model, and the function of the adjusted AI model is the same as that of the group model.

[0069] If the group model has not converged, each client in the group still trains the adjusted AI model based on the local training data, and sends a data set to the server, which includes multiple network parameters of the trained AI model. The server obtains the group model corresponding to the group based on the data set sent by each client in the group, and sends multiple network parameters of the group model to each client in the group. Each client in the group adjusts the local AI model based on the multiple network parameters of the group model. Repeat the above process until the group model converges.

[0070] The AI ​​models included in each client in the same group may have similar or identical functions. For example, still taking the AI ​​model recommended by the above information as an example, for each client, the client performs model training based on local training data (records of user browsing information) to obtain an AI model, and sends a data set to the server. The data set includes multiple network parameters of the AI ​​model, and the multiple network parameters can reflect the user's browsing information preferences. The server performs clustering operations based on the data set sent by each client to obtain multiple clusters, and the multiple clusters correspond one-to-one to multiple groups, and the clients corresponding to each data set included in the same cluster are divided into a group. Based on the data set sent by each client in the group, the group model corresponding to the group is obtained, and multiple network parameters of the group model are sent to each client in the group. Each client in the group adjusts the local AI model based on the network parameters of the group model.

[0071] The browsing information preferences of the users corresponding to each client in the group may be the same, and the group model corresponding to the group is used to recommend information corresponding to the preferences. For example, assuming that the users corresponding to each client in the group like to browse entertainment information, the group model corresponding to the group is more inclined to recommend entertainment information to the users. Each client in the group adjusts the local AI model based on the network parameters of the group model, so that the AI ​​model in each client in the group is more inclined to recommend entertainment information that meets the personal preferences of the users.

[0072] Among them, the clustering operation itself is a computational process with high computational complexity, and the data set sent by each client includes multiple network parameters of the AI ​​model. The network parameters of the AI ​​model may be thousands, tens of thousands, or millions, or even more. Therefore, the amount of data set sent by each client is very large. In this way, the server performs clustering operations on the data sets of each client, resulting in a very large amount of calculation for the clustering operation, which reduces the grouping efficiency.

[0073] When there are many clients, the server may not have enough storage resources to store the data sets sent by the clients, or the server may not have enough computing resources to perform clustering operations on the data sets sent by the clients, resulting in the inability to perform grouping work in a timely manner.

[0074] To solve the above problems, see Figure 1An embodiment of the present application provides a federated learning system 100, which includes: a management node 101, multiple access nodes 102 communicating with the management node 101, and a client 103 connected to each access node 102. For each access node 102, the number of clients 103 connected to the access node 102 is multiple.

[0075] Each client 103 in the federated learning system 100 locally includes training data. For each client 103, the client 103 performs model training based on the local training data of the client 103 to obtain a first AI model, and sends a data set to the access node 102 to which the client 103 accesses. The data set is used to describe the first AI model.

[0076] In some embodiments, the client 103 can receive the system global model sent by the access node 102, and train the system global model based on the local training data of the client 103 to obtain a first AI model.

[0077] In some embodiments, the data set sent by the client 103 to the access node 102 may be a first data set or a second data set. The first data set includes a plurality of first data for describing the first AI model, and the second data set is a data set after sparse processing is performed on the first data set, and the data volume of the second data set is less than the data volume of the first data set.

[0078] For each access node 103 in the multiple access nodes 102 , for ease of description, the access node 102 is referred to as a first access node 102 , and the number of clients 103 accessing the first access node 102 is n, where n is an integer greater than 1.

[0079] The first access node 102 receives the data sets sent by the n clients 102, that is, the first access node 102 receives a total of n data sets, performs clustering operations on the n data sets to obtain multiple first clusters, and sends information about the multiple first clusters to the management node 101. Each of the other access nodes 102, like the first access node 102, also sends information about the multiple first clusters to the management node 101.

[0080] The management node 101 receives information of multiple first clusters sent by some or all of the multiple access nodes 102, and receives information of a first clusters in total, where a is an integer greater than 1. Based on the information of a first clusters, multiple global cluster centers are obtained, and the multiple global cluster centers are used to indicate multiple groups. The multiple global cluster centers are sent to some or all of the multiple access nodes 102.

[0081] The first access node 102 receives the multiple global cluster centers, and determines the group to which each of the n clients 103 belongs based on part or all of the multiple global cluster centers. Like the first access node 102, the other access nodes 102 also determine the group to which each of the multiple clients 103 accessing the respective clients 103 belongs based on part or all of the multiple global cluster centers.

[0082] Since multiple clients 103 in the federated learning system 100 are connected to multiple access nodes 102, for the first access node 102 and the n clients connected to the first access node 102, the first access node 102 performs clustering operations on the n data sets sent by the n clients to obtain multiple first clusters. In this way, multiple access nodes 102 can perform clustering operations in parallel, and the number of data sets for each access node 102 to perform clustering operations is greatly reduced, and the amount of data that needs to be calculated for clustering operations of each access node 102 is greatly reduced, so that each access node 102 can quickly and in real time with relatively few computing resources to obtain information of multiple first clusters and send the information of multiple first clusters to the management node 101. Since the amount of data of each first cluster information is small, each access node 102 can quickly send the information of multiple first clusters to the management node 101, and the management node 101 can quickly receive the information of multiple first clusters sent by each access node 102.

[0083] The data volume of the information of the multiple first clusters sent by the first access node 102 is much smaller than the data volume of the n data sets, so the amount of calculation required for the management node 101 to obtain the multiple global cluster centers is greatly reduced, and the multiple global cluster centers can be quickly obtained based on the information of the first cluster sent by each access node 102. The first access node 102 determines the group to which each of the n clients 103 belongs based on the multiple global cluster centers, thereby greatly improving the efficiency of grouping.

[0084] After determining the grouping of each client 103, the management node 103 is also used to obtain the grouping model corresponding to the first group based on the first data set of each client 103 in the first group, and the multiple groups include the first group, and send the grouping model corresponding to the first group to each client 103 in the first group through each access node 102.

[0085] In some embodiments, the management node 101 and each access node 102 may be different devices, such as different servers.

[0086] For example, see Figure 2The management node 101 and each access node 102 are different devices. For each client 103, the client 103 includes a local data set 1031, a local model library 1032, a local training module 1033, a sparsification module 1034 and a first communication model 1035, etc.

[0087] The local data set 1031 is used to store the training data and other contents in the client 103, and the local model library 1032 is used to store the AI ​​model obtained by the client 103. For example, the local model library 1032 is used to store the first AI model, the system global model, and the group model corresponding to the group to which the client 103 belongs. The system global model and the group model will be described in detail in the subsequent content and will not be introduced here.

[0088] The local training model 1033 is used to perform model training based on the training data. The sparse model 1034 is used to perform sparse processing on the data set to be sent to the access node 102 to reduce the amount of data in the data set, and send the sparsely processed data set to the access node 102 through the first communication module 1035. The detailed implementation process of the sparse processing will be described in detail in the subsequent content and will not be introduced here.

[0089] For each access node 102 , the access node 102 includes a client parameter data set 1021 , a dimension reduction processing module 1022 , a client management module 1023 , a model delivery module 1024 and a second communication module 1025 .

[0090] The model sending module 1024 is used to send the system global model or the group model corresponding to the group to which the client 103 belongs to to the client 103 through the second communication module 1025 .

[0091] The client parameter data set 1021 is used to store data sets sent by multiple clients 103 accessing the access node 102, that is, to store multiple data sets. Optionally, the client parameter data set 1021 can also be used to store information such as identification information of each client 103, the state of each client 103, and / or the group identification of the group to which each client 103 belongs.

[0092] The dimension reduction processing module 1022 is used to reduce the dimension of the multiple data sets by clustering, perform clustering operations on the multiple data sets to obtain multiple first clusters, and send information of the multiple first clusters to the management node 101 through the second communication module 1025.

[0093] The client management module 1023 is used to manage the multiple clients 103 .

[0094] The management node 101 includes a feature extraction module 1011 , a client clustering module 1012 , a model aggregation module 1013 , a model management module 1014 , a global model library 1015 and a third communication module 1016 .

[0095] The feature extraction module 1011 is used to receive information of multiple first clusters sent by each access node 102 through the third communication module 1016, and extract or construct feature information for client cluster grouping based on the received information of each first cluster.

[0096] The client clustering module 1012 is used to obtain multiple global clustering centers based on the extracted or constructed feature information.

[0097] The model aggregation module 1013 is used to generate a group model corresponding to each group according to the local model parameters uploaded by each client and with reference to the group to which the client belongs.

[0098] The model management module 104 is used to manage the life cycle of the system global model and each group model, and control the update and aging processes of the system global model.

[0099] The global model library 1015 is used to store the system global model and the group model corresponding to each group. Optionally, the system global model and the group model corresponding to each group can be deployed in the access node 102 in the form of a copy or a mirror.

[0100] In some embodiments, the management node 101 and each access node 102 may be different virtualized instances in a device or a cluster. For example, the management node 101 and each access node 102 may be different virtual machines in a server or a cluster. For another example, the management node 101 and each access node 102 may be different containers in a server or a cluster. For another example, the management node 101 and each access node 102 may be different microservices, processes or threads in a server or a cluster.

[0101] For example, see Figure 3 The management node 101 and each access node 102 are different virtualized instances in a device or a cluster, and the management node 101 and each access node 102 use a common second communication module to communicate with each client 103 .

[0102] In some embodiments, the client 103 may be a physical device such as a terminal device or an edge server, or may be an application deployed on the physical device.

[0103] See also Figure 4 The present application embodiment provides a grouping method 400, the method 400 is applied to Figure 1 , Figure 2 or Figure 3 In the federated learning system 100 shown, in the method 400, the client performs model training to obtain a first AI model, obtains a first data set for describing the first AI model, performs sparse processing on the first data set to obtain a second data set, and sends the second data set.

[0104] For each client accessing the access node, the access node obtains multiple first clusters based on the second data set of each client, and sends the information of the multiple first clusters to the management node. The management node obtains multiple global cluster centers based on the information of the multiple first clusters sent by each access node, and the multiple global cluster centers correspond to multiple groups one by one, and sends the multiple global cluster centers to each access node. The access node determines the group to which the client belongs based on the multiple global cluster centers. The method 400 includes the following process.

[0105] Step 401: The first client performs model training based on local training data to obtain a first AI model. The first client is any client in the federated learning system.

[0106] In some embodiments, the first client obtains the second AI model, performs model training on the second AI model based on local training data, and obtains the first AI model.

[0107] In some embodiments, the second AI model may be a system global model, and each client in the federated learning system may include the same second AI model. The first client locally includes an AI model framework, and the second AI model is obtained based on the AI ​​model framework.

[0108] The following two methods for obtaining the second AI model are listed below.

[0109] Method 1: The management node initializes multiple network parameters of the global model of the first system, the first client obtains multiple network parameters of the global model of the first system, adjusts the local AI model framework based on the multiple network parameters of the global model of the first system, and obtains a second AI model, which may be the same or similar to the global model of the first system. Then, the first client performs model training on the second AI model based on local training data to obtain the first AI model.

[0110] In some embodiments, the management node randomly initializes multiple network parameters of the first system global model, or uses historically saved model parameters to initialize multiple network parameters of the first system global model.

[0111] In some embodiments, after the management node initializes multiple network parameters of the first system global model, the management node may send multiple network parameters of the first system global model to each access node. For each access node, the access node receives multiple network parameters of the first system global model and sends multiple network parameters of the first system global model to each client accessing the access node. Each client accessing the access node receives multiple network parameters of the first system global model.

[0112] In some embodiments, after the management node initializes multiple network parameters of the global model of the first system, it obtains a storage path of a storage space, the storage space is used to store multiple network parameters of the global model of the first system, and sends the storage path to each access node. For each access node, the access node receives the storage path and sends the storage path to each client connected to the access node. Each client connected to the access node receives the storage path and downloads the multiple network parameters of the global model of the first system from the management node based on the storage path. Optionally, the storage path can be a uniform resource locator (URL), etc.

[0113] Method 2: The first client obtains the second AI model through a general federated learning method.

[0114] In mode 2, the general federated learning method includes the following operations of steps 1011-1016.

[0115] 1011: The first client obtains the first system global model by using the above method 1, trains the first system global model based on local training data, and obtains the target AI model.

[0116] 1012: The first client sends a first target data set to the management node, where the first target data set is used to describe the target AI model.

[0117] In some embodiments, the first target data set includes multiple network parameters of the target AI model. Alternatively, the first target data set includes multiple network parameter update values ​​of the target AI model, and the multiple network parameter update values ​​are the differences between the multiple network parameters of the target AI model and the multiple network parameters of the first system global model.

[0118] Optionally, the first target data set may be a matrix, vector or array, etc., the elements of which are multiple network parameters of the target AI model, or the elements of which are multiple network parameter update values ​​of the target AI model.

[0119] In 1012, the first client accesses the first access node, and the first client sends a first target data set to the first access node. There are multiple clients accessing the first access node, and the first access node not only receives the first target data set sent by the first client, but also receives the first target data set sent by clients other than the first client among the multiple clients. In other words, the first access node receives the first target data set sent by multiple clients, and forwards the first target data set sent by the multiple clients to the management node. There are multiple access nodes communicating with the management node, and the access nodes other than the first access node among the multiple access nodes also forward the first target data set sent by the client to the management node.

[0120] 1013: The management node receives a first target data set sent by each client in the federated learning system, and acquires a second system global model based on the first target data set sent by each client.

[0121] In some embodiments, the first target data set includes multiple network parameters of the target AI model, and a weighted average operation is performed based on the weight corresponding to each client and the first target data set sent by each client to obtain multiple network parameters of the second system global model.

[0122] In some embodiments, the first target data set includes multiple network parameter update values ​​of the target AI model, and a weighted average operation is performed based on the weight corresponding to each client and the first target data set sent by each client to obtain multiple network parameter update values ​​of the second system global model.

[0123] The above-mentioned method of using weighted average operation is only one implementation method of obtaining multiple network parameters of the global model of the second system. Of course, in actual implementation, other methods can also be used to obtain multiple network parameters of the global model of the second system. For example, other methods may be a federated normalized averaging algorithm (FedNova), a federated optimization of heterogeneous networks (FedProx) algorithm, or a stochastic controlled averaging algorithm (SCAFFOLD).

[0124] 1014: The management node sends a second target data set to each client, where the second target data set is used to describe the second system global model.

[0125] The second target data set includes a plurality of network parameters of the second system global model, or a plurality of network parameter update values ​​of the second system global model.

[0126] In 1014, the management node sends the second target data set to the multiple access nodes. For each access node, the access node sends the second target data set to multiple clients accessing the access node.

[0127] 1015: The first client receives the second target data set, and adjusts the local first system global model based on the second target data set to obtain the second system global model.

[0128] In some embodiments, the second target data set includes a plurality of network parameters of the second system global model, and the local first system global model is adjusted based on the plurality of network parameters of the second system global model to obtain the second system global model.

[0129] In some embodiments, the second target data set includes multiple network parameter update values ​​of the second system global model, and the multiple network parameters of the second system global model are obtained based on the multiple network parameter update values ​​of the second system global model and the multiple network parameters of the local first system global model. The local first system global model is adjusted based on the multiple network parameters of the second system global model to obtain the second system global model.

[0130] 1016: The first client trains the global model of the second system based on the local training data, obtains the target AI model, and returns to execute 1012.

[0131] In some embodiments, when the loop is executed a specified number of times, the first client uses the trained system global model as the second AI model; or, when the trained system global model converges, the first client uses the converged system global model as the second AI model.

[0132] The so-called convergence means that the system global model is tested using at least one test data to obtain the test accuracy of the system global model, and when the test accuracy exceeds the accuracy threshold, it is determined that the system global model converges. Alternatively, for multiple system global models obtained through multiple consecutive trainings, if the difference between the test accuracy of the multiple system global models is less than the difference threshold, any one of the multiple system global models converges.

[0133] In some embodiments, the first client is any one of Q clients selected by the management node from the federated learning system, where Q is an integer greater than 1. That is, each of the Q clients respectively executes the method 400 provided in the embodiment of the present application starting from the above step 401.

[0134] In some embodiments, the management node randomly selects Q clients from the federated learning system.

[0135] In some embodiments, the management node selects Q clients according to the time sequence of accessing the federated learning system. For example, the Q clients that first access the federated learning system may be selected, or the Q clients that last access the federated learning system may be selected.

[0136] In some embodiments, the management node selects Q clients from the federated learning system according to the selection rule. For example, Q clients whose recent local model test accuracy is lower than the accuracy threshold are selected. For another example, Q clients are selected according to at least one of the client's region, country code, address, user profile, model and / or version information. For another example, Q clients are selected according to part or all of the parameter information of the local model, and / or the feature map information of the local model.

[0137] In some embodiments, after selecting Q clients, the management node and the access node will perform system initialization operations and inform the selected Q clients to perform initialization operations. The initialization operations include one or more of the following operations: registering a device, handshaking a communication, applying for resources, or resetting a status.

[0138] Step 402: The first client performs sparse processing on the first data set to obtain a second data set, where the first data set is used to describe the first AI model, and the amount of data in the second data set is smaller than that in the first data set.

[0139] The first data set may be a matrix, a vector or an array, etc. The second data set obtained by sparsifying the first data set may include a number of data that is less than or equal to the number of data included in the first data set, or the dimension of the second data set may be less than or equal to the dimension of the first data set.

[0140] The first data set includes a plurality of first data for describing the first AI model. Optionally, the plurality of first data are a plurality of network parameters of the first AI model, or the plurality of first data are a plurality of network parameter update values ​​of the first AI model, and the plurality of network parameter update values ​​of the first AI model are the difference between the plurality of network parameters of the first AI model and the plurality of network parameters of the second AI model.

[0141] In step 402, the following two thinning processing methods are listed.

[0142] Method 1: select m first data satisfying a first condition from a first data set, where m is an integer greater than or equal to 1; and obtain a second data set based on the m first data.

[0143] In some embodiments, the m first data include the m first data with the largest absolute value in the first data set. Or,

[0144] The m first data include m first data in the first data set that are within a first range. Optionally, the first range may be a range greater than a first threshold, or the first range may be a range greater than the first threshold or less than a second threshold, and the second threshold is less than the first threshold.

[0145] For example, assuming that the first threshold is 0.3 and the second threshold is -0.2, the first range is a range greater than 0.3, that is, the first data greater than 0.3 is selected from the first data set. Alternatively, the first range is a range greater than 0.3 or less than -0.2, that is, the first data greater than 0.3 and the first data less than -0.2 are selected from the first data set.

[0146] Regarding the second data set obtained by method one, several examples of the second data set are listed below.

[0147] Example 1: The second data set includes the m first data and the sequence numbers of the m first data in the first data set.

[0148] The m first data are data with significant characteristics in the first data set. The distribution characteristics and / or trends of the first data set can be expressed by the m first data and the serial numbers of the m first data. Therefore, the first data set can be replaced by the second data set, and the second data set can be used for grouping in subsequent grouping operations.

[0149] For the second data set in Example 1, since the second data set includes m first data in the first data set, the number of first data included in the second data set is smaller than the number of first data included in the first data set, so that the data amount of the second data set is smaller than the data amount of the first data set.

[0150] For example, assuming that the first data set includes 10,000 first data, 500 first data are selected from the first data set, that is, m=500, and the second data set includes the 500 first data and the sequence numbers of the 500 first data in the first data set. In this way, the number of first data included in the second data set is much smaller than the number of first data included in the first data set, so that the data volume of the second data set is smaller than the data volume of the first data set.

[0151] Example 2: The second data set includes m first bits corresponding to the m first data and second bits corresponding to data other than the m first data in the first data set.

[0152] Assume that the number of first data included in the first data set is x, where x is an integer greater than m, the data in the first data set other than the m first data include xm first data, and the second data set includes m first bits corresponding to the m first data and xm second bits corresponding to the xm first data.

[0153] The first bit may be bit 1 and the second bit may be bit 0; or, the first bit may be bit 0 and the second bit may be bit 1.

[0154] For the second data set in Example 2, the number of bits included in the second data set is equal to the number of first data included in the first data set, that is, the second data set includes x bits. However, since the second data set includes x bits and the first data set includes x first data, the amount of data in the second data set is smaller than the amount of data in the first data set.

[0155] Example 3, the second data set includes m second data corresponding to the m first data and the serial number of each second data, the serial number of each second data is the serial number of the first data corresponding to each second data in the first data set, and the bit width of each first data is greater than the bit width of the second data corresponding to each first data.

[0156] In Example 3, the m first data are quantized to obtain m second data, and the m first data correspond to the m second data one by one, so that the bit width of each first data is larger than the bit width of the second data corresponding to each first data.

[0157] For example, the m first data are m characters, and it is assumed that the bit width of each of the m first data is 32 bits, that is, each first data is 32 bits in length. The m first data are quantized to obtain m integer data with a length of 8 bits. That is, the second data set includes the m integer data, so that the data volume of the second data set is less than the data volume of the first data set.

[0158] In some embodiments, part of the m first data may also be quantized to obtain second data corresponding to the part of the first data, so the second data set includes the second data corresponding to the part of the first data and the data among the m first data except the part of the first data.

[0159] In a second method, the type of the first data set is a matrix or a vector. A fifth data set is generated, and the dimension of the fifth data set is smaller than the dimension of the first data set. The fifth data set and the first data set are operated at least once to obtain a second data set.

[0160] Optionally, a fifth data set is randomly generated, the fifth data set is a matrix or a vector, each data included in the fifth data set is less than 1 and greater than -1, each data in the fifth data set is a decimal or 0, etc. The fifth data set is multiplied by the first data set at least once to obtain a second data set.

[0161] Optionally, the operation of performing at least one multiplication operation on the fifth data set and the first data set may be: performing multiplication operation on the fifth data set and the first data set to obtain a first intermediate set; then, performing multiplication operation on the fifth data set and the first intermediate set to obtain a second intermediate set; then, performing multiplication operation on the fifth data set and the second intermediate set to obtain a third intermediate set. Repeat the above process until the intermediate set obtained satisfies the specified condition, and the intermediate set satisfying the specified condition is used as the second data set.

[0162] Since each data included in the fifth data set is less than 1 and greater than -1, the data included in the intermediate set obtained by each operation gradually becomes smaller. For the intermediate set obtained by each operation, when the ratio of the number of data included in the intermediate set that is less than the first threshold to the total number of data included in the intermediate set is lower than the ratio threshold, it is determined that the intermediate set meets the specified condition.

[0163] The dimension of the second data set is equal to the dimension of the fifth data set, and the dimension of the second data set is smaller than the dimension of the first data set, so that the data volume of the second data set is smaller than the data volume of the first data set.

[0164] In some embodiments, the first AI model includes a multi-layer network, and the first data set includes the first data of the multi-layer network. When the first data set is subjected to sparse processing, the first data of the a-layer network can be selected from the first data set, where a is an integer greater than or equal to 1. The first data of the a-layer network in the first data set is subjected to sparse processing to obtain a second data set. The second data set may include the data after the sparse processing of the first data of the a-layer network and the first data of the network other than the a-layer network in the first data set. Optionally, the a-layer network may be the last few layers of the network or a specific network of the first AI model.

[0165] For example, the first AI model is a deep neural network (DNN), and the specific network is the last fully connected layer network of the DNN, or part or all of the batch normalization (BN) layer network of the DNN.

[0166] In some embodiments, a sparse matrix representation method may be used to store the second data set after sparse processing. The sparse matrix representation method includes one or more of the following methods: coordinate matrix (COO), compressed sparse row matrix (CSR), compressed sparse column matrix (CSC), block sparse row matrix (BSR), linked list matrix (LIL), diagonal matrix (DIA).

[0167] Step 403: The first client sends a second data set to a first access node, where the first access node is an access node accessed by the first client.

[0168] Since the data volume of the second data set is smaller than the data volume of the first data set, the network resources occupied by sending the second data set can be reduced.

[0169] The number of clients connected to the first access node is n, where n is an integer greater than 1. For the other n-1 clients connected to the first access node, the n-1 clients, like the first client, execute the above steps 401-403 to send the second data set to the first access node.

[0170] Step 404: The first access node receives n second data sets, where the n second data sets correspond to the n clients.

[0171] The n second data sets are second data sets sent by each of the n clients.

[0172] Step 405: The first access node performs a clustering operation on the n second data sets to obtain a plurality of first clusters.

[0173] In step 405, the first access node performs clustering operations on the n second data sets by a clustering algorithm to obtain multiple first clusters. Optionally, the clustering algorithm may include a combination of one or more of the following algorithms: k-means clustering algorithm (k-Means), optimized k-means clustering algorithm (k-Means++), bisection k-means clustering algorithm (bi-kmeans), density-based spatial clustering of applications with noise (DBSCAN), ordering points to identify the clustering structure (OPTICS), hierarchical clustering algorithm or Coreset algorithm, etc.

[0174] For each first cluster, the first cluster includes one or more data sets, which may be part of the n second data sets, or the one or more data sets may be data sets obtained based on the n second data sets. There is no intersection between any two first clusters.

[0175] In some embodiments, the first access node further obtains a weight corresponding to each first cluster. Optionally, the weight corresponding to the first cluster is obtained based on the number of data sets included in the first cluster. For example, the weight corresponding to the first cluster is equal to the number of data sets included in the first cluster, or the weight corresponding to the first cluster is the product of the number of data sets included in the first cluster and a specified coefficient.

[0176] In some embodiments, the first access node directly performs a clustering operation on the n second data sets to obtain a plurality of first clusters.

[0177] For each second data set, the second data set includes m first data and the sequence numbers of the m first data, and the first access node performs a clustering operation on the n second data sets based on the m first data and the sequence numbers of the m first data included in each second data set to obtain multiple first clusters, each of which includes a portion of the second data sets in the n second data sets. Or,

[0178] The second data set includes m second data and sequence numbers of the m second data, and the first access node performs a clustering operation on the n second data sets based on the m second data and sequence numbers of each second data set to obtain multiple first clusters, each of which includes a portion of the second data sets in the n second data sets. Or,

[0179] The second data set includes x bits, and the first access node performs a clustering operation on the n second data sets based on the x bits included in each second data set to obtain multiple first clusters, each of which includes a portion of the second data sets in the n second data sets. Or,

[0180] The second data set is a data set obtained by adopting the above-mentioned method 2, and the dimension of the second data set is smaller than the dimension of the first data set. The first access node performs a clustering operation on the n second data sets based on the data included in each second data set to obtain multiple first clusters, each of which includes part of the second data sets in the n second data sets.

[0181] In some embodiments, for each of the n second data sets, the second data set includes m first data and the sequence numbers of the m first data sets, and the first access node obtains n intermediate data sets corresponding to the n second data sets based on xm default data, the m first data included in each of the n second data sets, and the sequence numbers of the m first data. For each second data set, the intermediate data set corresponding to the second data set includes the m first data in the second data set and the xm default data. A clustering operation is performed on the n intermediate data sets to obtain multiple first clusters, each of which includes part of the intermediate data sets in the n intermediate data sets. Optionally, the default data can be 0, 1, or 2, etc.

[0182] For each intermediate data set, the intermediate data set includes x data, that is, the number of data included in the intermediate data set is equal to the number of first data included in the first data set, and both are equal to x.

[0183] For any second data set, the process of obtaining the intermediate data set corresponding to the second data set can be: based on the sequence numbers of the m first data included in the second data set, fill the m first data into the intermediate data set, and then fill xm default data into the intermediate data set so that the intermediate data set includes x data.

[0184] Step 406: The first access node sends information of multiple first clusters to the management node.

[0185] In step 406, the first access node may also send weights corresponding to the multiple first clusters to the management node.

[0186] For any first cluster among the plurality of first clusters, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set and the center of the first cluster satisfy the second similarity condition. Therefore, the total number of data sets included in the information of the plurality of first clusters is less than n.

[0187] In some embodiments, the second similarity condition refers to selecting at least one data set from the first cluster whose similarity with the center of the first cluster is the greatest, or selecting at least one data set from the first cluster whose similarity with the center of the first cluster exceeds a second similarity threshold.

[0188] The center of the first cluster may be a certain data set included in the first cluster, or the center of the first cluster may be a new data set obtained based on the data sets included in the first cluster.

[0189] For each first cluster, the first access node calculates the similarity between each data set included in the first cluster and the center of the first cluster, and selects at least one data set with the greatest similarity to the center of the first cluster as the information of the first cluster.

[0190] For each data set, the similarity between the data set and the center of the first cluster may include the distance between the data set and the center of the first cluster, or the cosine similarity between the data set and the center of the first cluster, etc.

[0191] For multiple access nodes communicating with the management node, each access node except the first access node in the multiple access nodes sends information of multiple first clusters to the management node by executing the process of steps 404 to 406 above.

[0192] Step 407: The management node receives information of a first clusters, where the information of the a first clusters comes from part or all of the multiple access nodes, and obtains multiple global cluster centers based on the information of the a first clusters, where the multiple global cluster centers are used to indicate multiple groups.

[0193] For each first cluster information, the first cluster information includes at least one data set in the first cluster. In step 407, the management node performs clustering operations on the data sets included in the information of a first clusters to obtain multiple global clusters, obtains the centers of the multiple global clusters, and uses the centers of the multiple global clusters as multiple global cluster centers.

[0194] In some embodiments, the management node further receives weights corresponding to multiple first clusters sent by each access node. Based on the weights corresponding to the a first clusters, a clustering operation is performed on a data set included in the information of the a first clusters to obtain multiple global clusters.

[0195] In some embodiments, the management node performs clustering operations on the data set included in each received first cluster information through a clustering algorithm to obtain multiple global clusters.

[0196] Since the data set included in the information of each first cluster is a partial data set in each first cluster, the total number of data sets included in the received information of each first cluster is less than Q. In this way, the total data volume of the data sets included in the received information of each first cluster is less than the total data volume of the first data sets of Q clients, thereby reducing the computational amount of the management node for clustering operations and improving the efficiency of obtaining the multiple global cluster centers.

[0197] Step 408: The management node sends the multiple global cluster centers to part or all of the multiple access nodes.

[0198] In some embodiments, the management node may send the multiple global cluster centers to some access nodes, and may send some of the multiple global cluster centers to some access nodes. For example, for the first access node, the first access node sends information of c first clusters, c is an integer greater than or equal to 1, and the multiple global cluster centers include c global cluster centers corresponding to the c first clusters, and the c global cluster centers can be sent to the first access node.

[0199] For each access node, after receiving the multiple global cluster centers, each access node determines the group to which the client connected to each access node belongs through the following step 409. The following step 409 is described using the first access node as an example. By replacing the first access node with other access nodes, the implementation process of other access nodes can be obtained.

[0200] Step 409: The first access node receives a plurality of global cluster centers, and determines a group to which each of the n clients belongs based on a part or all of the plurality of global cluster centers.

[0201] In some embodiments, in step 409, the n data sets are clustered based on the multiple global cluster centers to obtain multiple second clusters, the multiple global cluster centers correspond to the multiple second clusters, and for each second cluster, the second cluster includes a portion of the n data sets, and the global cluster center corresponding to the second cluster satisfies the first similarity condition. The client corresponding to each data set included in the second cluster is determined as a group.

[0202] In some embodiments, the first access node sends information of c first clusters, and the first access node may receive c global cluster centers, or may receive more than c global cluster centers. In step 409, the n data sets are clustered based on the c global cluster centers to obtain c second clusters, and the c global cluster centers correspond to the c second clusters. For each second cluster, the second cluster includes a partial data set in the n data sets, and the partial data set and the global cluster center corresponding to the second cluster meet the first similarity condition. The client corresponding to each data set included in the second cluster is determined as a group.

[0203] For each of the n data sets, based on the data set and each global cluster center, the similarity between the data set and each global cluster center is obtained, and the global cluster center with the greatest similarity to the data set is selected, or a global cluster center with a similarity to the data set exceeding a first similarity threshold is selected, and the selected global cluster center and the data set meet the first similarity condition, and the data set is added to the second cluster corresponding to the selected global cluster center. In this way, the n data sets are clustered into multiple second clusters.

[0204] Step 410: The first access node sends to each client a group identifier of the group to which each client belongs.

[0205] Each client includes a first client, and each client performs the following operation of step 411 like the first client.

[0206] Step 411: the first client receives a group identifier of a group to which the first client belongs, and sends a first training request to a first access node, where the first training request includes a first data set and a group identifier of a first group to which the first client belongs.

[0207] For the other n-1 clients accessing the first access node, the n-1 clients, like the first client, also send a first training request to the first access node.

[0208] Step 412: The first access node receives the first training request sent by each client, and acquires a third data set corresponding to the first group based on the first data set and group identifier included in each received first training request.

[0209] In step 412, the first access node receives the first training request sent by each client, and obtains the first data set of each client and the group identifier of the group to which each client belongs from the first training request sent by each client. Based on the group identifier of the first group, the first data set of each client in the first group is obtained, and a weighted average is performed based on the weight of each client and the first data set of each client to obtain a third data set corresponding to the first group, and the third data set includes multiple network parameters or multiple network parameter update values. The multiple network parameters included in the third data set are obtained by weighted averaging the multiple network parameters included in the first data set of each client in the first group, or the multiple network parameters included in the third data set are obtained by weighted averaging the multiple network parameter update values ​​included in the first data set of each client in the first group. According to the above process, the first group is one of the multiple groups, and the first access node can obtain the third data set corresponding to each other group.

[0210] The above-mentioned weighted average method is only an implementation example of obtaining the third data set. Of course, other methods can also be used to obtain the first data set, which will not be listed here one by one.

[0211] Step 413: The first access node sends a second training request to the management node, where the second training request includes a third data set corresponding to the first group.

[0212] The data volume of the third data set corresponding to the first group is smaller than the total data volume of the first data set of each client in the first group, so the first access node sends the third data set corresponding to the first group to the management node, which can reduce the occupation of network resources.

[0213] In some embodiments, the second training request further includes a group identifier of the first group, and / or a third data set corresponding to each other group and a group identifier of each other group.

[0214] For other access nodes communicating with the management node, the other access nodes, like the first access node, send the third data set corresponding to each group and the group identifier of each group to the management node.

[0215] Step 414: the management node receives the second training request sent by each access node, and acquires the group model corresponding to the first group based on the third data set corresponding to the first group included in each received second training request.

[0216] In step 414, the management node obtains the third data set corresponding to the first group from each received second training request based on the group identifier of the first group, thereby obtaining multiple third data sets corresponding to multiple first groups. The multiple third data sets are averaged to obtain a fourth data set corresponding to the first group, and the fourth data set includes multiple network parameters or multiple network parameter update values. The multiple network parameters in the fourth data set are obtained by averaging the multiple network parameters in the multiple third data sets, or the multiple network parameter update values ​​in the fourth data set are obtained by averaging the multiple network parameter update values ​​in the multiple third data sets. The fourth data set is used to describe the group model corresponding to the first group. According to the above process, the management node can obtain the fourth data set of the group model corresponding to each other group.

[0217] In some embodiments, averaging the multiple third data sets is only one way to obtain the fourth data set corresponding to the first group. Of course, other methods can also be used to process the multiple third data sets to obtain the fourth data set corresponding to the first group, which will not be listed one by one here.

[0218] Step 415: The management node sends a fourth data set to each access node, where the fourth data set is used to describe the group model corresponding to the first group.

[0219] The management node may also obtain fourth data sets corresponding to other groups, and send the fourth data sets corresponding to other groups to each access node.

[0220] Step 416: The first access node receives the fourth data set, and sends the fourth data set to the clients belonging to the first group among the n clients.

[0221] The first client belongs to the first group, and the first client receives the fourth data set. When the first client receives the fourth data set, the following operations are performed.

[0222] Step 417: The first client receives the fourth data set, and adjusts the first AI model based on the fourth data set.

[0223] The adjusted first AI model is the same as or similar to the group model corresponding to the first group.

[0224] In some embodiments, after the first access node receives the first data set of each client in the first group, it can send the first data set of each client in the first group to the management node. The management node receives the first data set of each client in the first group sent by each access node, thereby obtaining the first data set sent by each client in the first group in the federated learning system, and based on the first data set of each client in the first group and the weight of each client in the first group, weighted average is performed to obtain the group model corresponding to the first group, and the management node sends a fourth data set to each access node, and the fourth data set is used to describe the group model corresponding to the first group.

[0225] In step 417, after adjusting the first AI model based on the fourth data set, the first client trains the adjusted first AI model based on the local training data to obtain a new AI model, uses the new AI model as the first AI model, and performs a second round of training of the group model corresponding to each group starting from the above step 411. After the second round of training of the group model corresponding to each group is completed, the third round of training of the group model corresponding to each group is performed starting from the above step 411. After multiple rounds of training, a group model corresponding to each group is obtained, and any client includes the group model corresponding to the group to which the client belongs.

[0226] That is, see Figure 5 , first initialize the first system global model, then group the Q clients through the above 401-410, and after grouping, perform the i-th round of training of the group model corresponding to each group through the above steps 411-417, i=1, 2, 3, ..., and end when i reaches the threshold T. That is, after grouping the Q clients once, perform T-1 rounds of training the group model.

[0227] In some embodiments, see Figure 6 When executing the i-th round of training the group model corresponding to each group, the access node receives the first data set sent by the client, calculates the similarity between the first data set sent by the client and the global cluster center corresponding to the group to which the client belongs, and when the similarity is lower than the similarity threshold, regroups the Q clients through the above steps 401-410, and after grouping, executes the i+1-th round of training the group model corresponding to each group through the above steps 411-417.

[0228] In some embodiments, see Figure 7, first initialize the first system global model, then perform the i-th grouping of multiple clients through the above 401-410, and after the i-th grouping, train the grouping model corresponding to each group through the above steps 411-417, and end when i reaches the threshold T. That is, the above steps 401-417 are executed T-1 times in total.

[0229] For a second client other than the Q clients in the federated learning system, when it is necessary to train the AI ​​model in the second client, the second AI model is sent to the second client. After the second client receives the second AI model, the group to which the second client belongs is determined according to the above steps 401-410, and a data set describing the group model corresponding to the group that has been recently trained is sent to the second client. The second client receives the data set, adjusts the local AI model based on the data set, and then continues to loop through the above steps 411-417.

[0230] In an embodiment of the present application, after the first client obtains the first AI model based on the local training data training, the first data set used to describe the first AI model is sparsely processed to obtain a second data set with a data volume less than that of the first data set, and the second data set is sent to the first access node. The first access node receives n second data sets sent by n clients, performs clustering operations on the n second data sets, and obtains information of multiple first clusters. Since the data volume of the second data set is reduced, the storage resources required by the first access node to store the second data set are also reduced. The other access nodes in the federated learning system are the same as the first access node, so that each access node performs clustering operations on multiple second data sets sent by multiple clients connected to itself in parallel, and the number of second data sets that each access node needs to cluster is greatly reduced, and the data volume of the second data sets that need to be clustered is reduced, thereby greatly improving the efficiency of clustering operations. Each access node sends the information of multiple first clusters obtained by clustering to the management node, and the information of the first cluster includes at least one data set with the greatest similarity to the center of the first cluster. The number of data sets included in the information of the first cluster is less than the number of data sets included in the first cluster, which reduces the number of data sets that the management node needs to perform clustering operations on, and improves the efficiency of obtaining multiple global cluster centers. Therefore, each access node can quickly obtain the multiple global cluster centers, and determine the group to which each client accessing itself belongs based on the multiple global cluster centers, thereby improving the efficiency of grouping.

[0231] See also Figure 8 , the present application embodiment provides a grouping method 800, the method 800 is applied to Figure 1In the federated learning system 100 shown, in the method 800, the client performs model training to obtain a first AI model, obtains a first data set for describing the first AI model, performs sparse processing on the first data set to obtain a second data set, and sends the second data set. For each client connected to the access node, the access node obtains multiple first clusters based on the second data set of each client, and sends the centers of the multiple first clusters to the management node. The management node obtains multiple global cluster centers based on the centers of the multiple first clusters sent by each access node, and the multiple global cluster centers correspond one-to-one to multiple groups, and sends the multiple global cluster centers to each access node. The access point determines the group to which the client belongs based on the multiple global information. The method 800 includes the following process.

[0232] Steps 801-804: respectively Figure 2 Steps 401 - 404 in the illustrated method 400 are the same and will not be described in detail herein.

[0233] Step 805: The first access node receives a plurality of global cluster centers sent by the management node, and performs clustering operations on the n second data sets based on the plurality of global cluster centers to obtain a plurality of first clusters.

[0234] The management node may first initialize multiple global cluster centers, and send the multiple global cluster centers to multiple access nodes communicating with the management node, where the multiple access nodes include the first access node.

[0235] In step 805, the first access node receives a plurality of global cluster centers sent by the management node, and based on the plurality of global cluster centers, performs clustering operations on the n second data sets by using a clustering algorithm to obtain a plurality of first clusters.

[0236] For each first cluster, the first cluster includes one or more data sets, which may be part of the n second data sets, or the one or more data sets may be data sets obtained based on the n second data sets.

[0237] In some embodiments, the first access node further obtains a weight corresponding to each first cluster. Optionally, the weight corresponding to the first cluster is obtained based on the number of data sets included in the first cluster. For example, the weight corresponding to the first cluster is equal to the number of data sets included in the first cluster, or the weight corresponding to the first cluster is the product of the number of data sets included in the first cluster and a specified coefficient.

[0238] In some embodiments, the first access node directly performs clustering operations on the n second data sets based on the multiple global cluster centers to obtain multiple first clusters.

[0239] For each second data set, the second data set includes m first data and the sequence numbers of the m first data, the first access node performs clustering operation on the n second data sets based on the multiple global cluster centers, the m first data included in each second data set and the sequence numbers of the m first data, to obtain multiple first clusters, each of which includes part of the second data sets in the n second data sets. Or,

[0240] The second data set includes m second data and sequence numbers of the m second data, and the first access node performs clustering operation on the n second data sets based on the multiple global cluster centers, the m second data included in each second data set and the sequence numbers of the m second data to obtain multiple first clusters, each of which includes part of the second data sets in the n second data sets. Or,

[0241] The second data set includes x bits, and the first access node performs clustering operation on the n second data sets based on the multiple global cluster centers and the x bits included in each second data set to obtain multiple first clusters, each of which includes part of the second data sets in the n second data sets. Or,

[0242] The second data set is a data set obtained by adopting the above-mentioned method 2, and the dimension of the second data set is smaller than the dimension of the first data set. The first access node performs a clustering operation on the n second data sets based on the multiple global cluster centers and the data included in each second data set to obtain multiple first clusters, each of which includes part of the second data sets in the n second data sets.

[0243] In some embodiments, for each of the n second data sets, the second data set includes m first data and the sequence numbers of the m first data sets, and the first access node obtains n intermediate data sets corresponding to the n second data sets based on xm default data, the m first data included in each of the n second data sets, and the sequence numbers of the m first data. For each second data set, the intermediate data set corresponding to the second data set includes the m first data in the second data set and the xm default data. Based on the multiple global clustering centers, clustering operations are performed on the n intermediate data sets to obtain multiple first clusters, each of which includes part of the intermediate data sets in the n intermediate data sets. Optionally, the default data can be 0, 1, or 2, etc.

[0244] Step 806: The first access node sends the centers of multiple first clusters to the management node.

[0245] In some embodiments, the first access node further sends weights corresponding to the plurality of first clusters to the management node.

[0246] For multiple access nodes communicating with the management node, each access node except the first access node in the multiple access nodes sends the centers of multiple first clusters to the management node by executing the process of steps 804 to 806 above.

[0247] Step 807: The management node receives a centers of the first clusters, where the a centers of the first clusters come from part or all of the multiple access nodes, and acquires multiple global cluster centers based on the a centers of the first clusters.

[0248] In step 807, for the center of the i-th first cluster sent by each access node, i=1, 2, 3, ..., each access node is an access node that sends the center of the first cluster. The management node obtains the i-th global cluster center based on the weight corresponding to the i-th first cluster sent by each access node and the center of the i-th first cluster sent by each access node. Optionally, in implementation:

[0249] The management node performs weighted averaging, linear transformation and / or kernel function transformation based on the weight of the i-th first cluster sent by each access node and the center of the i-th first cluster sent by each access node to obtain the i-th global cluster center.

[0250] Since the data volume of the center of each first cluster is small, the total data volume of each first cluster center received is smaller than the total data volume of the Q data sets, thereby improving the efficiency of obtaining the multiple global cluster centers.

[0251] Step 808: The management node sends the multiple global cluster centers to part or all of the multiple access nodes.

[0252] For each access node, after receiving the multiple first global centers, each access node determines whether the multiple first cluster centers meet the convergence condition through the following step 809.

[0253] Step 809: The first access node receives the multiple global cluster centers and determines whether the multiple global cluster centers meet the convergence condition. If the multiple global cluster centers do not meet the convergence condition, return to execute step 805; if the multiple global cluster centers meet the convergence condition, return to execute step 810.

[0254] In some embodiments, the convergence condition may mean that for the multiple global cluster centers obtained last time, the difference between the multiple global cluster centers obtained currently and the multiple global cluster centers obtained last time is less than a difference threshold.

[0255] Steps 810-818: respectively Figure 2 Steps 409-417 in the illustrated method 400 are the same and will not be described in detail herein.

[0256] In an embodiment of the present application, after the first client obtains the first AI model based on the local training data training, the first data set used to describe the first AI model is sparsely processed to obtain a second data set with a data volume less than that of the first data set, and the second data set is sent to the first access node. The first access node receives n second data sets sent by n clients and multiple global clustering centers sent by the receiving management node, and performs clustering operations on the n second data sets based on the multiple global clustering centers to obtain multiple first clustering centers. Since the data volume of the second data set is reduced, the storage resources required by the first access node to store the second data set are also reduced. The other access nodes in the federated learning system are the same as the first access node, so that each access node performs clustering operations on multiple second data sets sent by multiple clients connected to itself in parallel, and the number of second data sets that each access node needs to cluster is greatly reduced, and the data volume of the second data sets that need to be clustered is reduced, thereby greatly improving the efficiency of clustering operations. Each access node sends the centers of multiple first clusters obtained by clustering to the management node. The data volume of the centers of the first clusters is very small, which reduces the amount of data that the management node needs to perform clustering operations on, and improves the efficiency of obtaining multiple global cluster centers. Thus, each access node can quickly obtain the multiple global cluster centers, and determine the group to which each client accessing itself belongs based on the multiple global cluster centers, thereby improving the efficiency of grouping.

[0257] See also Fig. 9 , the present application embodiment provides a grouping method 900, the method 900 is applied to Figure 1 In the federated learning system 100 shown, in the method 900, the client performs model training to obtain a first AI model, obtains a first data set for describing the first AI model, and sends the first data set. For each client connected to the access node, the access node obtains multiple first clusters based on the first data set of each client, and sends information of the multiple first clusters to the management node. The management node obtains multiple global cluster centers based on the information of the multiple first clusters sent by each access node, and the multiple global cluster centers correspond one-to-one to multiple groups, and sends the multiple global cluster centers to each access node. The access point determines the group to which the client belongs based on the multiple global cluster centers. The method 900 includes the following process.

[0258] Step 901: Figure 2 Step 401 of the method 400 is the same and will not be described in detail here.

[0259] Step 902: The first client sends a first data set to a first access node, where the first data set is used to describe a first AI model. The first access node is an access node to which the first client accesses.

[0260] The number of clients accessing the first access node is n, where n is an integer greater than 1. For the other n-1 clients accessing the first access node, the n-1 clients, like the first client, execute the above steps 901-902 to send the first data set to the first access node.

[0261] Step 903: The first access node receives n first data sets, where the n first data sets correspond to the n clients.

[0262] The n first data sets are first data sets sent by each of the n clients.

[0263] Step 904: The first access node performs a clustering operation on the n first data sets to obtain a plurality of first clusters.

[0264] In step 904, the first access node performs clustering operations on the n first data sets by using a clustering algorithm to obtain a plurality of first clusters.

[0265] In step 904, the first access node performs a sparse process on the n first data sets to obtain n second data sets, where the n first data sets correspond one to one with the n second data sets. The n second data sets are clustered using a clustering algorithm to obtain multiple first clusters.

[0266] For each first data set, the first access node performs a sparse processing on the first data set to obtain a second data set corresponding to the first data set. The detailed implementation process of the sparse processing performed by the first access node is shown in Figure 2 The relevant contents of step 402 of the method 900 are not described in detail here.

[0267] Steps 905-907: respectively Figure 2 Steps 406-408 of the method shown are the same and will not be described in detail here.

[0268] Step 908: The first access node receives the multiple global cluster centers, and determines the group to which each of the n clients belongs based on the multiple global cluster centers.

[0269] In step 908, the n data sets are clustered based on the multiple global cluster centers to obtain multiple second clusters, the n data sets are the above-mentioned n first data sets, or the n data sets are the above-mentioned n second data sets, and the multiple global cluster centers correspond to the multiple second clusters. For each second cluster, the second cluster includes a portion of the data sets in the n data sets, and the global cluster center corresponding to the partial data set and the second cluster satisfies the first similarity condition. The client corresponding to each data set included in the second cluster is determined as a group.

[0270] For each data set in the n data sets, based on the data set and each global cluster center, the similarity between the data set and each global cluster center is obtained, and the global cluster center with the greatest similarity to the data set is selected. The selected global cluster center and the data set meet the first similarity condition, and the data set is added to the second cluster corresponding to the selected global cluster center. In this way, the n data sets are clustered into multiple second clusters.

[0271] Step 909: The first access node obtains a third data set corresponding to the first group based on the first data set of each client in the first group, where the first group is any one of the multiple groups.

[0272] In step 909, when the first access node determines the group to which each of the n clients belongs, for any group, for the sake of convenience, the group is referred to as the first group, and the first data set of each client belonging to the first group is obtained. Based on the weight of each client and the first data set of each client, a weighted average is performed to obtain a third data set corresponding to the first group, and the third data set includes multiple network parameters or multiple network parameter update values. According to the above process, the first access node can obtain the third data set corresponding to each other group.

[0273] The weighted average method is only one implementation method for obtaining the third data set corresponding to the first group. Of course, other implementation methods can also be used to obtain the third data set corresponding to the first group, which will not be listed here one by one.

[0274] Steps 910-914: respectively Figure 2 Steps 413 - 417 of the illustrated method 400 are the same and will not be described in detail herein.

[0275] In an embodiment of the present application, after the first client obtains the first AI model based on the local training data training, it sends the first data set used to describe the first AI model to the first access node. The first access node receives n first data sets sent by n clients, performs clustering operations on the n first data sets, and obtains information of multiple first clusters. The other access nodes in the federated learning system are the same as the first access node, so that each access node performs clustering operations on multiple first data sets sent by multiple clients connected to itself in parallel, and the number of first data sets that each access node needs to cluster is greatly reduced, thereby greatly improving the efficiency of clustering operations. Each access node sends the information of multiple first clusters obtained by clustering to the management node, which reduces the amount of data that the management node needs to cluster and improves the efficiency of obtaining multiple global cluster centers. Thus, each access node can quickly obtain the multiple global cluster centers, and determine the group to which each client connected to itself belongs based on the multiple global cluster centers, thereby improving the efficiency of grouping.

[0276] See also Fig.10 The present application embodiment provides a grouping device 1000, the device 1000 is applied to a first access node, the first access node is Figure 1 , Figure 2 or Figure 3 The federated learning system shown includes any one of the multiple access nodes, the federated learning system also includes a management node and n clients accessed to the first access node, n is an integer greater than 1, and the device 1000 includes: a receiving unit 1001, a sending unit 1002 and a processing unit 1003.

[0277] The receiving unit 1001 is used to receive n data sets from n clients;

[0278] The processing unit 1003 is used to cluster the n data sets to obtain information of a plurality of first clusters;

[0279] A sending unit 1002 is configured to send information of multiple first clusters to a management node;

[0280] The receiving unit 1001 is further used to receive multiple first global cluster centers sent by the management node, where the multiple first global cluster centers are obtained by the management node based on information of a first clusters, where the information of the a first clusters comes from part or all of the multiple access nodes, where a is an integer greater than 1, and the multiple first global cluster centers are used to indicate multiple groups;

[0281] The processing unit 1003 is configured to determine the group to which each client belongs based on part or all of the multiple first global cluster centers.

[0282] Optionally, the detailed implementation process of the receiving unit 1001 receiving n data sets is shown in Figure 4 Step 404 of the method 400 is shown. Figure 8 Step 804 of method 800 or Fig. 9 The relevant contents of step 903 of the method 900 are not described in detail here.

[0283] Optionally, the detailed implementation process of the sending unit 1002 sending the information of the plurality of first clusters can be found in Figure 4 Step 406 of the method 400 shown in FIG. Figure 8 Step 806 of method 800 or Fig. 9 The relevant contents of step 905 of the method 900 are not described in detail here.

[0284] Optionally, the detailed implementation process of the receiving unit 1001 receiving multiple first global clustering centers can be found in Figure 4 Step 409 of the method 400 shown in FIG. Figure 8 Step 809 of method 800 or Fig. 9 The relevant contents of step 909 of the method 900 are not described in detail here.

[0285] Optionally, the detailed implementation process of the processing unit 1003 determining the group to which each client belongs based on the multiple first global cluster centers is shown in Figure 4 Step 409 of the method 400 shown in FIG. Figure 8 Step 809 of method 800 or Fig. 9 The relevant contents of step 909 of the method 900 are not described in detail here.

[0286] Optionally, the processing unit 1003 is configured to:

[0287] Clustering the n data sets based on the multiple first global cluster centers to obtain multiple second clusters, the multiple first global cluster centers correspond to the multiple second clusters, and for each second cluster, the second cluster includes a partial data set in the n data sets, and the partial data set and the first global cluster center corresponding to the second cluster meet a first similarity condition;

[0288] The clients corresponding to each data set included in the second cluster are determined as a group.

[0289] Optionally, the processing unit 1003 clusters the n data sets based on the multiple first global cluster centers to obtain multiple second clusters, and determines the client corresponding to each data set included in the second cluster as a group. For a detailed implementation process, see Figure 4 Step 409 of the method 400 shown in FIG. Figure 8 Step 809 of method 800 or Fig. 9 The relevant contents of step 909 of the method 900 are not described in detail here.

[0290] Optionally, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set and the center of the first cluster satisfy a second similarity condition.

[0291] Optionally, the information of the first cluster includes the center of the first cluster, and the multiple first global cluster centers do not meet the convergence condition;

[0292] The processing unit 1003 is further configured to cluster the n data sets based on the multiple first global clustering centers to obtain multiple third clusters;

[0293] The sending unit 1002 is further configured to send information of the plurality of third clusters to the management node;

[0294] The receiving unit 1001 is further configured to receive a plurality of second global cluster centers sent by the management node, where the plurality of second global cluster centers are used to indicate the plurality of groups;

[0295] The processing unit 1003 is further configured to obtain the group to which each client belongs based on the multiple second global clustering centers when the multiple second global clustering centers meet the convergence condition.

[0296] Optionally, the n data sets are n second data sets from n clients. For each client, the second data set corresponding to the client is obtained by the client performing sparse processing on the first data set. The first data set is used to describe the first AI model obtained by the client performing model training. The amount of data in the second data set is smaller than that in the first data set.

[0297] Optionally, the first data set includes multiple first data for describing a first AI model in the client, the second data set includes m first data and serial numbers of the m first data in the first data set, m is an integer greater than 1, and the m first data are the first data in the first data set that satisfy the first condition. The processing unit 1003 is also used to perform clustering operations based on the m first data and the serial numbers of the m first data included in each of the n second data sets to obtain information of multiple first clusters.

[0298] Optionally, the n data sets are n first data sets from the n clients, and for each client, the first data set corresponding to the client is used to describe a first AI model in the client, and the processing unit 1003 is further used to:

[0299] Performing sparse processing on n first data sets to obtain n second data sets, the n first data sets correspond to the n second data sets, and for each first data set, the data amount of the first data set is less than the data amount of the second data set corresponding to the first data set;

[0300] A clustering operation is performed on the n second data sets to obtain information of the multiple first clusters.

[0301] Optionally, the receiving unit 1001 is further configured to receive multiple first data sets sent by multiple clients in a first group, the n clients include multiple clients, the multiple first data sets are used to describe a first AI model in the multiple clients, and the multiple groups include the first group;

[0302] The sending unit 1002 is further configured to send a third data set to the management node, where the third data set is obtained based on the multiple first data sets;

[0303] The receiving unit 1001 is further configured to receive a fourth data set sent by the management node, where the fourth data set is used to describe a grouping model corresponding to the first grouping;

[0304] The sending unit 1002 is further configured to send a fourth data set to each client in the first group.

[0305] Optionally, the detailed implementation process of the sending unit 1002 sending the third data set is shown in Figure 4 Step 413 of the method 400 shown in FIG. Figure 8 Step 814 of method 800 or Fig. 9 The relevant contents of step 910 of the method 900 are not described in detail here.

[0306] Optionally, the detailed implementation process of the receiving unit 1001 receiving the fourth data set is shown in Figure 4 Step 413 of the method 400 shown in FIG. Figure 8 Step 816 of method 800 or Fig. 9 The relevant contents of step 912 of the method 900 are not described in detail here.

[0307] Optionally, the detailed implementation process of the sending unit 1002 sending the fourth data set is shown in Figure 4 Step 414 of the method 400 shown in FIG. Figure 8 Step 817 of method 800 or Fig. 9 The relevant contents of step 913 of the method 900 are not described in detail here.

[0308] In an embodiment of the present application, since the federated learning system includes multiple access nodes, the receiving unit of each access node receives the data set sent by the respective accessed client, and the processing unit of each access node performs clustering operations on the received data set in parallel to obtain multiple first cluster information. The sending unit sends multiple first cluster information to the management node. The amount of data of the first cluster information is small, and the management node quickly obtains multiple first global cluster centers based on the first cluster information sent by some or all of the multiple access nodes. The processing unit determines the group to which each client belongs based on the multiple first global cluster centers, thereby improving the grouping efficiency.

[0309] See also Fig.11 The present application embodiment provides a grouping device 1100, the device 1100 is applied to Figure 1 , Figure 2 or Figure 3 The federated learning system shown includes a management node, the federated learning system also includes multiple access nodes and multiple clients accessing each access node, the multiple access nodes include a first access node, and the device 1100 includes:

[0310] A receiving unit 1101 is configured to receive a first cluster of information, where the a first cluster of information comes from part or all of the multiple access nodes, a is an integer greater than 1, and the multiple first clusters of information sent by the first access node are obtained by clustering n data sets, n is an integer greater than 1, and the n data sets come from n clients accessing the first access node;

[0311] The processing unit 1102 is used to obtain a plurality of first global cluster centers based on information of a first clusters, where the plurality of first global cluster centers are used to indicate a plurality of groups of clients;

[0312] The sending unit 1103 is used to send multiple first global cluster centers to some or all of the multiple access nodes, and the first access node is used to determine the group to which each of the n clients belongs based on some or all of the multiple first global cluster centers.

[0313] Optionally, the detailed implementation process of the receiving unit 1101 receiving the information of the plurality of first clusters is shown in Figure 4 Step 407 of the method 400 shown in FIG. Figure 8 Step 807 of method 800 or Fig. 9 The relevant contents of step 906 of the method 900 are not described in detail here.

[0314] Optionally, the detailed implementation process of the processing unit 1102 obtaining multiple first global clustering centers can be found in Figure 4 Step 407 of the method 400 shown in FIG. Figure 8 Step 807 of method 800 or Fig. 9 The relevant contents of step 906 of the method 900 are not described in detail here.

[0315] Optionally, the detailed implementation process of the sending unit 1103 sending multiple first global clustering centers to each access node can be found in Figure 4 Step 408 of the method 400 shown in FIG. Figure 8 Step 808 of method 800 or Fig. 9 The relevant contents of step 907 of the method 900 are not described in detail here.

[0316] Optionally, for each received first cluster, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set and the center of the first cluster satisfy a second similarity condition;

[0317] The processing unit 1102 is configured to perform a clustering operation on a data set including information of a first clusters to obtain a plurality of first global clusters, wherein the plurality of first global cluster centers include a plurality of centers of the first global clusters.

[0318] Optionally, the detailed implementation process of the processing unit 1102 obtaining multiple first global clustering centers can be found in Figure 4 Step 407 of method 400 or Fig. 9 The relevant contents of step 906 of the method 900 are not described in detail here.

[0319] Optionally, for each received first cluster, the information of the first cluster includes a center of the first cluster;

[0320] The processing unit 1102 is used to obtain multiple first global cluster centers based on the weights corresponding to the a first clusters and the centers of the a first clusters.

[0321] Optionally, the detailed implementation process of the processing unit 1102 obtaining multiple first global clustering centers can be found in Figure 8 The relevant contents of step 807 of the method 800 are not described in detail here.

[0322] Optionally, the plurality of first global cluster centers do not satisfy the convergence condition,

[0323] The receiving unit 1101 is further configured to receive information of b third clusters, where the information of the b third clusters comes from part or all of the multiple access nodes, b is an integer greater than 1, and the third cluster sent by the first access node is obtained by clustering the n data sets by the first access node based on multiple first global clustering centers;

[0324] The processing unit 1102 is further configured to obtain a plurality of second global cluster centers based on information of the b third clusters;

[0325] The sending unit 1103 is further configured to send multiple second global clustering centers to some or all of the multiple access nodes, and the first access node is configured to determine the group to which each of the n clients belongs based on the multiple second global clustering centers.

[0326] Optionally, the receiving unit 1101 is further configured to receive multiple third data sets, where the multiple third data sets come from part or all of the multiple access nodes, and the third data set sent by the first access node is used to describe a first AI module in multiple clients belonging to a first group, where the n clients include the multiple clients, and the multiple groups include the first group;

[0327] The processing unit 1102 is further configured to obtain a grouping model corresponding to the first grouping based on the plurality of third data sets;

[0328] The sending unit 1103 is further configured to send a fourth data set to part or all of the multiple access nodes, where the fourth data set is used to describe the grouping model.

[0329] Optionally, the detailed implementation process of the receiving unit 1101 receiving the third data set sent by each access node is shown in Figure 4 Step 414 of the method 400 shown in FIG. Figure 8 Step 815 of method 800 or Fig. 9 The relevant contents of step 911 of the method 900 are not described in detail here.

[0330] Optionally, the detailed implementation process of the processing unit 1102 acquiring the group model corresponding to the first group based on the third data set sent by each access node can be found in Figure 4 Step 414 of the method 400 shown in FIG. Figure 8 Step 815 of method 800 or Fig. 9 The relevant contents of step 911 of the method 900 are not described in detail here.

[0331] Optionally, the detailed implementation process of the sending unit 1103 sending the fourth data set to each access node is shown in Figure 4 Step 415 of the method 400 shown in FIG. Figure 8 Step 816 of method 800 or Fig. 9 The relevant contents of step 912 of the method 900 are not described in detail here.

[0332] In an embodiment of the present application, since the federated learning system includes multiple access nodes, each access node receives a data set sent by a respective accessed client, and each access node performs clustering operations on the received data set in parallel to obtain information of multiple first clusters. The receiving unit receives information of multiple first clusters sent by each access node. The amount of information is small, and the processing unit quickly obtains multiple first global cluster centers based on the information of the first cluster sent by each access node. The sending unit sends the multiple first global cluster centers to each access node, and the first access node determines the group to which each client belongs based on the multiple first global cluster centers, thereby improving the grouping efficiency.

[0333] See also Fig.12 The present application embodiment provides a grouping device 1200, wherein the device 1200 is applied Figure 1 , Figure 2 or Figure 3 In the client of the system shown, the device 1200 stores training data, and the device 1200 includes:

[0334] The processing unit 1201 is used to perform model training based on the training data to obtain a first artificial intelligence AI model;

[0335] The processing unit 1201 is further configured to perform a sparse processing on the first data set to obtain a second data set, where the first data set is used to describe the first AI model, and the amount of data in the second data set is less than the amount of data in the first data set;

[0336] The sending unit 1202 is used to send a second data set, where the second data set is used to request to determine the group to which the device 1200 belongs.

[0337] Optionally, the processing unit 1202 performs model training based on the training data to obtain the first artificial intelligence AI model. Figure 4 Step 401 of the method 400 or Figure 8 The relevant contents of step 801 of the method 800 are not described in detail here.

[0338] Optionally, the processing unit 1202 performs sparse processing on the first data set to obtain the second data set. Figure 4 Step 402 of method 400 or Figure 8 The relevant contents of step 802 of the method 800 are not described in detail here.

[0339] Optionally, the detailed implementation process of the sending unit 1203 sending the second data set is shown in Figure 4 Step 403 of method 400 or Figure 8The relevant contents of step 803 of the method 800 are not described in detail here.

[0340] Optionally, the first data set includes a plurality of first data for describing the first AI model, and the processing unit 1201 is configured to:

[0341] Selecting m first data satisfying a first condition from the first data set, where m is an integer greater than or equal to 1;

[0342] A second data set is obtained based on the m first data.

[0343] Optionally, the detailed implementation process of the processing unit 1202 selecting m first data satisfying the first condition from the first data set is shown in Figure 4 Step 402 of method 400 or Figure 8 The relevant contents of step 802 of the method 800 are not described in detail here.

[0344] Optionally, the detailed implementation process of the processing unit 1202 obtaining the second data set based on the m first data can be found in Figure 4 Step 402 of method 400 or Figure 8 The relevant contents of step 802 of the method 800 are not described in detail here.

[0345] Optionally, the first condition includes that the m first data are the m data with the largest absolute values ​​in the first data set, or the m first data are the m data with numerical values ​​in the first data set falling within a first range.

[0346] Optionally, the second data set includes m first data and m sequence numbers of the first data in the first data set; or,

[0347] The second data set includes m first bits and xm second bits, x is the number of first data included in the first data set, x is an integer greater than m, the m first bits correspond to the m first data, the xm second bits correspond to the xm first data, and the xm first data are the data in the first data set except the m first data.

[0348] Optionally, the processing unit 1201 is configured to:

[0349] The m first data are quantized to obtain m second data, the m first data correspond to the m second data, the bit width of each first data is respectively greater than the bit width of the second data corresponding to each first data, and the second data set includes the m second data and the serial number of the first data corresponding to each second data in the first data set.

[0350] Optionally, the detailed implementation process of the processing unit 1202 performing quantization processing on the m first data to obtain the m second data can be found in Figure 4 Step 402 of method 400 or Figure 8 The relevant contents of step 802 of the method 800 are not described in detail here.

[0351] Optionally, the type of the first data set is a matrix or a vector, and the processing unit 1201 is configured to:

[0352] Acquire a fifth data set, where the dimension of the fifth data set is smaller than the dimension of the first data set;

[0353] At least one operation is performed on the fifth data set and the first data set to obtain a second data set.

[0354] Optionally, the detailed implementation process of the processing unit 1202 obtaining the fifth data set is shown in Figure 4 Step 402 of method 400 or Figure 8 The relevant contents of step 802 of the method 800 are not described in detail here.

[0355] Optionally, the processing unit 1202 performs at least one operation on the fifth data set and the first data set to obtain the second data set. Figure 4 Step 402 of method 400 or Figure 8 The relevant contents of step 802 of the method 800 are not described in detail here.

[0356] Optionally, the first data set is obtained based on network parameters of the second AI model and / or network parameters of the first AI model, and the first AI model is obtained by training the second AI model based on training data.

[0357] Optionally, the device 1200 further includes: a receiving unit 1203;

[0358] The sending unit 1202 is further configured to send a first data set;

[0359] A receiving unit 1203 is configured to receive a fourth data set, where the fourth data set is used to describe a group model corresponding to the group, where the group model is obtained based on a first data set sent by each client in the group;

[0360] The processing unit 1201 is further used to adjust the network parameters of the first AI model based on the fourth data set.

[0361] Optionally, the detailed implementation process of the sending unit 1203 sending the first data set is shown in Figure 4 Step 411 of method 400 or Figure 8The relevant contents of step 812 of the method 800 are not described in detail here.

[0362] Optionally, the detailed implementation process of the receiving unit 1203 receiving the fourth data set is shown in Figure 4 Step 411 of method 400 or Figure 8 The relevant contents of step 817 of the method 800 are not described in detail here.

[0363] Optionally, the detailed implementation process of the processing unit 1202 adjusting the network parameters of the first AI model based on the fourth data set is shown in Figure 4 Step 417 of method 400 or Figure 8 The relevant contents of step 818 of the method 800 are not described in detail here.

[0364] In an embodiment of the present application, since the processing unit performs sparse processing on the first data set to obtain the second data set, the data volume of the second data set is smaller than the data volume of the first data set. In this way, when the second data set of each client is used for grouping, the data volume of the grouping operation is reduced, thereby improving the grouping efficiency.

[0365] See also Fig.13 , an embodiment of the present application provides a schematic diagram of a device 1300. The device 1300 may be an access node provided by any of the above embodiments, and the device 1300 includes at least one processor 1301, an internal connection 1302, a memory 1303 and at least one transceiver 1304.

[0366] The device 1300 is a hardware structure device that can be used to implement Fig.10 The functional modules in the device 1000 are as follows. For example, those skilled in the art may think of Fig.10 The processing unit 1003 in the device 1000 shown can be implemented by the at least one processor 1301 calling the code in the memory 1303. Fig.10 The receiving unit 10201 and the sending unit 1002 in the device 1000 shown may be implemented by the at least one transceiver 1304 .

[0367] The device 1300 may also be used to implement the function of the access node in any of the above embodiments.

[0368] The processor 1301 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0369] The internal connection 1302 may include a path to transmit information between the components. The internal connection 1302 may be a single board or a bus.

[0370] The at least one transceiver 1304 is used to communicate with other devices or communication networks.

[0371] The memory 1303 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0372] The memory 1303 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 1301. The processor 1301 is used to execute the application code stored in the memory 1303, and cooperate with at least one transceiver 1304, so that the device 1300 implements the functions of the method of the present patent.

[0373] In a specific implementation, as an embodiment, the processor 1301 may include one or more CPUs, such as Fig.13 CPU0 and CPU1 in.

[0374] In a specific implementation, as an embodiment, the device 1300 may include multiple processors, such as Fig.13 1301 and processor 1307 in the embodiment of the present invention. Each of these processors may be a single-CPU processor or a multi-CPU processor. The processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0375] See also Fig.14 , an embodiment of the present application provides a schematic diagram of a device 1400. The device 1400 may be a management node provided by any of the above embodiments. The device 1400 includes at least one processor 1401, an internal connection 1402, a memory 1403 and at least one transceiver 1404.

[0376] The device 1400 is a hardware structure device that can be used to implement Fig.11 The functional modules in the device 1100 are as follows. For example, those skilled in the art may think of Fig.11 The processing unit 1102 in the device 1100 shown can be implemented by the at least one processor 1401 calling the code in the memory 1403. Fig.11 The receiving unit 1101 and the sending unit 1103 in the illustrated device 1100 may be implemented by the at least one transceiver 1404 .

[0377] The device 1400 may also be used to implement the function of the management node in any of the above embodiments.

[0378] The processor 1401 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0379] The internal connection 1402 may include a path to transmit information between the components. The internal connection 1402 may be a single board or a bus.

[0380] The at least one transceiver 1404 is used to communicate with other devices or communication networks.

[0381] The memory 1403 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0382] The memory 1403 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 1401. The processor 1401 is used to execute the application code stored in the memory 1403, and cooperate with at least one transceiver 1404, so that the device 1400 implements the functions of the method of the present patent.

[0383] In a specific implementation, as an embodiment, the processor 1401 may include one or more CPUs, such as Fig.14 CPU0 and CPU1 in.

[0384] In a specific implementation, as an embodiment, the device 1400 may include multiple processors, such as Fig.14 1401 and 1407 in the embodiment of the present invention. Each of these processors may be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0385] See also Fig.15 , an embodiment of the present application provides a schematic diagram of a device 1500. The device 1500 may be a client provided by any of the above embodiments. The device 1500 includes at least one processor 1501, an internal connection 1502, a memory 1503 and at least one transceiver 1504.

[0386] The device 1500 is a hardware structure device that can be used to implement Fig.12 The functional modules in the device 1200 are as follows. For example, those skilled in the art may think of Fig.12 The processing unit 1201 in the device 1200 shown can be implemented by the at least one processor 1501 calling the code in the memory 1503. Fig.12 The sending unit 1202 and the receiving unit 1203 in the device 1200 shown may be implemented by the at least one transceiver 1504 .

[0387] The device 1500 can also be used to implement the functions of the client in any of the above embodiments.

[0388] The processor 1501 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0389] The internal connection 1502 may include a path to transmit information between the components. The internal connection 1502 may be a single board or a bus.

[0390] The at least one transceiver 1504 is used to communicate with other devices or communication networks.

[0391] The memory 1503 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0392] The memory 1503 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 1501. The processor 1501 is used to execute the application code stored in the memory 1503, and cooperate with at least one transceiver 1504, so that the device 1500 implements the functions of the method of the present patent.

[0393] In a specific implementation, as an embodiment, the processor 1501 may include one or more CPUs, such as Fig.15 CPU0 and CPU1 in.

[0394] In a specific implementation, as an embodiment, the device 1500 may include multiple processors, such as Fig.15 1501 and processor 1507 in the embodiment of the present invention. Each of these processors may be a single-CPU processor or a multi-CPU processor. The processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0395] See also Fig.16 The embodiment of the present application provides a federated learning system 1600, the federated learning system 1600 includes a management node 1601, multiple access nodes and multiple clients connected to each access node, the multiple access nodes include a first access node 1602, the first client 1603 is one of n clients connected to the first access node 1602, n is an integer greater than 1,

[0396] The first client 1603 is used to send a data set to the first access node 1602, where the data set is used to describe a first artificial intelligence AI model in the first client 1603;

[0397] The first access node 1602 is used to cluster data sets from n clients to obtain information of multiple first clusters, and to send the information of multiple first clusters to the management node 1601;

[0398] The management node 1601 is used to obtain multiple first global cluster centers based on information of a first clusters, where the information of a first clusters comes from part or all of the multiple access nodes, a is an integer greater than 1, and the multiple first global cluster centers are used to indicate multiple groups of clients;

[0399] The management node 1601 is further configured to send multiple first global clustering centers to some or all of the multiple access nodes;

[0400] The first access node 1602 is further configured to obtain a group to which each of the n clients belongs based on part or all of the plurality of first global clustering centers.

[0401] Optionally, the n data sets are first data sets from n clients;

[0402] The first access node 1602 is used to perform sparse processing on the n first data sets to obtain n second data sets, the n first data sets correspond to the n second data sets, for each first data set, the data amount of the first data set is smaller than the data amount of the second data set corresponding to the first data set, and clustering operations are performed on the n second data sets to obtain information of multiple first clusters.

[0403] Optionally, the n data sets are second data sets from the n clients;

[0404] The first client is used to perform sparse processing on the first data set to obtain a second data set, the first data set is used to describe the first AI model, the data volume of the second data set is smaller than the data volume of the first data set, and the second data set is sent to the first access node 1602.

[0405] Optionally, the first data set includes multiple first data for describing the first AI model, and the second data set includes m first data and sequences of the m first data in the first data set, where m is an integer greater than 1, and the m first data are the first data in the first data set that satisfy the first condition;

[0406] The first access node 1602 is used to perform a clustering operation based on the m first data and the sequence numbers of the m first data included in each of the n second data sets to obtain information of multiple first clusters.

[0407] Optionally, the first access node 1602 is used to cluster the n data sets based on the multiple first global cluster centers to obtain multiple second clusters, the multiple first global cluster centers correspond to the multiple second clusters, and for each second cluster, the second cluster includes a partial data set in the n data sets, and the partial data set and the first global cluster center corresponding to the second cluster meet the first similarity condition, and the client corresponding to each data set included in the second cluster is determined as a group.

[0408] Optionally, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies a second similarity condition with the center of the first cluster. The management node 1601 is used to perform clustering operations on the data sets included in the information of a first clusters to obtain multiple first global clusters, and the multiple first global cluster centers include multiple centers of first global clusters.

[0409] Optionally, the information of the first cluster includes the center of the first cluster, and the management node 1601 is used to obtain multiple first global cluster centers based on the weights corresponding to the a first clusters and the centers of the a first clusters.

[0410] Optionally, the plurality of first global cluster centers do not satisfy a convergence condition,

[0411] The first access node 1602 is used to cluster the n data sets based on the multiple first global clustering centers to obtain multiple third clusters;

[0412] The first access node 1602 is further configured to send information of multiple third clusters to the management node 1601;

[0413] The management node 1601 is further configured to obtain multiple second global cluster centers based on information of b third clusters, where the information of the b third clusters comes from some or all of the multiple access nodes, and b is an integer greater than 1, and send the multiple second global cluster centers to some or all of the multiple access nodes;

[0414] The first access node 1602 is further configured to obtain the group to which each client belongs based on the multiple second global clustering centers when the multiple second global clustering centers meet a convergence condition.

[0415] Optionally, the n clients include a plurality of second clients belonging to the first group, and the plurality of groups include the first group;

[0416] The first access point 1602 is further configured to receive multiple first data sets, where the multiple first data sets are data sets corresponding to multiple second clients, and the multiple first data sets are used to describe the first AI models in the multiple second clients, and to send a third data set to the management node 1601, where the third data set is obtained based on the multiple first data sets;

[0417] The management node 1601 is further configured to obtain a group model corresponding to the first group based on multiple third data sets, where the multiple third data sets come from some or all of the multiple access nodes, and send a fourth data set to some or all of the multiple access nodes, where the fourth data set is used to describe the group model corresponding to the first group;

[0418] The first access node 1602 is further configured to send a fourth data set to multiple second clients.

[0419] In an embodiment of the present application, since the federated learning system includes multiple access nodes, each access node receives a data set sent by a respective accessed client, and each access node performs clustering operations on the received data set in parallel to obtain information of multiple first clusters. The first access node sends information of multiple first clusters to the management node. The amount of data in the information of the first cluster is small, and the management node quickly obtains multiple first global cluster centers based on the received information of the first cluster. The first access node determines the group to which each client belongs based on the multiple first global cluster centers, thereby improving the grouping efficiency.

[0420] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0421] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A federated learning system, characterized in that: The federated learning system includes a management node, multiple access nodes, and multiple clients connected to each access node, wherein the multiple access nodes include a first access node, the first client is one of n clients connected to the first access node, and n is an integer greater than 1. The first client is used to send a data set to the first access node, where the data set is used to describe a first artificial intelligence AI model in the first client; The first access node is used to cluster the data sets from the n clients to obtain information of a plurality of first clusters, and is used to send the information of the plurality of first clusters to the management node; The management node is used to obtain a plurality of first global cluster centers based on information of a first clusters, where the information of the a first clusters comes from part or all of the plurality of access nodes, a is an integer greater than 1, and the plurality of first global cluster centers are used to indicate a plurality of groups of clients; The management node is further configured to send the plurality of first global clustering centers to some or all of the plurality of access nodes; The first access node is further configured to obtain a group to which each of the n clients belongs based on part or all of the plurality of first global clustering centers.

2. The system according to claim 1, characterized in that The n data sets are first data sets from the n clients; The first access node is used to perform sparse processing on the n first data sets to obtain n second data sets, the n first data sets correspond to the n second data sets, for each first data set, the data volume of the first data set is smaller than the data volume of the second data set corresponding to the first data set, and clustering operations are performed on the n second data sets to obtain information of the multiple first clusters.

3. The system according to claim 1, characterized in that The n data sets are second data sets from the n clients; The first client is used to perform sparse processing on the first data set to obtain a second data set, the first data set is used to describe the first AI model, the data volume of the second data set is smaller than the data volume of the first data set, and the second data set is sent to the first access node.

4. The system according to claim 3, characterized in that The first data set includes multiple first data for describing the first AI model, the second data set includes m first data and sequence numbers of the m first data in the first data set, m is an integer greater than 1, and the m first data are the first data in the first data set that meet the first condition; The first access node is used to perform a clustering operation based on the m first data and the sequence numbers of the m first data included in each of the n second data sets to obtain information of the multiple first clusters.

5. The system according to any one of claims 1 to 4, characterized in that: The first access node is used to cluster the n data sets based on the multiple first global cluster centers to obtain multiple second clusters, the multiple first global cluster centers correspond to the multiple second clusters, for each second cluster, the second cluster includes a partial data set in the n data sets, the partial data set and the first global cluster center corresponding to the second cluster meet a first similarity condition, and the client corresponding to each data set included in the second cluster is determined as a group.

6. The system according to any one of claims 1 to 5, characterized in that: The information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies a second similarity condition with the center of the first cluster. The management node is used to perform clustering operations on the data sets included in the information of the a first clusters to obtain multiple first global clusters, and the multiple first global cluster centers include the centers of the multiple first global clusters.

7. The system according to any one of claims 1 to 4, characterized in that: The information of the first cluster includes the center of the first cluster, and the management node is used to obtain the multiple first global cluster centers based on the weights corresponding to the a first clusters and the centers of the a first clusters.

8. The system according to claim 7, characterized in that The plurality of first global cluster centers do not satisfy the convergence condition, The first access node is used to cluster the n data sets based on the multiple first global clustering centers to obtain multiple third clusters; The first access node is further configured to send information of the plurality of third clusters to the management node; The management node is further configured to obtain the plurality of second global cluster centers based on information of b third clusters, where the information of the b third clusters comes from some or all of the plurality of access nodes, and b is an integer greater than 1, and send the plurality of second global cluster centers to some or all of the plurality of access nodes; The first access node is further configured to obtain the group to which each client belongs based on the multiple second global clustering centers when the multiple second global clustering centers meet the convergence condition.

9. The system according to any one of claims 1 to 8, characterized in that: The n clients include a plurality of second clients belonging to a first group, and the plurality of groups include the first group; The first access point is further configured to receive a plurality of first data sets, the plurality of first data sets being data sets corresponding to the plurality of second clients, the plurality of first data sets being used to describe a first AI model in the plurality of second clients, and to send a third data set to the management node, the third data set being obtained based on the plurality of first data sets; The management node is further configured to obtain a group model corresponding to the first group based on multiple third data sets, the multiple third data sets are from some or all of the multiple access nodes, and send a fourth data set to some or all of the multiple access nodes, the fourth data set is used to describe the group model corresponding to the first group; The first access node is further configured to send the fourth data set to the multiple second clients.

10. A grouping method, characterized in that: The method is applied to a first access node, the first access node is any access node among multiple access nodes included in a federated learning system, the federated learning system further includes a management node and n clients accessing the first access node, n is an integer greater than 1, and the method includes: Receiving n data sets from the n clients; Clustering the n data sets to obtain information of a plurality of first clusters; Sending information of multiple first clusters to the management node; receiving a plurality of first global cluster centers sent by the management node, where the plurality of first global cluster centers are obtained by the management node based on information of a first clusters, where the information of the a first clusters comes from part or all of the plurality of access nodes, where a is an integer greater than 1, and the plurality of first global cluster centers are used to indicate a plurality of groups of clients; The group to which each client belongs is obtained based on part or all of the plurality of first global cluster centers.

11. The method according to claim 10, characterized in that The obtaining the group to which each client belongs based on part or all of the plurality of first global clustering centers includes: Clustering the n data sets based on the multiple first global cluster centers to obtain multiple second clusters, the multiple first global cluster centers correspond to the multiple second clusters, and for each second cluster, the second cluster includes a partial data set in the n data sets, and the partial data set and the first global cluster center corresponding to the second cluster meet a first similarity condition; The clients corresponding to each data set included in the second cluster are determined as a group.

12. The method according to claim 10 or 11, characterized in that The information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies a second similarity condition with the center of the first cluster.

13. The method according to claim 10 or 11, characterized in that: The information of the first cluster includes the center of the first cluster, the multiple first global cluster centers do not meet the convergence condition, and obtaining the group to which each client belongs based on part or all of the multiple first global cluster centers includes: Clustering the n data sets based on the multiple first global clustering centers to obtain multiple third clusters; Sending information of the plurality of third clusters to the management node; receiving a plurality of second global cluster centers sent by the management node, wherein the plurality of second global cluster centers are used to indicate the plurality of groups; When the multiple second global cluster centers satisfy the convergence condition, the group to which each client belongs is obtained based on the multiple second global cluster centers.

14. The method according to any one of claims 10 to 13, characterized in that: The n data sets are n first data sets from the n clients. For each client, the first data set of the client is used to describe a first artificial intelligence AI model in the client. The clustering of the n data sets to obtain information of a plurality of first clusters includes: Performing sparse processing on the n first data sets to obtain n second data sets, the n first data sets correspond to the n second data sets, and for each first data set, the data amount of the first data set is smaller than the data amount of the second data set corresponding to the first data set; A clustering operation is performed on the n second data sets to obtain information of the multiple first clusters.

15. The method according to any one of claims 10 to 14, characterized in that: The method further comprises: Receiving multiple first data sets sent by multiple clients in a first group, the n clients include the multiple clients, the multiple first data sets are used to describe a first AI model in the multiple clients, and the multiple groups include the first group; Sending a third data set to the management node, where the third data set is obtained based on the multiple first data sets; receiving a fourth data set sent by the management node, where the fourth data set is used to describe a grouping model corresponding to the first grouping; The fourth data set is sent to each client in the first group.

16. A grouping method, characterized in that: The method is applied to a management node in a federated learning system, the federated learning system includes multiple access nodes and multiple clients accessing each access node, the multiple access nodes include a first access node, and the method includes: Receive a first cluster of information, where the a first cluster of information comes from part or all of the multiple access nodes, a is an integer greater than 1, the first cluster of information sent by the first access node is obtained by clustering n data sets, n is an integer greater than 1, and the n data sets come from n clients accessing the first access node; Acquire a plurality of first global cluster centers based on the information of the a first clusters, wherein the plurality of first global cluster centers are used to indicate a plurality of groups of the client; The multiple first global cluster centers are sent to part or all of the multiple access nodes, and the first access nodes are used to obtain the group to which each of the n clients belongs based on part or all of the multiple first global cluster centers.

17. The method according to claim 16, characterized in that The information of the first cluster includes at least one data set in the first cluster, and the at least one data set and the center of the first cluster satisfy a second similarity condition; The obtaining a plurality of first global cluster centers based on the information of the a first clusters includes: A clustering operation is performed on a data set including information of the a first clusters to obtain a plurality of first global clusters, wherein the plurality of first global cluster centers include centers of the plurality of first global clusters.

18. The method according to claim 16, characterized in that The information of the first cluster includes the center of the first cluster; The obtaining a plurality of first global cluster centers based on the information of the a first clusters comprises: Based on the weights corresponding to the a first clusters and the centers of the a first clusters, the multiple first global cluster centers are obtained.

19. The method according to claim 18, characterized in that The plurality of first global cluster centers do not satisfy a convergence condition, and the method further comprises: receiving b third clustering information, where the b third clustering information comes from part or all of the multiple access nodes, b is an integer greater than 1, and the third clustering sent by the first access node is obtained by clustering the n data sets by the first access node based on the multiple first global clustering centers; Based on the information of the b third clusters, obtaining a plurality of second global cluster centers; The plurality of second global cluster centers are sent to part or all of the plurality of access nodes.

20. The method according to any one of claims 16 to 19, characterized in that: The method further comprises: receiving a plurality of third data sets, the plurality of third data sets coming from part or all of the plurality of access nodes, the third data set sent by the first access node being used to describe a first AI module in a plurality of clients belonging to a first group, the n clients including the plurality of clients, and the plurality of groups including the first group; Acquire a grouping model corresponding to the first grouping based on multiple third data sets; A fourth data set is sent to part or all of the plurality of access nodes, where the fourth data set is used to describe the grouping model.

21. A grouping method, characterized in that: The method is applied to a client, the client includes training data, and the method includes: Perform model training based on the training data to obtain a first artificial intelligence AI model; Performing a sparse processing on the first data set to obtain a second data set, where the first data set is used to describe the first AI model, and the amount of data in the second data set is less than the amount of data in the first data set; The second data set is sent, where the second data set is used to request to determine the group to which the client belongs.

22. The method according to claim 21, characterized in that The first data set includes a plurality of first data for describing the first AI model, and the sparse processing of the first data set to obtain the second data set includes: Selecting m first data satisfying a first condition from the first data set, where m is an integer greater than or equal to 1; The second data set is acquired based on the m first data.

23. The method of claim 22, wherein: The first condition includes that the m first data are the m data with the largest absolute values ​​in the first data set, or the m first data are the m data with numerical values ​​in the first data set that are within a first range.

24. The method according to claim 22 or 23, characterized in that The second data set includes the m first data and the sequence numbers of the m first data in the first data set; or, The second data set includes m first bits and xm second bits, x is the number of first data included in the first data set, x is an integer greater than m, the m first bits correspond to the m first data, the xm second bits correspond to the xm first data, and the xm first data are the data in the first data set other than the m first data.

25. The method according to claim 22 or 23, characterized in that The acquiring the second data set based on the m first data includes: The m first data are quantized to obtain m second data, the m first data correspond to the m second data, the bit width of each first data is respectively greater than the bit width of the second data corresponding to each first data, and the second data set includes the m second data and the serial number of the first data corresponding to each second data in the first data set.

26. The method of claim 21, wherein: The type of the first data set is a matrix or a vector, and the sparse processing of the first data set to obtain the second data set includes: Acquire a fifth data set, wherein the dimension of the fifth data set is smaller than the dimension of the first data set; The fifth data set and the first data set are operated at least once to obtain the second data set.

27. The method according to any one of claims 21 to 26, characterized in that The first data set is obtained based on network parameters of the second AI model and / or network parameters of the first AI model, and the first AI model is obtained by training the second AI model based on the training data.

28. The method according to any one of claims 21 to 27, characterized in that The method further comprises: sending the first data set; receiving a fourth data set, the fourth data set being used to describe a group model corresponding to the group, the group model being obtained based on a first data set sent by each client in the group; Adjust network parameters of the first AI model based on the fourth data set.

29. A grouping device, characterized in that: The device is any one of a plurality of access nodes included in a federated learning system, the federated learning system further includes a management node and n clients accessing the device, where n is an integer greater than 1, and the device includes: A receiving unit, configured to receive n data sets from the n clients; A processing unit, used for clustering the n data sets to obtain information of a plurality of first clusters; A sending unit, configured to send information of a plurality of first clusters to the management node; The receiving unit is further used to receive a plurality of first global cluster centers sent by the management node, where the plurality of first global cluster centers are obtained by the management node based on information of a first clusters, where the information of the a first clusters comes from part or all of the plurality of access nodes, where a is an integer greater than 1, and the plurality of first global cluster centers are used to indicate a plurality of groups of clients; The processing unit is configured to determine the group to which each client belongs based on part or all of the multiple first global clustering centers.

30. The device according to claim 29, characterized in that The processing unit is used for: Clustering the n data sets based on the multiple first global cluster centers to obtain multiple second clusters, the multiple first global cluster centers correspond to the multiple second clusters, and for each second cluster, the second cluster includes a partial data set in the n data sets, and the partial data set and the first global cluster center corresponding to the second cluster meet a first similarity condition; The clients corresponding to each data set included in the second cluster are determined as a group.

31. The device according to claim 29 or 30, characterized in that The information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies a second similarity condition with the center of the first cluster.

32. The device according to claim 29 or 30, characterized in that The information of the first cluster includes the center of the first cluster, and the multiple first global cluster centers do not meet the convergence condition; The processing unit is further used to cluster the n data sets based on the multiple first global clustering centers to obtain multiple third clusters; The sending unit is further configured to send information of the plurality of third clusters to the management node; The receiving unit is further used to receive a plurality of second global cluster centers sent by the management node, where the plurality of second global cluster centers are used to indicate the plurality of groups; The processing unit is further configured to obtain the group to which each client belongs based on the multiple second global clustering centers when the multiple second global clustering centers meet the convergence condition.

33. The device according to any one of claims 29 to 32, characterized in that The n data sets are n first data sets from the n clients, and for each client, the first data set of the client is used to describe a first AI model in the client, and the processing unit is further used to: Performing sparse processing on the n first data sets to obtain n second data sets, the n first data sets correspond to the n second data sets, and for each first data set, the data amount of the first data set is smaller than the data amount of the second data set corresponding to the first data set; A clustering operation is performed on the n second data sets to obtain information of the multiple first clusters.

34. The device according to any one of claims 29 to 33, characterized in that The receiving unit is further configured to receive multiple first data sets sent by multiple clients in a first group, the n clients include the multiple clients, the multiple first data sets are used to describe a first AI model in the multiple clients, and the multiple groups include the first group; The sending unit is further configured to send a third data set to the management node, where the third data set is obtained based on the multiple first data sets; The receiving unit is further used to receive a fourth data set sent by the management node, where the fourth data set is used to describe a grouping model corresponding to the first grouping; The sending unit is further configured to send the fourth data set to each client in the first group.

35. A grouping device, characterized in that: The device is applied to a management node in a federated learning system, the federated learning system includes multiple access nodes and multiple clients connected to each access node, the multiple access nodes include a first access node, and the device includes: a receiving unit, configured to receive a first cluster information, wherein the a first cluster information comes from part or all of the multiple access nodes, a is an integer greater than 1, the first cluster information sent by the first access node is obtained by clustering n data sets, n is an integer greater than 1, and the n data sets come from n clients accessing the first access node; A processing unit, configured to obtain a plurality of first global cluster centers based on information of the a first clusters, wherein the plurality of first global cluster centers are used to indicate a plurality of groups of the client; A sending unit is used to send the multiple first global cluster centers to some or all of the multiple access nodes, and the first access node is used to determine the group to which each of the n clients belongs based on some or all of the multiple first global cluster centers.

36. The device according to claim 35, characterized in that The information of the first cluster includes at least one data set in the first cluster, and the at least one data set and the center of the first cluster satisfy a second similarity condition; The processing unit is used to perform clustering operation on the data set included in the information of the a first clusters to obtain multiple first global clusters, and the multiple first global cluster centers include the centers of the multiple first global clusters.

37. The device according to claim 35, characterized in that The information of the first cluster includes the center of the first cluster; The processing unit is used to obtain the multiple first global cluster centers based on the weights corresponding to the a first clusters and the centers of the a first clusters.

38. The device according to claim 37, characterized in that The plurality of first global cluster centers do not satisfy the convergence condition, The receiving unit is further configured to receive information of b third clusters, where the information of the b third clusters comes from part or all of the multiple access nodes, b is an integer greater than 1, and the third cluster sent by the first access node is obtained by clustering the n data sets by the first access node based on the multiple first global cluster centers; The processing unit is further used to obtain multiple second global cluster centers based on the information of the b third clusters; The sending unit is further configured to send the multiple second global clustering centers to part or all of the multiple access nodes.

39. The device according to any one of claims 35 to 38, characterized in that The receiving unit is further configured to receive multiple third data sets, where the multiple third data sets come from part or all of the multiple access nodes, and the third data set sent by the first access node is used to describe a first AI module in multiple clients belonging to a first group, where the n clients include the multiple clients, and the multiple groups include the first group; The processing unit is further configured to obtain a grouping model corresponding to the first grouping based on the multiple third data sets; The sending unit is further used to send a fourth data set to part or all of the multiple access nodes, where the fourth data set is used to describe the grouping model.

40. A grouping device, characterized in that: The device stores training data, and the device comprises: A processing unit, configured to perform model training based on the training data to obtain a first artificial intelligence AI model; The processing unit is further used to perform sparse processing on the first data set to obtain a second data set, the first data set is used to describe the first AI model, and the amount of data in the second data set is less than the amount of data in the first data set; The sending unit is used to send the second data set, where the second data set is used to request to determine the group to which the device belongs.

41. The device according to claim 40, characterized in that The first data set includes a plurality of first data for describing the first AI model, and the processing unit is configured to: Selecting m first data satisfying a first condition from the first data set, where m is an integer greater than or equal to 1; The second data set is acquired based on the m first data.

42. The device according to claim 41, characterized in that The first condition includes that the m first data are the m data with the largest absolute values ​​in the first data set, or the m first data are the m data with numerical values ​​in the first data set that are within a first range.

43. The device according to claim 41 or 42, characterized in that The second data set includes the m first data and the sequence numbers of the m first data in the first data set; or, The second data set includes m first bits and xm second bits, x is the number of first data included in the first data set, x is an integer greater than m, the m first bits correspond to the m first data, the xm second bits correspond to the xm first data, and the xm first data are the data in the first data set other than the m first data.

44. The device according to claim 41 or 42, characterized in that The processing unit is used for: The m first data are quantized to obtain m second data, the m first data correspond to the m second data, the bit width of each first data is respectively greater than the bit width of the second data corresponding to each first data, and the second data set includes the m second data and the serial number of the first data corresponding to each second data in the first data set.

45. The device according to claim 40, characterized in that The type of the first data set is a matrix or a vector, and the processing unit is used for: Acquire a fifth data set, wherein the dimension of the fifth data set is smaller than the dimension of the first data set; The fifth data set and the first data set are operated at least once to obtain the second data set.

46. ​​The device according to any one of claims 40 to 45, characterized in that The first data set is obtained based on network parameters of the second AI model and / or network parameters of the first AI model, and the first AI model is obtained by training the second AI model based on the training data.

47. The device according to any one of claims 40 to 46, characterized in that The device further comprises: a receiving unit; The sending unit is further used to send the first data set; The receiving unit is configured to receive a fourth data set, where the fourth data set is used to describe a group model corresponding to the group, where the group model is obtained based on a first data set sent by each client in the group; The processing unit is further used to adjust the network parameters of the first AI model based on the fourth data set.

48. A computing device, characterized in that Each device in the computing device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions so that the computing device executes the method according to any one of claims 10-28.

49. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 10 to 28 is implemented.

50. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 10 to 28 is implemented.

Images