Grouping method and apparatus, and system and storage medium

Abstract

The present application belongs to the field of communications. Disclosed are a grouping method and apparatus, and a system and a storage medium. The method comprises: receiving n data sets from n clients; clustering the n data sets to obtain information of one or more first clusters; sending the information of the one or more first clusters to a management node; receiving one or more first global clustering centers sent by the management node, wherein the one or more first global clustering centers are obtained by means of the management node on the basis of information of a first clusters, the information of the a first clusters is from some or all of a plurality of access nodes, a is an integer greater than 1, and the one or more first global clustering centers are used for indicating one or more groups; and on the basis of some or all of the one or more first global clustering centers, determining a group to which each client belongs. The present application can improve the efficiency of grouping.

Description

Grouping method, device, system and storage medium

[0001] This application claims priority to Chinese patent application No. 202310803757.3, filed on June 30, 2023, entitled “Grouping method, device, system and storage medium,” the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of communications, and in particular to a grouping method, device, system, and storage medium. Background Art

[0003] A federated learning system is a distributed machine learning system used to train models. It includes a server and multiple clients, each of which locally contains training data and an artificial intelligence (AI) model. Each client uses its local training data to train its local AI model and sends a first data set to the server. The first data set includes multiple network parameters of the trained AI model.

[0004] The server receives a first data set sent by each client and divides the multiple clients into multiple groups based on the first data set sent by each client. For each group, the server aggregates the network parameters included in the first data set sent by each client in the group to obtain a group model corresponding to the group. The server then sends a second data set to each client in the group, the second data set including multiple network parameters of the group model. The clients in the group adjust their local AI models based on the second data set.

[0005] The total data volume of the first data sets sent by the multiple clients is often very large, which results in a large amount of computation required for the server to perform grouping operations based on the first data sets sent by each client, thereby reducing grouping efficiency.

[0006] Summary of the Invention

[0007] This application provides a grouping method, device, system, and storage medium to improve grouping efficiency. The technical solution is as follows:

[0008] In a first aspect, the present application provides a federated learning system, comprising a management node, multiple access nodes, and multiple clients connected to each access node, the multiple access nodes including a first access node, the first client being one of n clients connected to the first access node, where n is an integer greater than 1. The first client is configured to send a data set to the first access node, the n clients connected to the first access node including the first client, the data set describing a first artificial intelligence (AI) model in the first client. The first access node is configured to cluster the data sets from the n clients to obtain information about one or more first clusters, and to send the information about the one or more first clusters to the management node. The management node is configured to obtain one or more first global cluster centers based on information about a first clusters, the information about the a first clusters being from some or all of the multiple access nodes, where a is an integer greater than 1, the multiple first global cluster centers indicating multiple groupings of clients. The management node is further configured to send the one or more first global cluster centers to some or all of the multiple access nodes. The first access node is further configured to obtain a group to which each of the n clients belongs based on some or all of the one or more first global cluster centers.

[0009] Because the federated learning system includes multiple access nodes, each access node receives a data set sent by its respective client, and each access node performs clustering operations on the received data set in parallel to obtain information about one or more first clusters. The first access node sends information about one or more first clusters to the management node. The amount of data in the first cluster information is relatively small, and the management node quickly obtains one or more first global cluster centers based on the received information about the one or more first clusters. The first access node quickly receives the one or more first global cluster centers and determines the group to which each client belongs based on the one or more first global cluster centers, thereby improving grouping efficiency.

[0010] In one possible implementation, the n data sets are first data sets from n clients. The first access node is configured to perform a sparsification process on the n first data sets to obtain n second data sets, where the n first data sets correspond to the n second data sets. For each first data set, the data volume of the first data set is smaller than the data volume of the corresponding second data set. A clustering operation is performed on the n second data sets to obtain information about one or more first clusters. This can reduce the amount of data required for the clustering operation and improve the efficiency of the clustering operation.

[0011] In another possible implementation, the n data sets are second data sets from n clients. The first client is configured to perform a sparsification process on the first data set to obtain a second data set, the first data set being used to describe the first AI model, the second data set having a smaller data volume than the first data set, and the second data set is transmitted to the first access node. This reduces the network resources required to transmit the second data set and the amount of data required to perform clustering operations on the second data set.

[0012] In another possible implementation, the first data set includes multiple first data for describing the first AI model, and the second data set includes m first data and the serial numbers of the m first data in the first data set, where m is an integer greater than 1, and the m first data are the first data in the first data set that meet the first condition. The first access node is used to perform clustering operations based on the m first data and the serial numbers of the m first data included in each of the n second data sets to obtain information of one or more first clusters. Among them, the m first data that meet the first condition are data with significant features in the first data set, and the clustering operation is performed based on the m first data and the serial numbers of the m first data included in each of the n second data sets to avoid affecting the accuracy of the clustering.

[0013] In another possible implementation, the first access node is used to cluster n data sets based on one or more first global cluster centers to obtain one or more second clusters, where the one or more first global cluster centers correspond to the one or more second clusters. For each second cluster, the second cluster includes a portion of the n data sets, where the portion of the data set and the first global cluster center corresponding to the second cluster satisfy a first similarity condition, and the clients corresponding to each data set included in the second cluster are determined as a group. In this way, the grouping of each client can be accurately determined using the one or more first global cluster centers.

[0014] In another possible implementation, the first cluster information includes at least one data set in the first cluster, and the at least one data set satisfies a second similarity condition with the center of the first cluster. The management node is configured to perform a clustering operation on the a data sets included in the first cluster information to obtain one or more first global clusters, where the one or more first global cluster centers include the centers of the one or more first global clusters. This can reduce the number of data sets included in the first cluster information and reduce the amount of data in the first cluster information.

[0015] In another possible implementation, the information about the first cluster includes the center of the first cluster, and the management node is configured to obtain one or more first global cluster centers based on the weights corresponding to the a first clusters and the centers of the a first clusters. The first cluster centers included in the information about the first cluster reduce the amount of data in the information about the first cluster.

[0016] In another possible implementation, the one or more first global cluster centers do not meet the convergence condition. The first access node is used to cluster n data sets based on the one or more first global cluster centers to obtain one or more third clusters. The first access node is also used to send information of one or more third clusters to the management node. The management node is also used to obtain one or more second global cluster centers based on the information of b third clusters, the information of b third clusters comes from some or all of the multiple access nodes, b is an integer greater than 1, and the one or more second global cluster centers are sent to some or all of the multiple access nodes. The first access node is also used to obtain the group to which each client belongs based on the one or more second global cluster centers when the one or more second global cluster centers meet the convergence condition. In this way, one or more global cluster centers can be obtained cyclically until one or more global cluster centers that meet the convergence condition are obtained, and the grouping accuracy is improved by the one or more global cluster centers that meet the convergence condition.

[0017] In another possible implementation, the n clients include multiple second clients belonging to a first group, and the multiple groups include the first group. The first access point is also used to receive multiple first data sets, which are data sets corresponding to the multiple second clients, and the multiple first data sets are used to describe the first AI model in the multiple second clients, and send a third data set to the management node, where the third data set is obtained based on the multiple first data sets. The management node is also used to obtain the group model corresponding to the first group based on multiple third data sets, and the multiple third data sets come from part or all of the multiple access nodes, and send a fourth data set to the multiple access nodes, where the fourth data set is used to describe the group model corresponding to the first group. The first access node is also used to send a fourth data set to the multiple second clients. In this way, a distributed approach is adopted to obtain the group model corresponding to the first group, thereby improving the efficiency of obtaining the group model.

[0018] In a second aspect, the present application provides a grouping method, which is applied to a first access node, which is any access node among the multiple access nodes included in the federated learning system. The federated learning system also includes a management node and n clients connected to the first access node, where n is an integer greater than 1. In the method, n data sets are received from the n clients. The n data sets are clustered to obtain information of one or more first clusters. The information of one or more first clusters is sent to the management node. One or more first global cluster centers are received from the management node, and the one or more first global cluster centers are obtained by the management node based on information of a first clusters, and the information of a first clusters comes from part or all of the multiple access nodes, where a is an integer greater than 1, and the one or more first global cluster centers are used to indicate one or more groups of clients. The group to which each client belongs is obtained based on part or all of the one or more first global cluster centers.

[0019] Because the federated learning system includes multiple access nodes, each access node receives a data set sent by its respective client. Each access node performs clustering operations on the received data set in parallel to obtain information about one or more first clusters. The first access node sends information about one or more first clusters to the management node. The first cluster information is relatively small in size, and the management node quickly obtains one or more first global cluster centers based on the received first cluster information. The first access node then determines the group to which each client belongs based on these one or more first global cluster centers, thereby improving grouping efficiency.

[0020] In one possible implementation, the n data sets are clustered based on the one or more first global cluster centers to obtain one or more second clusters, the one or more first global cluster centers corresponding to the one or more second clusters, and for each second cluster, the second cluster includes a portion of the n data sets, and the portion of the data set and the first global cluster center corresponding to the second cluster meet a first similarity condition. The client corresponding to each data set included in the second cluster is determined as a group. In this way, the grouping of each client can be accurately determined using the one or more first global cluster centers.

[0021] In another possible implementation, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies the second similarity condition with the center of the first cluster, thereby reducing the number of data sets included in the information of the first cluster and reducing the data volume of the information of the first cluster.

[0022] In another possible implementation, the information of the first cluster includes the center of the first cluster, and the one or more first global cluster centers do not meet the convergence condition. Based on the one or more first global cluster centers, the n data set is clustered to obtain one or more third clusters. The information of the one or more third clusters is sent to the management node. One or more second global cluster centers sent by the management node are received, and the one or more second global cluster centers are used to indicate the one or more groups. When the one or more second global cluster centers meet the convergence condition, the group to which each client belongs is obtained based on the one or more second global cluster centers. In this way, one or more global cluster centers can be obtained cyclically until one or more global cluster centers that meet the convergence condition are obtained, and the grouping accuracy is improved by the one or more global cluster centers that meet the convergence condition.

[0023] In another possible implementation, the n data sets are n second data sets from n clients. For each client, the second data set corresponding to the client is obtained by sparsifying the first data set. The first data set is used to describe the first AI model in the client, and the data volume of the second data set is smaller than that of the first data set. This reduces the network resources required to transmit the second data set and the amount of data required for clustering operations on the second data set.

[0024] In another possible implementation, the first data set includes multiple first data used to describe a first AI model obtained by model training performed by a client, and the second data set includes m first data and m sequence numbers of the first data in the first data set, where m is an integer greater than 1, and the m first data are first data in the first data set that meet a first condition. A clustering operation is performed based on the m first data and the m sequence numbers of the first data included in each of the n second data sets to obtain information about the one or more first clusters.

[0025] Among them, the m first data that meet the first condition are data with significant features in the first data set, and clustering operations are performed based on the m first data and the serial numbers of the m first data included in each second data set in the n second data sets to avoid affecting the accuracy of clustering.

[0026] In another possible implementation, the n data sets are n first data sets from n clients. For each client, the first data set corresponding to the client is used to describe the first AI model in the client. The n first data sets are sparsely processed to obtain n second data sets. The n first data sets correspond to the n second data sets. For each first data set, the amount of data in the first data set is less than the amount of data in the second data set corresponding to the first data set. Clustering operations are performed on the n second data sets to obtain information of one or more first clusters. This can reduce the amount of data required to calculate the clustering operation and improve the efficiency of the clustering operation.

[0027] In another possible implementation, multiple first data sets are received from multiple clients in a first group, where the n clients include multiple clients, and the multiple first data sets are used to describe a first AI model in the multiple clients, and the multiple groups include the first group. A third data set is sent to a management node, where the third data set is obtained based on the multiple first data sets. A fourth data set is received from the management node, where the fourth data set is used to describe a group model corresponding to the first group. The fourth data set is sent to each client in the first group. In this way, a distributed approach is used to obtain a group model corresponding to the first group, thereby improving the efficiency of obtaining the group model.

[0028] In a third aspect, the present application provides a grouping method, which is applied to a management node in a federated learning system, wherein the federated learning system includes multiple access nodes and multiple clients connected to each access node, and the multiple access nodes include a first access node. In the method, information of a first cluster is received, and the information of the a first cluster comes from part or all of the multiple access nodes, a is an integer greater than 1, and the information of the first cluster sent by the first access node is obtained by clustering n data sets, n is an integer greater than 1, and the n data sets come from n clients connected to the first access node. Based on the information of the a first clusters, one or more first global cluster centers are obtained, and the one or more first global cluster centers are used to indicate one or more groups of clients. The one or more first global cluster centers are sent to part or all of the multiple access nodes, and the first access node is used to obtain the group to which each of the n clients belongs based on part or all of the one or more first global cluster centers.

[0029] Because the federated learning system includes multiple access nodes, each access node receives a data set sent by its respective client, and each access node performs clustering operations on the received data set in parallel to obtain information about one or more first clusters. The management node receives information about one or more first clusters sent by each access node. The amount of data in the first cluster information is relatively small, and the management node quickly obtains one or more first global cluster centers based on the information sent by some or all of the multiple access nodes. The management node then sends these one or more first global cluster centers to some or all of the multiple access nodes. The first access node then determines the group to which each client belongs based on these one or more first global cluster centers, thereby improving grouping efficiency.

[0030] In one possible implementation, for each received first cluster, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies a second similarity condition with the center of the first cluster. A clustering operation is performed on the data sets included in the information of a first cluster to obtain one or more first global clusters, and the one or more first global cluster centers include the centers of the one or more first global clusters. Since the information of the first cluster includes at least one data set in the first cluster, the number of data sets included in the information of the first cluster can be reduced, and the amount of data of the information of the first cluster can be reduced, that is, the amount of data required for the clustering operation can be reduced, thereby improving the efficiency of the clustering operation.

[0031] In another possible implementation, the information about the first cluster includes the center of the first cluster. Based on the weights corresponding to the a first clusters and the centers of the a first clusters, one or more first global cluster centers are obtained. Because the information about the first cluster includes the center of the first cluster, the amount of data in the information about the first cluster can be reduced, that is, the amount of data required to calculate when obtaining the global cluster centers is reduced.

[0032] In another possible implementation, the one or more first global cluster centers do not meet the convergence condition. Receive information of b third clusters, the information of the b third clusters comes from some or all of the one or more access nodes, and the one or more third clusters sent by the first access node are obtained by the first access node clustering the n data sets based on the one or more first global cluster centers. Based on the information of the b third clusters, obtain one or more second global cluster centers. Send the one or more second global cluster centers to some or all of the multiple access nodes. In this way, one or more global cluster centers can be obtained cyclically until one or more global cluster centers that meet the convergence condition are obtained, and the grouping accuracy is improved by the one or more global cluster centers that meet the convergence condition.

[0033] In another possible implementation, multiple third data sets are received, each of which is from some or all of the multiple access nodes. The third data set sent by the first access node is used to describe the first AI module in multiple clients belonging to a first group, where the n clients include the multiple clients, and the multiple groups include the first group. A group model corresponding to the first group is obtained based on the multiple third data sets. A fourth data set is sent to some or all of the multiple access nodes, where the fourth data set is used to describe the group model. In this way, a distributed approach is used to obtain the group model corresponding to the first group, thereby improving the efficiency of obtaining the group model.

[0034] In a fourth aspect, the present application provides a grouping method, the method being applied to a client, the client including training data. In the method, model training is performed based on the training data to obtain a first artificial intelligence (AI) model. The first data set is subjected to sparsification processing to obtain a second data set, the first data set being used to describe the first AI model, the second data set having a smaller data volume than the first data set. The second data set is sent, and the second data set is used to request determination of the group to which the client belongs.

[0035] Since the second data set is obtained by performing sparse processing on the first data set, the data volume of the second data set is smaller than that of the first data set. In this way, when the second data set of each client is used for grouping, the data volume of the grouping operation is reduced, thereby improving the grouping efficiency.

[0036] In one possible implementation, the first data set includes multiple first data for describing the first AI model. m first data that meet a first condition are selected from the first data set, where m is an integer greater than or equal to 1. A second data set is obtained based on the m first data. The m first data that meet the first condition are data with significant features in the first data set. The second data set is obtained based on the m first data. The second data set can be used instead of the first data set for grouping to avoid affecting the accuracy of the grouping.

[0037] In another possible implementation, the first condition includes the m first data being the m data with the largest absolute values ​​in the first data set, or the m first data being the m data with values ​​within a first range in the first data set. This ensures that the selected m first data are data with significant features in the first data set.

[0038] In another possible implementation, the second data set includes m first data and the serial numbers of the m first data in the first data set; or, the second data set includes m first bits and xm second bits, x is the number of first data included in the first data set, x is an integer greater than m, the m first bits correspond to the m first data, the xm second bits correspond to the xm first data, and the xm first data are the data in the first data set other than the m first data.

[0039] In another possible implementation, m first data are quantized to obtain m second data. The m first data correspond to m second data, and the bit width of each first data is greater than the bit width of the second data corresponding to each first data. The second data set includes the m second data and the sequence number of the first data corresponding to each second data in the first data set. This can reduce the data size of the second data set.

[0040] In another possible implementation, the first data set is a matrix or vector. A fifth data set is obtained, where the dimension of the fifth data set is smaller than that of the first data set. At least one operation is performed on the fifth data set and the first data set to obtain a second data set. This method achieves sparsification by reducing the dimensionality, thereby improving the flexibility of sparsification.

[0041] In another possible implementation, the first data set is obtained based on network parameters of the second AI model and / or network parameters of the first AI model, and the first AI model is obtained by training the second AI model based on the training data.

[0042] In another possible implementation, a first data set is sent, a fourth data set is received, and the fourth data set is used to describe a group model corresponding to the group, where the group model is derived based on the first data set sent by each client in the group. Network parameters of the first AI model are adjusted based on the fourth data set.

[0043] In a fifth aspect, the present application provides a grouping device for executing the method in the second aspect or any possible implementation of the second aspect. Specifically, the device includes a unit for executing the method in the second aspect or any possible implementation of the second aspect.

[0044] In a sixth aspect, the present application provides a grouping device for executing the method in the third aspect or any possible implementation of the third aspect. Specifically, the device includes a unit for executing the method in the third aspect or any possible implementation of the third aspect.

[0045] In a seventh aspect, the present application provides a grouping device for executing the method in the fourth aspect or any possible implementation of the fourth aspect. Specifically, the device includes a unit for executing the method in the fourth aspect or any possible implementation of the fourth aspect.

[0046] In an eighth aspect, the present application provides a grouping device comprising at least one processor and a memory, wherein the at least one processor is used to couple with the memory, read and execute instructions in the memory, so as to implement the method in the second aspect or any possible implementation of the second aspect.

[0047] In the ninth aspect, the present application provides a grouping device comprising at least one processor and a memory, wherein the at least one processor is used to couple with the memory, read and execute instructions in the memory, so as to implement the method in the third aspect or any possible implementation of the third aspect.

[0048] In the tenth aspect, the present application provides a grouping device comprising at least one processor and a memory, wherein the at least one processor is used to couple with the memory, read and execute instructions in the memory to implement the method in the fourth aspect or any possible implementation of the fourth aspect.

[0049] In the eleventh aspect, the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium, and the computer program is loaded by a processor to implement the above-mentioned second aspect, third aspect, fourth aspect, any possible implementation of the second aspect, any possible implementation of the third aspect, or any possible implementation of the fourth aspect.

[0050] In the twelfth aspect, the present application provides a computer-readable storage medium for storing a computer program, which is loaded by a processor to execute the method of the second aspect, the third aspect, the fourth aspect, any possible implementation of the second aspect, any possible implementation of the third aspect, or any possible implementation of the fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] FIG1 is a schematic diagram of the structure of a federated learning system provided in an embodiment of the present application;

[0052] FIG2 is a schematic diagram of the structure of another federated learning system provided in an embodiment of the present application;

[0053] FIG3 is a schematic diagram of the structure of another federated learning system provided in an embodiment of the present application;

[0054] FIG4 is a flow chart of a grouping method provided in an embodiment of the present application;

[0055] FIG5 is a flow chart of another grouping method provided in an embodiment of the present application;

[0056] FIG6 is a flow chart of another grouping method provided in an embodiment of the present application;

[0057] FIG7 is a flow chart of another grouping method provided in an embodiment of the present application;

[0058] FIG8 is a flow chart of another grouping method provided in an embodiment of the present application;

[0059] FIG9 is a flow chart of another grouping method provided in an embodiment of the present application;

[0060] FIG10 is a schematic structural diagram of a grouping device provided in an embodiment of the present application;

[0061] FIG11 is a schematic structural diagram of another grouping device provided in an embodiment of the present application;

[0062] FIG12 is a schematic structural diagram of another grouping device provided in an embodiment of the present application;

[0063] FIG13 is a schematic diagram of a device structure provided in an embodiment of the present application;

[0064] FIG14 is a schematic diagram of another device structure provided in an embodiment of the present application;

[0065] FIG15 is a schematic diagram of another device structure provided in an embodiment of the present application;

[0066] FIG16 is a schematic diagram of the structure of another federated learning system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] The embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0068] As the application of artificial intelligence (AI) technology continues to gain popularity, the privacy and security of users' personal data are receiving increasing attention. AI technology requires model training. The system used for model training consists of a server and multiple clients, each of which stores local training data. For each client, this local training data may contain the user's personal data, including privacy. Each client sends its local training data to the server, which then trains the model based on the data sent by each client and distributes the trained model to each client. However, this training data sent by each client may be leaked, potentially compromising user privacy.

[0069] For example, to train an information recommendation model, each client's local training data includes user browsing history. This data can reflect user browsing preferences. For example, some users may prefer entertainment information, others political information, and still others financial information. If each client sends its local training data to the server, it could be leaked, allowing others to infer user preferences and compromise personal privacy.

[0070] In order to protect personal privacy and improve the accuracy of training AI models. For each client, the client performs model training based on local training data to obtain an AI model, and sends a data set to the server. The data set includes multiple network parameters of the AI ​​model. The server performs clustering operations based on the data set sent by each client to obtain multiple clusters. The multiple clusters correspond one-to-one to multiple groups, and the clients corresponding to each data set included in the same cluster are divided into a group. For each group, the group model corresponding to the group is obtained based on the data set sent by each client in the group, and the multiple network parameters of the group model are sent to each client in the group. Optionally, the multiple network parameters of the group model are obtained by weighted averaging the data set sent by each client. Each client in the group adjusts the local AI model based on the multiple network parameters of the group model, and the function of the adjusted AI model is the same as that of the group model.

[0071] If the group model has not converged, each client in the group continues to train the adjusted AI model based on local training data and sends a data set to the server, which includes multiple network parameters of the trained AI model. The server obtains the group model corresponding to the group based on the data set sent by each client in the group and sends the multiple network parameters of the group model to each client in the group. Each client in the group adjusts the local AI model based on the multiple network parameters of the group model. The above process is repeated until the group model converges.

[0072] The AI ​​models included in each client in the same group may have similar or identical functions. For example, still taking the AI ​​model recommended by the above information as an example, for each client, the client performs model training based on local training data (records of user browsing information) to obtain an AI model, and sends a data set to the server. The data set includes multiple network parameters of the AI ​​model, and the multiple network parameters can reflect the user's browsing information preferences. The server performs clustering operations based on the data set sent by each client to obtain multiple clusters. The multiple clusters correspond one-to-one to multiple groups, and the clients corresponding to each data set included in the same cluster are divided into a group. Based on the data set sent by each client in the group, the group model corresponding to the group is obtained, and multiple network parameters of the group model are sent to each client in the group. Each client in the group adjusts the local AI model based on the network parameters of the group model.

[0073] The users corresponding to each client in the group may have the same browsing preferences, and the group model corresponding to the group is used to recommend information corresponding to these preferences. For example, assuming that the users corresponding to each client in the group like to browse entertainment information, the group model corresponding to the group is more inclined to recommend entertainment information to the users. Each client in the group adjusts its local AI model based on the network parameters of the group model, so that the AI ​​model in each client in the group is more inclined to recommend entertainment information that matches the user's personal preferences.

[0074] Clustering itself is a computationally complex process, and the data set sent by each client includes multiple network parameters for the AI ​​model. These parameters can number in the thousands, tens of thousands, or even millions, or even more. Therefore, the data set sent by each client is very large. Therefore, the server performs clustering operations on each client's data set, resulting in a very large amount of computational effort and reduced grouping efficiency.

[0075] When there are many clients, the server may not have enough storage resources to store the data sets sent by the clients, or the server may not have enough computing resources to perform clustering operations on the data sets sent by the clients, resulting in the inability to perform grouping work in a timely manner.

[0076] To solve the above-mentioned problems, referring to FIG1 , an embodiment of the present application provides a federated learning system 100, which includes: a management node 101, multiple access nodes 102 communicating with the management node 101, and clients 103 connected to each access node 102. For each access node 102, there are multiple clients 103 connected to the access node 102.

[0077] Each client 103 in the federated learning system 100 locally includes training data. For each client 103, the client 103 performs model training based on the local training data of the client 103 to obtain a first AI model, and sends a data set to the access node 102 to which the client 103 accesses. The data set is used to describe the first AI model.

[0078] In some embodiments, the client 103 can receive the system global model sent by the access node 102, train the system global model based on the local training data of the client 103, and obtain a first AI model.

[0079] In some embodiments, the data set sent by the client 103 to the access node 102 may be a first data set or a second data set. The first data set includes multiple first data used to describe the first AI model, and the second data set is a data set after the first data set is subjected to sparse processing. The amount of data in the second data set is smaller than that in the first data set.

[0080] For each access node 102 in the multiple access nodes 102 , for ease of description, the access node 102 is referred to as a first access node 102 . The number of clients 103 accessing the first access node 102 is n, where n is an integer greater than 1.

[0081] The first access node 102 receives the data sets sent by the n clients 103, that is, the first access node 102 receives a total of n data sets, performs a clustering operation on the n data sets to obtain multiple first clusters, and sends information about the multiple first clusters to the management node 101. Like the first access node 102, each of the other access nodes 102 also sends information about the multiple first clusters to the management node 101.

[0082] The management node 101 receives information of multiple first clusters sent by some or all of the multiple access nodes 102, and receives information of a first clusters in total, where a is an integer greater than 1. Based on the information of a first clusters, multiple global cluster centers are obtained, and the multiple global cluster centers are used to indicate multiple groups, and the multiple global cluster centers are sent to some or all of the multiple access nodes 102.

[0083] The first access node 102 receives the multiple global cluster centers and determines, based on some or all of the multiple global cluster centers, a group to which each of the n clients 103 belongs. Like the first access node 102, the other access nodes 102 also determine, based on some or all of the multiple global cluster centers, a group to which each of the multiple clients 103 connected to them belongs.

[0084] Because multiple clients 103 in the federated learning system 100 are connected to multiple access nodes 102, the first access node 102 performs a clustering operation on the n data sets sent by the n clients connected to the first access node 102 to obtain multiple first clusters. This allows multiple access nodes 102 to perform clustering operations in parallel, significantly reducing the number of data sets required for clustering operations by each access node 102 and the amount of data required for clustering operations by each access node 102. This allows each access node 102 to quickly perform clustering operations in real time using relatively few computing resources, obtain information about multiple first clusters, and send this information to the management node 101. Because the amount of data in each first cluster is relatively small, each access node 102 can quickly send all the information about the multiple first clusters to the management node 101, and the management node 101 can quickly receive the information about the multiple first clusters sent by each access node 102.

[0085] The amount of data in the information about the multiple first clusters sent by first access node 102 is much smaller than the amount of data in n data sets. This significantly reduces the amount of computation required for management node 101 to obtain the multiple global cluster centers. Management node 101 can quickly obtain multiple global cluster centers based on the information about the first clusters sent by each access node 102. First access node 102 determines the group to which each of the n clients 103 belongs based on the multiple global cluster centers, thereby significantly improving grouping efficiency.

[0086] After determining the grouping of each client 103, the management node 101 is also used to obtain the group model corresponding to the first group based on the first data set of each client 103 in the first group. The multiple groups include the first group, and the group model corresponding to the first group is sent to each client 103 in the first group through each access node 102.

[0087] In some embodiments, the management node 101 and each access node 102 may be different devices, such as different servers.

[0088] For example, referring to FIG2 , the management node 101 and each access node 102 are different devices. For each client 103 , the client 103 includes a local data set 1031 , a local model library 1032 , a local training module 1033 , a sparsification module 1034 , a first communication module 1035 , and the like.

[0089] The local dataset 1031 is used to store the training data and other content in the client 103, and the local model library 1032 is used to store the AI ​​model obtained by the client 103. For example, the local model library 1032 is used to store the first AI model, the system global model, and the group model corresponding to the group to which the client 103 belongs. The system global model and group model will be described in detail in the following content and will not be introduced here.

[0090] The local training module 1033 is configured to perform model training based on the training data. The sparsification module 1034 is configured to perform sparsification processing on the data set to be sent to the access node 102 to reduce the data volume of the data set, and transmit the sparsified data set to the access node 102 via the first communication module 1035. The detailed implementation process of the sparsification processing will be described in detail later and will not be introduced here.

[0091] For each access node 102 , the access node 102 includes a client parameter data set 1021 , a dimensionality reduction processing module 1022 , a client management module 1023 , a model issuing module 1024 and a second communication module 1025 .

[0092] The model sending module 1024 is used to send the system global model or the group model corresponding to the group to which the client 103 belongs to the client 103 through the second communication module 1025.

[0093] The client parameter data set 1021 is used to store data sets sent by multiple clients 103 connected to the access node 102, that is, to store multiple data sets. Optionally, the client parameter data set 1021 may also be used to store information such as identification information of each client 103, the status of each client 103, and / or the group identifier of the group to which each client 103 belongs.

[0094] The dimensionality reduction processing module 1022 is used to reduce the dimensionality of the multiple data sets by clustering, perform clustering operations on the multiple data sets to obtain multiple first clusters, and send information of the multiple first clusters to the management node 101 through the second communication module 1025.

[0095] The client management module 1023 is used to manage the multiple clients 103 .

[0096] The management node 101 includes a feature extraction module 1011 , a client clustering module 1012 , a model aggregation module 1013 , a model management module 1014 , a global model library 1015 and a third communication module 1016 .

[0097] The feature extraction module 1011 is configured to receive information of multiple first clusters sent by each access node 102 through the third communication module 1016, and extract or construct feature information for client cluster grouping based on the received information of each first cluster.

[0098] The client clustering module 1012 is configured to obtain multiple global cluster centers based on the extracted or constructed feature information.

[0099] The model aggregation module 1013 is used to generate a group model corresponding to each group based on the local model parameters uploaded by each client and with reference to the group to which the client belongs.

[0100] The model management module 1014 is used to manage the lifecycle of the system global model and each group model, and to control the updating and aging processes of the system global model.

[0101] The global model library 1015 is used to store the system global model and the group models corresponding to each group. Optionally, the system global model and the group models corresponding to each group can be deployed in the access node 102 in the form of copies or mirrors.

[0102] In some embodiments, the management node 101 and each access node 102 may be different virtualized instances within a single device or a cluster. For example, the management node 101 and each access node 102 may be different virtual machines within a single server or a cluster. For another example, the management node 101 and each access node 102 may be different containers within a single server or a cluster. For another example, the management node 101 and each access node 102 may be different microservices, processes, or threads within a single server or a cluster.

[0103] For example, referring to FIG3 , the management node 101 and each access node 102 are different virtualized instances in a device or a cluster, and the management node 101 and each access node 102 use a common communication module (such as the second communication module or the third communication module) to communicate with each client 103 .

[0104] In some embodiments, the client 103 may be a physical device such as a terminal device or an edge server, or may be an application deployed on the physical device.

[0105] Referring to Figure 4, an embodiment of the present application provides a grouping method 400, which is applied to the federated learning system 100 shown in Figure 1, Figure 2 or Figure 3. In the method 400, the client performs model training to obtain a first AI model, obtains a first data set for describing the first AI model, performs sparse processing on the first data set to obtain a second data set, and sends the second data set.

[0106] For each client connected to the access node, the access node obtains multiple first clusters based on each client's second data set and sends information about the multiple first clusters to the management node. Based on the information about the multiple first clusters sent by each access node, the management node obtains multiple global cluster centers, each corresponding to a plurality of groups, and sends these multiple global cluster centers to each access node. The access node determines the group to which the client belongs based on the multiple global cluster centers. Method 400 includes the following process.

[0107] Step 401: A first client performs model training based on local training data to obtain a first AI model. The first client is any client in the federated learning system.

[0108] In some embodiments, the first client obtains the second AI model, performs model training on the second AI model based on local training data, and obtains the first AI model.

[0109] In some embodiments, the second AI model may be a global model, and each client in the federated learning system may include the same second AI model. Alternatively, the first client may locally include an AI model framework, and the second AI model is derived based on the AI ​​model framework.

[0110] The following two methods are listed below to obtain the second AI model.

[0111] Method 1: The management node initializes multiple network parameters of the first system's global model. The first client obtains the multiple network parameters of the first system's global model and adjusts the local AI model framework based on the multiple network parameters of the first system's global model to obtain a second AI model. The second AI model may be the same as or similar to the first system's global model. The first client then trains the second AI model based on local training data to obtain the first AI model.

[0112] In some embodiments, the management node randomly initializes multiple network parameters of the first system global model, or uses historically saved model parameters to initialize multiple network parameters of the first system global model.

[0113] In some embodiments, after initializing multiple network parameters of the first system global model, the management node may send the multiple network parameters of the first system global model to each access node. Each access node receives the multiple network parameters of the first system global model and sends the multiple network parameters of the first system global model to each client connected to the access node. Each client connected to the access node receives the multiple network parameters of the first system global model.

[0114] In some embodiments, after initializing multiple network parameters of the first system global model, the management node obtains a storage path of a storage space used to store the multiple network parameters of the first system global model and sends the storage path to each access node. Each access node receives the storage path and sends the storage path to each client connected to the access node. Each client connected to the access node receives the storage path and downloads the multiple network parameters of the first system global model from the management node based on the storage path. Optionally, the storage path can be a uniform resource locator (URL), etc.

[0115] Method 2: The first client obtains the second AI model through a general federated learning method.

[0116] In mode 2, the general federated learning method includes the following operations of steps 1011-1016.

[0117] 1011: The first client obtains the first system global model using the above method 1, trains the first system global model based on local training data, and obtains the target AI model.

[0118] 1012: The first client sends a first target data set to the management node, where the first target data set is used to describe the target AI model.

[0119] In some embodiments, the first target data set includes a plurality of network parameters of the target AI model. Alternatively, the first target data set includes a plurality of network parameter update values ​​of the target AI model, wherein the plurality of network parameter update values ​​are differences between the plurality of network parameters of the target AI model and the plurality of network parameters of the first system global model.

[0120] Optionally, the first target data set may be a matrix, vector or array, etc., and the elements in the matrix, vector or array are multiple network parameters of the target AI model, or the elements in the matrix, vector or array are multiple network parameter update values ​​of the target AI model.

[0121] In 1012, a first client accesses a first access node and sends a first target data set to the first access node. There are multiple clients accessing the first access node. In addition to receiving the first target data set sent by the first client, the first access node also receives first target data sets sent by clients other than the first client among the multiple clients. In other words, the first access node receives the first target data sets sent by the multiple clients and forwards the first target data sets sent by the multiple clients to the management node. There are multiple access nodes communicating with the management node, and the access nodes other than the first access node among the multiple access nodes also forward the first target data sets sent by the clients to the management node.

[0122] 1013: The management node receives a first target data set sent by each client in the federated learning system, and obtains a second system global model based on the first target data set sent by each client.

[0123] In some embodiments, the first target data set includes multiple network parameters of the target AI model, and a weighted average operation is performed based on the weight corresponding to each client and the first target data set sent by each client to obtain multiple network parameters of the second system global model.

[0124] In some embodiments, the first target data set includes multiple network parameter update values ​​of the target AI model, and a weighted average operation is performed based on the weight corresponding to each client and the first target data set sent by each client to obtain multiple network parameter update values ​​of the second system global model.

[0125] The above-mentioned method of using weighted averaging operation is only one implementation method for obtaining multiple network parameters of the global model of the second system. Of course, in actual implementation, other methods can also be used to obtain multiple network parameters of the global model of the second system. For example, other methods can be federated normalized averaging algorithm (FedNova), federated optimization of heterogeneous networks (FedProx) algorithm or stochastic controlled averaging algorithm (SCAFFOLD), etc.

[0126] 1014: The management node sends a second target data set to each client, where the second target data set is used to describe the second system global model.

[0127] The second target data set includes a plurality of network parameters of the second system global model, or a plurality of updated values ​​of the network parameters of the second system global model.

[0128] In 1014 , the management node sends a second target data set to the multiple access nodes. For each access node, the access node sends the second target data set to multiple clients connected to the access node.

[0129] 1015: The first client receives the second target data set, and adjusts the local first system global model based on the second target data set to obtain a second system global model.

[0130] In some embodiments, the second target data set includes multiple network parameters of the second system global model, and the local first system global model is adjusted based on the multiple network parameters of the second system global model to obtain the second system global model.

[0131] In some embodiments, the second target data set includes multiple network parameter update values ​​for the second system global model. The multiple network parameters of the second system global model are obtained based on the multiple network parameter update values ​​of the second system global model and the multiple network parameters of the local first system global model. The local first system global model is adjusted based on the multiple network parameters of the second system global model to obtain the second system global model.

[0132] 1016: The first client trains the second system global model based on the local training data to obtain the target AI model, and returns to execute 1012.

[0133] In some embodiments, when the loop is executed a specified number of times, the first client uses the trained system global model as the second AI model; or, when the trained system global model converges, the first client uses the converged system global model as the second AI model.

[0134] Convergence refers to testing the system global model using at least one test data set, obtaining the test accuracy of the system global model, and determining that the system global model has converged when the test accuracy exceeds a threshold. Alternatively, for multiple system global models trained multiple times, any one of the multiple system global models has converged when the difference in their test accuracy is less than a threshold.

[0135] In some embodiments, the first client is any one of Q clients selected by the management node from the federated learning system, where Q is an integer greater than 1. That is, each of the Q clients executes the method 400 provided in the embodiment of the present application starting from step 401 above.

[0136] In some embodiments, the management node randomly selects Q clients from the federated learning system.

[0137] In some embodiments, the management node selects the Q clients according to the order in which they access the federated learning system. For example, the management node may select the Q clients that first access the federated learning system, or the management node may select the Q clients that last access the federated learning system.

[0138] In some embodiments, the management node selects Q clients from the federated learning system according to a selection rule. For example, Q clients whose recent local model test accuracy is below a threshold are selected. For another example, Q clients are selected based on at least one of the client's region, country code, address, user profile, model, and / or version information. For another example, Q clients are selected based on partial or complete parameter information of the local model and / or feature graph information of the local model.

[0139] In some embodiments, after selecting Q clients, the management node and the access node will perform system initialization operations and notify the selected Q clients to perform initialization operations. The initialization operations may include one or more of the following operations: registering a device, performing a communication handshake, requesting resources, or resetting a status.

[0140] Step 402: The first client performs a sparsification process on the first data set to obtain a second data set, where the first data set is used to describe the first AI model, and the amount of data in the second data set is smaller than that in the first data set.

[0141] The first data set may be a matrix, a vector, an array, etc. For the second data set obtained by sparsifying the first data set, the number of data included in the second data set may be less than or equal to the number of data included in the first data set, or the dimension of the second data set may be less than or equal to the dimension of the first data set.

[0142] The first data set includes multiple first data used to describe a first AI model. Optionally, the multiple first data are multiple network parameters of the first AI model, or the multiple first data are multiple network parameter update values ​​of the first AI model, where the multiple network parameter update values ​​of the first AI model are the differences between the multiple network parameters of the first AI model and the multiple network parameters of the second AI model.

[0143] In step 402, the following two thinning processing methods are listed.

[0144] Method 1: select m first data satisfying a first condition from the first data set, where m is an integer greater than or equal to 1; and obtain a second data set based on the m first data.

[0145] In some embodiments, the m first data include the m first data with the largest absolute value in the first data set. Or,

[0146] The m first data include m first data in the first data set that are within a first range. Optionally, the first range may be a range greater than a first threshold, or the first range may be a range greater than the first threshold or less than a second threshold, and the second threshold is less than the first threshold.

[0147] For example, assuming the first threshold is 0.3 and the second threshold is -0.2, the first range is a range greater than 0.3, that is, the first data greater than 0.3 is selected from the first data set. Alternatively, the first range is a range greater than 0.3 or less than -0.2, that is, the first data greater than 0.3 and the first data less than -0.2 are selected from the first data set.

[0148] Regarding the second data set obtained by the first method, several examples of the second data set are listed below.

[0149] Example 1: The second data set includes the m first data and the sequence numbers of the m first data in the first data set.

[0150] The m first data are data with significant characteristics in the first data set. The distribution characteristics and / or trends of the first data set can be expressed through the m first data and the serial numbers of the m first data, so the first data set can be replaced by the second data set, and the second data set can be used for grouping in subsequent grouping operations.

[0151] For the second data set in Example 1, since the second data set includes m first data in the first data set, the number of first data included in the second data set is smaller than the number of first data included in the first data set, so that the data volume of the second data set is smaller than the data volume of the first data set.

[0152] For example, suppose a first data set includes 10,000 first data items. 500 of these items are selected from the first data set, i.e., m = 500. The second data set includes these 500 first data items and their sequence numbers in the first data set. Thus, the number of first data items included in the second data set is much smaller than that in the first data set, resulting in a smaller data size in the second data set than in the first data set.

[0153] Example 2: The second data set includes m first bits corresponding to the m first data and second bits corresponding to data other than the m first data in the first data set.

[0154] Assume that the number of first data included in the first data set is x, x is an integer greater than m, the data in the first data set other than the m first data include xm first data, and the second data set includes m first bits corresponding to the m first data and xm second bits corresponding to the xm first data.

[0155] The first bit may be bit 1 and the second bit may be bit 0; or the first bit may be bit 0 and the second bit may be bit 1.

[0156] For the second data set in Example 2, the number of bits included in the second data set is equal to the number of first data included in the first data set, that is, the second data set includes x bits. However, because the second data set includes x bits and the first data set includes x first data, the amount of data in the second data set is smaller than that in the first data set.

[0157] Example 3: The second data set includes m second data corresponding to the m first data and the serial number of each second data, the serial number of each second data is the serial number of the first data corresponding to each second data in the first data set, and the bit width of each first data is greater than the bit width of the second data corresponding to each first data.

[0158] In Example 3, the m first data are quantized to obtain m second data, and the m first data correspond to the m second data one-to-one, so that the bit width of each first data is larger than the bit width of the second data corresponding to each first data.

[0159] For example, the m first data are m characters. Assume that the bit width of each of the m first data is 32 bits, that is, each first data is 32 bits long. Quantize the m first data to obtain m integer data with a length of 8 bits. That is, the second data set includes the m integer data, so that the data size of the second data set is smaller than that of the first data set.

[0160] In some embodiments, part of the m first data may also be quantized to obtain second data corresponding to the part of the first data, so the second data set includes the second data corresponding to the part of the first data and the data in the m first data other than the part of the first data.

[0161] In a second method, the first data set is a matrix or a vector. A fifth data set is generated, wherein the dimension of the fifth data set is smaller than the dimension of the first data set. At least one operation is performed on the fifth data set and the first data set to obtain a second data set.

[0162] Optionally, a fifth data set is randomly generated, the fifth data set is a matrix or a vector, each data included in the fifth data set is less than 1 and greater than -1, each data in the fifth data set is a decimal or 0, etc. The fifth data set is multiplied by the first data set at least once to obtain a second data set.

[0163] Optionally, the operation of multiplying the fifth data set by the first data set at least once may include: multiplying the fifth data set by the first data set to obtain a first intermediate set; then multiplying the fifth data set by the first intermediate set to obtain a second intermediate set; then multiplying the fifth data set by the second intermediate set to obtain a third intermediate set. The above process is repeated until the obtained intermediate set meets a specified condition, and the intermediate set that meets the specified condition is used as the second data set.

[0164] Because each data point in the fifth data set is less than 1 and greater than -1, the data points in the intermediate set obtained with each calculation gradually decrease. For each intermediate set obtained with each calculation, if the ratio of the number of data points in the intermediate set that are less than the first threshold to the total number of data points in the intermediate set is less than the ratio threshold, the intermediate set is determined to meet the specified condition.

[0165] The dimension of the second data set is equal to the dimension of the fifth data set, and the dimension of the second data set is smaller than the dimension of the first data set, so that the data volume of the second data set is smaller than the data volume of the first data set.

[0166] In some embodiments, the first AI model includes a multi-layer network, and the first data set includes the first data of the multi-layer network. When the first data set is subjected to sparse processing, the first data of the a-layer network can be selected from the first data set, where a is an integer greater than or equal to 1. The first data of the a-layer network in the first data set is subjected to sparse processing to obtain a second data set. The second data set may include the data after the sparse processing of the first data of the a-layer network and the first data of networks other than the a-layer network in the first data set. Optionally, the a-layer network may be the last few layers of the first AI model or a specific network.

[0167] For example, the first AI model is a deep neural network (DNN), and the specific network is the last fully connected layer network of the DNN, or part or all of the batch normalization (BN) layer network of the DNN.

[0168] In some embodiments, a sparse matrix representation method may be used to store the second data set after the sparsification process. The sparse matrix representation method includes one or more of the following methods: coordinate matrix (COO), compressed sparse row matrix (CSR), compressed sparse column matrix (CSC), block sparse row matrix (BSR), linked list matrix (LIL), and diagonal matrix (DIA).

[0169] Step 403: The first client sends a second data set to a first access node, where the first access node is the access node to which the first client accesses.

[0170] Since the data volume of the second data set is smaller than that of the first data set, the network resources occupied by sending the second data set can be reduced.

[0171] The number of clients connected to the first access node is n, where n is an integer greater than 1. For the other n-1 clients connected to the first access node, the n-1 clients, like the first client, execute steps 401-403 to send the second data set to the first access node.

[0172] Step 404: The first access node receives n second data sets, where the n second data sets correspond to the n clients.

[0173] The n second data sets are second data sets sent by each of the n clients.

[0174] Step 405: The first access node performs a clustering operation on the n second data sets to obtain a plurality of first clusters.

[0175] In step 405, the first access node performs a clustering operation on the n second data sets using a clustering algorithm to obtain multiple first clusters. Optionally, the clustering algorithm may include a combination of one or more of the following algorithms: k-means clustering algorithm (k-Means), optimized k-means clustering algorithm (k-Means++), bisection k-means clustering algorithm (bi-kmeans), density-based spatial clustering of applications with noise (DBSCAN), ordering points to identify the clustering structure (OPTICS), hierarchical clustering algorithm, or Coreset algorithm.

[0176] For each first cluster, the first cluster includes one or more data sets, which may be part of the n second data sets, or may be data sets obtained based on the n second data sets. There is no intersection between any two first clusters.

[0177] In some embodiments, the first access node further obtains a weight corresponding to each first cluster. Optionally, the weight corresponding to the first cluster is obtained based on the number of data sets included in the first cluster. For example, the weight corresponding to the first cluster is equal to the number of data sets included in the first cluster, or the weight corresponding to the first cluster is the product of the number of data sets included in the first cluster and a specified coefficient.

[0178] In some embodiments, the first access node directly performs a clustering operation on the n second data sets to obtain multiple first clusters.

[0179] For each second data set, the second data set includes m first data and sequence numbers of the m first data. The first access node performs a clustering operation on the n second data sets based on the m first data and sequence numbers of the m first data included in each second data set to obtain multiple first clusters, each of which includes a portion of the second data sets in the n second data sets. Or,

[0180] The second data set includes m second data and sequence numbers of the m second data. The first access node performs a clustering operation on the n second data sets based on the m second data and sequence numbers of the m second data included in each second data set to obtain multiple first clusters, each of which includes a portion of the second data sets in the n second data sets. Or,

[0181] The second data set includes x bits, and the first access node performs a clustering operation on the n second data sets based on the x bits included in each second data set to obtain multiple first clusters, each of which includes a portion of the second data sets in the n second data sets. Or,

[0182] The second data set is a data set obtained using the second method described above, and the dimension of the second data set is smaller than the dimension of the first data set. The first access node performs a clustering operation on the n second data sets based on the data included in each second data set to obtain multiple first clusters, each of which includes a portion of the second data sets in the n second data sets.

[0183] In some embodiments, for each of the n second data sets, the second data set includes m first data and sequence numbers of the m first data sets. The first access node obtains n intermediate data sets corresponding to the n second data sets based on xm default data, the m first data included in each of the n second data sets, and the sequence numbers of the m first data. For each second data set, the intermediate data set corresponding to the second data set includes the m first data in the second data set and the xm default data. A clustering operation is performed on the n intermediate data sets to obtain multiple first clusters, each of which includes a portion of the intermediate data sets from the n intermediate data sets. Optionally, the default data can be 0, 1, or 2, etc.

[0184] For each intermediate data set, the intermediate data set includes x data, that is, the number of data included in the intermediate data set is equal to the number of first data included in the first data set, and both are equal to x.

[0185] For any second data set, the process of obtaining the intermediate data set corresponding to the second data set can be: based on the serial numbers of the m first data included in the second data set, the m first data are filled into the intermediate data set, and then xm default data are filled into the intermediate data set so that the intermediate data set includes x data.

[0186] Step 406: The first access node sends information of multiple first clusters to the management node.

[0187] In step 406, the first access node may further send weights corresponding to the multiple first clusters to the management node.

[0188] For any first cluster among the plurality of first clusters, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies the second similarity condition with the center of the first cluster. Therefore, the total number of data sets included in the information of the plurality of first clusters is less than n.

[0189] In some embodiments, the second similarity condition refers to selecting at least one data set from the first cluster whose similarity with the center of the first cluster is the greatest, or selecting at least one data set from the first cluster whose similarity with the center of the first cluster exceeds a second similarity threshold.

[0190] The center of the first cluster may be a certain data set included in the first cluster, or the center of the first cluster may be a new data set obtained based on the data sets included in the first cluster.

[0191] For each first cluster, the first access node calculates the similarity between each data set included in the first cluster and the center of the first cluster, and selects at least one data set with the greatest similarity to the center of the first cluster as the information of the first cluster.

[0192] For each data set, the similarity between the data set and the center of the first cluster may include the distance between the data set and the center of the first cluster, or the cosine similarity between the data set and the center of the first cluster.

[0193] For multiple access nodes communicating with the management node, each access node except the first access node in the multiple access nodes sends information of multiple first clusters to the management node by executing the above process of steps 404 to 406.

[0194] Step 407: The management node receives information of a first clusters, where the information of the a first clusters comes from part or all of the multiple access nodes, and obtains multiple global cluster centers based on the information of the a first clusters, where the multiple global cluster centers are used to indicate multiple groups.

[0195] For each first cluster's information, the first cluster's information includes at least one data set within the first cluster. In step 407, the management node performs a clustering operation on the data sets included in the a first cluster's information to obtain multiple global clusters, obtains centers of the multiple global clusters, and uses the centers of the multiple global clusters as multiple global cluster centers.

[0196] In some embodiments, the management node further receives weights corresponding to multiple first clusters sent by each access node. Based on the weights corresponding to the a first clusters, a clustering operation is performed on the data set included in the information of the a first clusters to obtain multiple global clusters.

[0197] In some embodiments, the management node performs a clustering operation on the data set included in each received first cluster information using a clustering algorithm to obtain multiple global clusters.

[0198] Since the data set included in the information of each first cluster is a partial data set in each first cluster, the total number of data sets included in the information of each first cluster received is less than Q. In this way, the total data volume of the data sets included in the information of each first cluster received is less than the total data volume of the first data sets of Q clients, thereby reducing the computational amount of the management node for clustering operations and improving the efficiency of obtaining the multiple global cluster centers.

[0199] Step 408: The management node sends the multiple global cluster centers to some or all of the multiple access nodes.

[0200] In some embodiments, the management node may send the multiple global cluster centers to some access nodes, and may send some of the multiple global cluster centers to some access nodes. For example, for the first access node, the first access node sends information of c first clusters, where c is an integer greater than or equal to 1, and the multiple global cluster centers include c global cluster centers corresponding to the c first clusters, and the c global cluster centers can be sent to the first access node. In other words, the first access node may also obtain information of a first cluster and send information of a first cluster to the management node. The management node may also obtain a global cluster center and send a global cluster center to the first access node, and the one global cluster center corresponds to a group. The first access node determines the grouping of the client based on the one global cluster center.

[0201] After receiving the multiple global cluster centers, each access node determines the group to which the client connected to each access node belongs in step 409. Step 409 is described using the first access node as an example. By replacing the first access node with another access node, the implementation process of the other access nodes can be obtained.

[0202] Step 409: The first access node receives a plurality of global cluster centers, and determines a group to which each of the n clients belongs based on a part or all of the plurality of global cluster centers.

[0203] In some embodiments, in step 409, the n data sets are clustered based on the multiple global cluster centers to obtain multiple second clusters. The multiple global cluster centers correspond to the multiple second clusters. For each second cluster, the second cluster includes a portion of the n data sets, and the global cluster center corresponding to the second cluster satisfies a first similarity condition. The client corresponding to each data set included in the second cluster is determined as a group.

[0204] In some embodiments, the first access node sends information about c first clusters. The first access node may receive c global cluster centers, or may receive more than c global cluster centers. In step 409, the n data sets are clustered based on the c global cluster centers to obtain c second clusters. The c global cluster centers correspond to the c second clusters. For each second cluster, the second cluster includes a portion of the n data sets, and the global cluster center corresponding to the second cluster satisfies the first similarity condition. The client corresponding to each data set included in the second cluster is determined as a group.

[0205] For each of the n data sets, based on the data set and each global cluster center, the similarity between the data set and each global cluster center is obtained, and the global cluster center with the greatest similarity to the data set is selected, or a global cluster center with a similarity to the data set exceeding a first similarity threshold is selected. If the selected global cluster center and the data set meet the first similarity condition, the data set is added to the second cluster corresponding to the selected global cluster center. In this way, the n data sets are clustered into multiple second clusters.

[0206] Step 410: The first access node sends the group identifier of the group to which each client belongs to each client.

[0207] Each client includes the first client, and each client performs the following operation of step 411 like the first client.

[0208] Step 411: The first client receives the group identifier of the group to which the first client belongs, and sends a first training request to the first access node. The first training request includes a first data set and the group identifier of the first group to which the first client belongs.

[0209] For the other n-1 clients connected to the first access node, the n-1 clients, like the first client, also send a first training request to the first access node.

[0210] Step 412: The first access node receives the first training request sent by each client, and acquires a third data set corresponding to the first group based on the first data set and group identifier included in each received first training request.

[0211] In step 412, the first access node receives the first training request sent by each client, and obtains the first data set of each client and the group identifier of the group to which each client belongs from the first training request sent by each client. Based on the group identifier of the first group, the first data set of each client in the first group is obtained, where the first group is one of the multiple groups. A weighted average is performed based on the weight of each client and the first data set of each client to obtain a third data set corresponding to the first group. The third data set includes multiple network parameters or multiple network parameter update values. The multiple network parameters included in the third data set are obtained by taking a weighted average of the multiple network parameters included in the first data set of each client in the first group, or the multiple network parameters included in the third data set are obtained by taking a weighted average of the multiple network parameter update values ​​included in the first data set of each client in the first group. According to the above process, the first access node can obtain the third data set corresponding to each other group.

[0212] The above-mentioned weighted average method is only an implementation example for obtaining the third data set. Of course, other methods can also be used to obtain the first data set, which will not be listed here one by one.

[0213] Step 413: The first access node sends a second training request to the management node, where the second training request includes a third data set corresponding to the first group.

[0214] The data volume of the third data set corresponding to the first group is smaller than the total data volume of the first data set of each client in the first group, so the first access node sends the third data set corresponding to the first group to the management node, which can reduce the occupation of network resources.

[0215] In some embodiments, the second training request further includes a group identifier of the first group, and / or a third data set corresponding to each of the other groups and a group identifier of each of the other groups.

[0216] For other access nodes communicating with the management node, like the first access node, the other access nodes send the third data set corresponding to each group and the group identifier of each group to the management node.

[0217] Step 414: The management node receives the second training request sent by each access node, and obtains the group model corresponding to the first group based on the third data set corresponding to the first group included in each received second training request.

[0218] In step 414, the management node obtains a third data set corresponding to the first group from each received second training request based on the group identifier of the first group, thereby obtaining multiple third data sets corresponding to multiple first groups. The multiple third data sets are averaged to obtain a fourth data set corresponding to the first group. The fourth data set includes multiple network parameters or multiple network parameter update values. The multiple network parameters in the fourth data set are obtained by averaging the multiple network parameters in the multiple third data sets, or the multiple network parameter update values ​​in the fourth data set are obtained by averaging the multiple network parameter update values ​​in the multiple third data sets. The fourth data set is used to describe the group model corresponding to the first group. According to the above process, the management node can obtain the fourth data set of the group model corresponding to each other group.

[0219] In some embodiments, averaging the multiple third data sets is only one way to obtain the fourth data set corresponding to the first group. Of course, other methods can also be used to process the multiple third data sets to obtain the fourth data set corresponding to the first group, which will not be listed one by one here.

[0220] Step 415: The management node sends a fourth data set to each access node, where the fourth data set is used to describe the group model corresponding to the first group.

[0221] The management node may also obtain fourth data sets corresponding to other groups, and send the fourth data sets corresponding to other groups to each access node.

[0222] Step 416: The first access node receives the fourth data set, and sends the fourth data set to the clients belonging to the first group among the n clients.

[0223] The first client belongs to the first group and receives the fourth data set. When the first client receives the fourth data set, the following operations are performed.

[0224] Step 417: The first client receives the fourth data set and adjusts the first AI model based on the fourth data set.

[0225] The adjusted first AI model is identical or similar to the group model corresponding to the first group.

[0226] In some embodiments, after receiving the first data set of each client in the first group, the first access node may send the first data set of each client in the first group to the management node. The management node receives the first data set of each client in the first group sent by each access node, thereby obtaining the first data set sent by each client in the first group in the federated learning system, and performs a weighted average based on the first data set of each client in the first group and the weight of each client in the first group to obtain a group model corresponding to the first group. The management node then sends a fourth data set to each access node, where the fourth data set is used to describe the group model corresponding to the first group.

[0227] In step 417, after adjusting the first AI model based on the fourth data set, the first client trains the adjusted first AI model based on the local training data to obtain a new AI model. This new AI model is used as the first AI model, and a second round of training of the group model corresponding to each group is performed starting from step 411 above. After the second round of training of the group model corresponding to each group is completed, a third round of training of the group model corresponding to each group is performed starting from step 411 above. After multiple rounds of training, a group model corresponding to each group is obtained, and any client includes the group model corresponding to the group to which the client belongs.

[0228] That is, referring to Figure 5 , the first system global model is initialized, and then Q clients are grouped through steps 401-410 described above. After grouping, the i-th round of training of the group model corresponding to each group is performed through steps 411-417 described above, where i = 1, 2, 3, ..., and the training ends when i reaches a threshold T. That is, after the Q clients are grouped once, T-1 rounds of group model training are performed.

[0229] In some embodiments, referring to FIG6 , when performing the i-th round of training of the group model corresponding to each group, the access node receives the first data set sent by the client, calculates the similarity between the first data set sent by the client and the global cluster center corresponding to the group to which the client belongs, and when the similarity is lower than the similarity threshold, the Q clients are regrouped through the above steps 401-410, and after grouping, the i+1-th round of training of the group model corresponding to each group is performed through the above steps 411-417.

[0230] In some embodiments, referring to FIG7 , a first system global model is first initialized, and then multiple clients are grouped for the i-th time through steps 401-410. After the i-th grouping, the grouping model corresponding to each group is trained through steps 411-417. The training process ends when i reaches a threshold value T. In other words, the process of steps 401-417 is executed T-1 times in total.

[0231] For a second client other than the Q clients in the federated learning system, when the AI ​​model in the second client needs to be trained, the second AI model is sent to the second client. After the second client receives the second AI model, the group to which the second client belongs is determined according to steps 401-410 above, and a data set describing the most recently trained group model corresponding to the group is sent to the second client. The second client receives this data set, adjusts its local AI model based on it, and then continues to loop through steps 411-417 above.

[0232] In an embodiment of the present application, after a first client trains a first AI model based on local training data, it performs a sparse processing on the first data set used to describe the first AI model to obtain a second data set with a smaller data size than the first data set, and then sends the second data set to the first access node. The first access node receives n second data sets sent by n clients, performs a clustering operation on the n second data sets, and obtains information about multiple first clusters. Since the data size of the second data sets is reduced, the storage resources required by the first access node to store the second data sets are also reduced. The other access nodes in the federated learning system are similar to the first access node. In this way, each access node performs a clustering operation on multiple second data sets sent by multiple clients connected to it in parallel. The number of second data sets that each access node needs to cluster is significantly reduced, and the data size of the second data sets that need to be clustered is also reduced, thereby significantly improving the efficiency of the clustering operation. Each access node sends information about the multiple first clusters obtained by clustering to the management node. The information about the first cluster includes at least one data set with the greatest similarity to the center of the first cluster. The number of data sets included in the first cluster information is smaller than the number of data sets included in the second cluster. This reduces the number of data sets that the management node needs to perform clustering operations on, improving the efficiency of obtaining multiple global cluster centers. As a result, each access node can quickly obtain these multiple global cluster centers and determine the group to which each client accessing the node belongs based on these multiple global cluster centers, thereby improving grouping efficiency.

[0233] Referring to Figure 8, an embodiment of the present application provides a grouping method 800, which is applied to the federated learning system 100 shown in Figure 1. In the method 800, the client performs model training to obtain a first AI model, obtains a first data set for describing the first AI model, performs sparse processing on the first data set to obtain a second data set, and sends the second data set. For each client connected to the access node, the access node obtains multiple first clusters based on the second data set of each client, and sends the centers of the multiple first clusters to the management node. The management node obtains multiple global cluster centers based on the centers of the multiple first clusters sent by each access node. The multiple global cluster centers correspond one-to-one to multiple groups and send the multiple global cluster centers to each access node. The access point determines the group to which the client belongs based on the multiple global information. The method 800 includes the following process.

[0234] Steps 801 - 804 are respectively the same as steps 401 - 404 in the method 400 shown in FIG. 4 and will not be described in detail here.

[0235] Step 805: The first access node receives the multiple global cluster centers sent by the management node, and performs clustering operations on the n second data sets based on the multiple global cluster centers to obtain multiple first clusters.

[0236] The management node may first initialize multiple global cluster centers, and send the multiple global cluster centers to multiple access nodes communicating with the management node, where the multiple access nodes include the first access node.

[0237] In step 805, the first access node receives a plurality of global cluster centers sent by the management node, and performs clustering operations on the n second data sets using a clustering algorithm based on the plurality of global cluster centers to obtain a plurality of first clusters.

[0238] For each first cluster, the first cluster includes one or more data sets, which may be part of the n second data sets, or the one or more data sets may be data sets obtained based on the n second data sets.

[0239] In some embodiments, the first access node further obtains a weight corresponding to each first cluster. Optionally, the weight corresponding to the first cluster is obtained based on the number of data sets included in the first cluster. For example, the weight corresponding to the first cluster is equal to the number of data sets included in the first cluster, or the weight corresponding to the first cluster is the product of the number of data sets included in the first cluster and a specified coefficient.

[0240] In some embodiments, the first access node directly performs a clustering operation on the n second data sets based on the multiple global cluster centers to obtain multiple first clusters.

[0241] For each second data set, the second data set includes m first data and sequence numbers of the m first data. The first access node performs a clustering operation on the n second data sets based on the multiple global cluster centers, the m first data included in each second data set, and the sequence numbers of the m first data, to obtain multiple first clusters, each of which includes a portion of the second data sets in the n second data sets. Or,

[0242] The second data set includes m second data and sequence numbers of the m second data. The first access node performs a clustering operation on the n second data sets based on the multiple global cluster centers, the m second data included in each second data set, and the sequence numbers of the m second data, to obtain multiple first clusters, each of which includes a portion of the second data sets in the n second data sets. Or,

[0243] The second data set includes x bits, and the first access node performs a clustering operation on the n second data sets based on the multiple global cluster centers and the x bits included in each second data set to obtain multiple first clusters, each of which includes a portion of the second data sets in the n second data sets. Or,

[0244] The second data set is a data set obtained using the second method described above, and the dimension of the second data set is smaller than the dimension of the first data set. The first access node performs a clustering operation on the n second data sets based on the multiple global cluster centers and the data included in each second data set to obtain multiple first clusters, each of which includes a portion of the second data sets in the n second data sets.

[0245] In some embodiments, for each of the n second data sets, the second data set includes m first data and the sequence numbers of the m first data sets. The first access node obtains n intermediate data sets corresponding to the n second data sets based on the xm default data, the m first data included in each of the n second data sets, and the sequence numbers of the m first data. For each second data set, the intermediate data set corresponding to the second data set includes the m first data in the second data set and the xm default data. Based on the multiple global cluster centers, a clustering operation is performed on the n intermediate data sets to obtain multiple first clusters, each of which includes a portion of the intermediate data sets in the n intermediate data sets. Optionally, the default data can be 0, 1, or 2, etc.

[0246] Step 806: The first access node sends the centers of multiple first clusters to the management node.

[0247] In some embodiments, the first access node further sends weights corresponding to the multiple first clusters to the management node.

[0248] For multiple access nodes communicating with the management node, each access node except the first access node in the multiple access nodes sends the centers of the multiple first clusters to the management node by executing the process of steps 804 to 806 above.

[0249] Step 807: The management node receives a centers of the first clusters, where the centers of the first clusters come from part or all of the multiple access nodes, and obtains multiple global cluster centers based on the centers of the first clusters.

[0250] In step 807, for the center of the i-th first cluster sent by each access node, i=1, 2, 3, ..., each access node is the access node that sent the center of the first cluster. The management node obtains the i-th global cluster center based on the weight corresponding to the i-th first cluster sent by each access node and the center of the i-th first cluster sent by each access node. Optionally, during implementation:

[0251] The management node performs weighted averaging, linear transformation and / or kernel function transformation based on the weight of the i-th first cluster sent by each access node and the center of the i-th first cluster sent by each access node to obtain the i-th global cluster center.

[0252] Since the data volume of the center of each first cluster is small, the total data volume of the received center of each first cluster is smaller than the total data volume of the Q data sets, thereby improving the efficiency of obtaining the multiple global cluster centers.

[0253] Step 808: The management node sends the multiple global cluster centers to some or all of the multiple access nodes.

[0254] For each access node, after receiving the multiple first global centers, each access node determines whether the multiple first cluster centers meet the convergence condition through the following step 809.

[0255] Step 809: The first access node receives the multiple global cluster centers and determines whether the multiple global cluster centers meet the convergence conditions. If the multiple global cluster centers do not meet the convergence conditions, return to execute step 805; if the multiple global cluster centers meet the convergence conditions, return to execute step 810.

[0256] In some embodiments, the convergence condition may mean that for the multiple global cluster centers obtained last time, the difference between the multiple global cluster centers obtained currently and the multiple global cluster centers obtained last time is less than a difference threshold.

[0257] Steps 810 - 818 are respectively the same as steps 409 - 417 in the method 400 shown in FIG. 4 and will not be described in detail here.

[0258] In an embodiment of the present application, after the first client obtains the first AI model based on local training data training, it performs sparse processing on the first data set used to describe the first AI model to obtain a second data set with a data volume smaller than that of the first data set, and sends the second data set to the first access node. The first access node receives n second data sets sent by n clients and multiple global clustering centers sent by the receiving management node, and performs clustering operations on the n second data sets based on the multiple global clustering centers to obtain multiple first cluster centers. Since the data volume of the second data set is reduced, the storage resources required by the first access node to store the second data set are also reduced. The other access nodes in the federated learning system are the same as the first access node. In this way, each access node performs clustering operations on multiple second data sets sent by multiple clients connected to itself in parallel. The number of second data sets that each access node needs to cluster is greatly reduced, and the data volume of the second data sets that need to be clustered is reduced, thereby greatly improving the efficiency of the clustering operation. Each access node sends the centers of multiple first clusters obtained through clustering to the management node. The data size of the centers of the first clusters is very small, which reduces the amount of data required for clustering calculations by the management node and improves the efficiency of obtaining multiple global cluster centers. As a result, each access node can quickly obtain these multiple global cluster centers and, based on these multiple global cluster centers, determine the group to which each client connected to the node belongs, improving grouping efficiency.

[0259] Referring to Figure 9, an embodiment of the present application provides a grouping method 900, which is applied to the federated learning system 100 shown in Figure 1. In the method 900, the client performs model training to obtain a first AI model, obtains a first data set for describing the first AI model, and sends the first data set. For each client connected to the access node, the access node obtains multiple first clusters based on the first data set of each client, and sends information about the multiple first clusters to the management node. The management node obtains multiple global cluster centers based on the information about the multiple first clusters sent by each access node. The multiple global cluster centers correspond one-to-one to multiple groups, and sends the multiple global cluster centers to each access node. The access point determines the group to which the client belongs based on the multiple global cluster centers. The method 900 includes the following process.

[0260] Step 901: is the same as step 401 of the method 400 shown in FIG4 , and will not be described in detail here.

[0261] Step 902: The first client sends a first data set to a first access node, where the first data set is used to describe a first AI model. The first access node is the access node to which the first client accesses.

[0262] The number of clients connected to the first access node is n, where n is an integer greater than 1. For the other n-1 clients connected to the first access node, the n-1 clients, like the first client, execute steps 901-902 to send the first data set to the first access node.

[0263] Step 903: The first access node receives n first data sets, where the n first data sets correspond to the n clients.

[0264] The n first data sets are first data sets sent by each of the n clients.

[0265] Step 904: The first access node performs a clustering operation on the n first data sets to obtain a plurality of first clusters.

[0266] In some embodiments, in step 904, the first access node performs a clustering operation on the n first data sets using a clustering algorithm to obtain a plurality of first clusters.

[0267] In some embodiments, in step 904, the first access node performs a sparse processing on the n first data sets to obtain n second data sets, where the n first data sets correspond one-to-one to the n second data sets. A clustering algorithm is used to perform a clustering operation on the n second data sets to obtain a plurality of first clusters.

[0268] For each first data set, the first access node performs a sparse processing on the first data set to obtain a second data set corresponding to the first data set. The detailed implementation process of the sparse processing performed by the first access node is described in step 402 of the method 400 shown in FIG4 , and will not be described in detail here.

[0269] Steps 905-907 are respectively the same as steps 406-408 of the method shown in FIG4 and will not be described in detail here.

[0270] Step 908: The first access node receives the multiple global cluster centers, and determines the group to which each of the n clients belongs based on the multiple global cluster centers.

[0271] In step 908, the n data sets are clustered based on the multiple global cluster centers to obtain multiple second clusters. The n data sets are the n first data sets, or the n data sets are the n second data sets, and the multiple global cluster centers correspond to the multiple second clusters. For each second cluster, the second cluster includes a portion of the n data sets, and the portion of the data set and the global cluster center corresponding to the second cluster meet the first similarity condition. The client corresponding to each data set included in the second cluster is determined as a group.

[0272] For each of the n data sets, based on the data set and each global cluster center, the similarity between the data set and each global cluster center is obtained, and the global cluster center with the greatest similarity to the data set is selected. If the selected global cluster center and the data set meet the first similarity condition, the data set is added to the second cluster corresponding to the selected global cluster center. In this way, the n data sets are clustered into multiple second clusters.

[0273] Step 909: The first access node obtains a third data set corresponding to the first group based on the first data set of each client in the first group, where the first group is any one of the multiple groups.

[0274] In step 909, the first access node determines the group to which each of the n clients belongs. For ease of explanation, for any group, referred to as the first group, the first data set for each client belonging to the first group is obtained. A weighted average is performed based on the weight of each client and the first data set for each client to obtain a third data set corresponding to the first group. The third data set includes multiple network parameters or multiple network parameter update values. Following this process, the first access node can obtain the third data set corresponding to each of the remaining groups.

[0275] The weighted average method is only one implementation method for obtaining the third data set corresponding to the first group. Of course, other implementation methods can also be used to obtain the third data set corresponding to the first group, which will not be listed here one by one.

[0276] Steps 910 - 914 are respectively the same as steps 413 - 417 of the method 400 shown in FIG. 4 and will not be described in detail here.

[0277] In an embodiment of the present application, after the first client obtains the first AI model based on local training data training, it sends the first data set used to describe the first AI model to the first access node. The first access node receives n first data sets sent by n clients, performs clustering operations on the n first data sets, and obtains information of multiple first clusters. The other access nodes in the federated learning system are the same as the first access node, so that each access node performs clustering operations on multiple first data sets sent by multiple clients connected to itself in parallel, and the number of first data sets that each access node needs to cluster is greatly reduced, thereby greatly improving the efficiency of clustering operations. Each access node sends the information of multiple first clusters obtained by clustering to the management node, which reduces the amount of data that the management node needs to cluster and improves the efficiency of obtaining multiple global cluster centers. As a result, each access node can quickly obtain the multiple global cluster centers, and determine the group to which each client connected to itself belongs based on the multiple global cluster centers, thereby improving the efficiency of grouping.

[0278] Referring to Figure 10, an embodiment of the present application provides a grouping device 1000, which is applied to a first access node. The first access node is any one of the multiple access nodes included in the federated learning system shown in Figure 1, Figure 2 or Figure 3. The federated learning system also includes a management node and n clients connected to the first access node, where n is an integer greater than 1. The device 1000 includes: a receiving unit 1001, a sending unit 1002 and a processing unit 1003.

[0279] The receiving unit 1001 is configured to receive n data sets from n clients;

[0280] The processing unit 1003 is configured to cluster the n data sets to obtain information of one or more first clusters;

[0281] A sending unit 1002 is configured to send information of one or more first clusters to a management node;

[0282] The receiving unit 1001 is further configured to receive one or more first global cluster centers sent by the management node, where the one or more first global cluster centers are obtained by the management node based on information of a first clusters, where the information of the a first clusters comes from some or all of the multiple access nodes, where a is an integer greater than 1, and the one or more first global cluster centers are used to indicate one or more groups;

[0283] The processing unit 1003 is configured to determine the group to which each client belongs based on part or all of the one or more first global cluster centers.

[0284] Optionally, the detailed implementation process of the receiving unit 1001 receiving n data sets can be found in step 404 of method 400 shown in FIG4 , step 804 of method 800 shown in FIG8 , or step 903 of method 900 shown in FIG9 , and will not be described in detail here.

[0285] Optionally, for the detailed implementation process of the sending unit 1002 sending the information of one or more first clusters, please refer to the relevant content of step 406 of method 400 shown in Figure 4, step 806 of method 800 shown in Figure 8, or step 905 of method 900 shown in Figure 9, which will not be described in detail here.

[0286] Optionally, the detailed implementation process of the receiving unit 1001 receiving one or more first global cluster centers can be found in step 409 of method 400 shown in FIG4 , step 809 of method 800 shown in FIG8 , or step 909 of method 900 shown in FIG9 , and will not be described in detail here.

[0287] Optionally, the detailed implementation process of the processing unit 1003 determining the group to which each client belongs based on the one or more first global clustering centers can be found in step 409 of method 400 shown in Figure 4, step 809 of method 800 shown in Figure 8, or step 909 of method 900 shown in Figure 9, which will not be described in detail here.

[0288] Optionally, the processing unit 1003 is configured to:

[0289] Clustering the n data sets based on the one or more first global cluster centers to obtain one or more second clusters, the one or more first global cluster centers corresponding to the one or more second clusters, and for each second cluster, the second cluster includes a portion of the n data sets, and the portion of the data set and the first global cluster center corresponding to the second cluster meet a first similarity condition;

[0290] The clients corresponding to each data set included in the second cluster are determined as a group.

[0291] Optionally, the processing unit 1003 clusters the n data sets based on the one or more first global clustering centers to obtain one or more second clusters, and determines the client corresponding to each data set included in the second cluster as a group. For the detailed implementation process, please refer to step 409 of method 400 shown in Figure 4, step 809 of method 800 shown in Figure 8, or step 909 of method 900 shown in Figure 9, and will not be described in detail here.

[0292] Optionally, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set and the center of the first cluster meet a second similarity condition.

[0293] Optionally, the information of the first cluster includes centers of the first cluster, and the one or more first global cluster centers do not meet a convergence condition;

[0294] The processing unit 1003 is further configured to cluster the n data sets based on the one or more first global cluster centers to obtain one or more third clusters;

[0295] The sending unit 1002 is further configured to send information of the one or more third clusters to the management node;

[0296] The receiving unit 1001 is further configured to receive one or more second global cluster centers sent by the management node, where the one or more second global cluster centers are used to indicate the one or more groups;

[0297] The processing unit 1003 is further configured to obtain the group to which each client belongs based on the one or more second global cluster centers when the one or more second global cluster centers meet a convergence condition.

[0298] Optionally, the n data sets are n second data sets from n clients. For each client, the second data set corresponding to the client is obtained by the client performing sparse processing on the first data set. The first data set is used to describe the first AI model obtained by the client performing model training. The amount of data in the second data set is smaller than that in the first data set.

[0299] Optionally, the first data set includes multiple first data used to describe the first AI model in the client, and the second data set includes m first data and serial numbers of the m first data in the first data set, where m is an integer greater than 1, and the m first data are the first data in the first data set that meet the first condition. The processing unit 1003 is also used to perform clustering operations based on the m first data and the serial numbers of the m first data included in each of the n second data sets to obtain information of one or more first clusters.

[0300] Optionally, the n data sets are n first data sets from the n clients. For each client, the first data set corresponding to the client is used to describe a first AI model in the client. The processing unit 1003 is further configured to:

[0301] Performing a sparsification process on n first data sets to obtain n second data sets, wherein the n first data sets correspond to the n second data sets, and for each first data set, the amount of data in the first data set is smaller than the amount of data in the second data set corresponding to the first data set;

[0302] A clustering operation is performed on the n second data sets to obtain information of the one or more first clusters.

[0303] Optionally, the receiving unit 1001 is further configured to receive multiple first data sets sent by multiple clients in a first group, where the n clients include multiple clients, the multiple first data sets are used to describe a first AI model in the multiple clients, and the multiple groups include the first group;

[0304] The sending unit 1002 is further configured to send a third data set to the management node, where the third data set is obtained based on the multiple first data sets;

[0305] The receiving unit 1001 is further configured to receive a fourth data set sent by the management node, where the fourth data set is used to describe a group model corresponding to the first group;

[0306] The sending unit 1002 is further configured to send a fourth data set to each client in the first group.

[0307] Optionally, the detailed implementation process of the sending unit 1002 sending the third data set refers to the relevant content of step 413 of method 400 shown in Figure 4, step 814 of method 800 shown in Figure 8 or step 910 of method 900 shown in Figure 9, and will not be described in detail here.

[0308] Optionally, the detailed implementation process of the receiving unit 1001 receiving the fourth data set refers to the relevant content of step 413 of method 400 shown in Figure 4, step 816 of method 800 shown in Figure 8 or step 912 of method 900 shown in Figure 9, and will not be described in detail here.

[0309] Optionally, for the detailed implementation process of sending the fourth data set by the sending unit 1002, please refer to the relevant content of step 414 of method 400 shown in Figure 4, step 817 of method 800 shown in Figure 8, or step 913 of method 900 shown in Figure 9, which will not be described in detail here.

[0310] In an embodiment of the present application, since the federated learning system includes multiple access nodes, the receiving unit of each access node receives the data set sent by the respective accessed client, and the processing unit of each access node performs clustering operations on the received data set in parallel to obtain information of one or more first clusters. The sending unit sends information of one or more first clusters to the management node. The amount of data of the information of the first cluster is relatively small, and the management node quickly obtains one or more first global cluster centers based on the information of the first cluster sent by some or all of the multiple access nodes. The processing unit determines the group to which each client belongs based on the one or more first global cluster centers, thereby improving the grouping efficiency.

[0311] Referring to FIG. 11 , an embodiment of the present application provides a grouping device 1100 . The device 1100 is applied to a management node included in the federated learning system shown in FIG. 1 , FIG. 2 , or FIG. 3 . The federated learning system further includes multiple access nodes and multiple clients connected to each access node. The multiple access nodes include a first access node. The device 1100 includes:

[0312] A receiving unit 1101 is configured to receive information of a first cluster, where the information of the a first cluster comes from some or all of the multiple access nodes, where a is an integer greater than 1, and the one or more first cluster information sent by the first access node is obtained by clustering n data sets, where n is an integer greater than 1, and the n data sets come from n clients connected to the first access node;

[0313] The processing unit 1102 is configured to obtain one or more first global cluster centers based on information of the a first clusters, where the one or more first global cluster centers are used to indicate one or more groups of clients.

[0314] The sending unit 1103 is used to send one or more first global cluster centers to some or all of the multiple access nodes, and the first access node is used to determine the group to which each of the n clients belongs based on some or all of the one or more first global cluster centers.

[0315] Optionally, for the detailed implementation process of the receiving unit 1101 receiving information of one or more first clusters, please refer to the relevant content of step 407 of method 400 shown in Figure 4, step 807 of method 800 shown in Figure 8, or step 906 of method 900 shown in Figure 9, which will not be described in detail here.

[0316] Optionally, the detailed implementation process of the processing unit 1102 obtaining one or more first global cluster centers can be found in step 407 of method 400 shown in Figure 4, step 807 of method 800 shown in Figure 8, or step 906 of method 900 shown in Figure 9, and will not be described in detail here.

[0317] Optionally, the detailed implementation process of the sending unit 1103 sending one or more first global clustering centers to each access node refers to the relevant content of step 408 of method 400 shown in Figure 4, step 808 of method 800 shown in Figure 8, or step 907 of method 900 shown in Figure 9, and will not be described in detail here.

[0318] Optionally, for each received first cluster, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set and the center of the first cluster satisfy a second similarity condition;

[0319] The processing unit 1102 is configured to perform a clustering operation on a data set including information of the a first clusters to obtain one or more first global clusters, wherein the one or more first global cluster centers include one or more centers of the first global clusters.

[0320] Optionally, the detailed implementation process of the processing unit 1102 obtaining one or more first global cluster centers can be found in step 407 of the method 400 shown in FIG4 or step 906 of the method 900 shown in FIG9 , which will not be described in detail here.

[0321] Optionally, for each received first cluster, the information of the first cluster includes a center of the first cluster;

[0322] The processing unit 1102 is configured to obtain one or more first global cluster centers based on the weights corresponding to the a first clusters and the centers of the a first clusters.

[0323] Optionally, the detailed implementation process of the processing unit 1102 obtaining one or more first global cluster centers can be found in the relevant content of step 807 of the method 800 shown in FIG8 , which will not be described in detail here.

[0324] Optionally, one or more first global cluster centers do not meet the convergence condition,

[0325] The receiving unit 1101 is further configured to receive information of b third clusters, where the information of the b third clusters comes from some or all of the multiple access nodes, where b is an integer greater than 1, and the third cluster sent by the first access node is obtained by clustering the n data sets by the first access node based on one or more first global cluster centers;

[0326] The processing unit 1102 is further configured to obtain one or more second global cluster centers based on information of the b third clusters;

[0327] The sending unit 1103 is further configured to send one or more second global cluster centers to some or all of the multiple access nodes, and the first access node is configured to determine the group to which each of the n clients belongs based on the one or more second global cluster centers.

[0328] Optionally, the receiving unit 1101 is further configured to receive multiple third data sets, where the multiple third data sets are from some or all of the multiple access nodes, where the third data set sent by the first access node is used to describe a first AI module in multiple clients belonging to a first group, where the n clients include the multiple clients, and the multiple groups include the first group;

[0329] The processing unit 1102 is further configured to obtain a grouping model corresponding to the first grouping based on the multiple third data sets;

[0330] The sending unit 1103 is further configured to send a fourth data set to some or all of the multiple access nodes, where the fourth data set is used to describe the grouping model.

[0331] Optionally, the detailed implementation process of the receiving unit 1101 receiving the third data set sent by each access node refers to the relevant content of step 414 of method 400 shown in Figure 4, step 815 of method 800 shown in Figure 8 or step 911 of method 900 shown in Figure 9, and will not be described in detail here.

[0332] Optionally, the detailed implementation process of the processing unit 1102 obtaining the group model corresponding to the first group based on the third data set sent by each access node can be found in step 414 of method 400 shown in Figure 4, step 815 of method 800 shown in Figure 8, or step 911 of method 900 shown in Figure 9, which will not be described in detail here.

[0333] Optionally, the detailed implementation process of the sending unit 1103 sending the fourth data set to each access node refers to the relevant content of step 415 of method 400 shown in Figure 4, step 816 of method 800 shown in Figure 8, or step 912 of method 900 shown in Figure 9, and will not be described in detail here.

[0334] In an embodiment of the present application, since the federated learning system includes multiple access nodes, each access node receives a data set sent by a respective accessed client, and each access node performs clustering operations on the received data set in parallel to obtain information of one or more first clusters. The receiving unit receives information of one or more first clusters sent by each access node. The amount of information is small, and the processing unit quickly obtains one or more first global cluster centers based on the information of the first cluster sent by each access node. The sending unit sends the one or more first global cluster centers to each access node, and the first access node determines the group to which each client belongs based on the one or more first global cluster centers, thereby improving the grouping efficiency.

[0335] Referring to FIG. 12 , an embodiment of the present application provides a grouping device 1200 . The device 1200 applies a client in the system shown in FIG. 1 , FIG. 2 , or FIG. 3 . The device 1200 stores training data and includes:

[0336] The processing unit 1201 is configured to perform model training based on the training data to obtain a first artificial intelligence (AI) model.

[0337] The processing unit 1201 is further configured to perform a sparsification process on the first data set to obtain a second data set, where the first data set is used to describe the first AI model, and the amount of data in the second data set is smaller than that in the first data set;

[0338] The sending unit 1202 is configured to send a second data set, where the second data set is used to request determination of the group to which the apparatus 1200 belongs.

[0339] Optionally, the processing unit 1201 performs model training based on the training data, and the detailed implementation process of obtaining the first artificial intelligence AI model can be found in step 401 of method 400 shown in Figure 4 or step 801 of method 800 shown in Figure 8, which will not be described in detail here.

[0340] Optionally, the detailed implementation process of the processing unit 1201 performing the sparse processing on the first data set to obtain the second data set can be found in step 402 of the method 400 shown in FIG4 or step 802 of the method 800 shown in FIG8 , and will not be described in detail here.

[0341] Optionally, the detailed implementation process of the sending unit 1202 sending the second data set refers to the relevant content of step 403 of the method 400 shown in Figure 4 or step 803 of the method 800 shown in Figure 8, which will not be described in detail here.

[0342] Optionally, the first data set includes multiple first data for describing the first AI model, and the processing unit 1201 is configured to:

[0343] Selecting m first data satisfying a first condition from the first data set, where m is an integer greater than or equal to 1;

[0344] A second data set is obtained based on the m first data.

[0345] Optionally, the detailed implementation process of the processing unit 1201 selecting m first data that meet the first condition from the first data set can be found in step 402 of method 400 shown in FIG4 or step 802 of method 800 shown in FIG8 , which will not be described in detail here.

[0346] Optionally, the detailed implementation process of the processing unit 1201 acquiring the second data set based on the m first data can be found in step 402 of the method 400 shown in FIG4 or step 802 of the method 800 shown in FIG8 , and will not be described in detail here.

[0347] Optionally, the first condition includes that the m first data are the m data with the largest absolute values ​​in the first data set, or the m first data are the m data in the first data set whose values ​​are within a first range.

[0348] Optionally, the second data set includes m first data and m sequence numbers of the first data in the first data set; or,

[0349] The second data set includes m first bits and xm second bits, where x is the number of first data included in the first data set, x is an integer greater than m, the m first bits correspond to the m first data, the xm second bits correspond to the xm first data, and the xm first data are the data in the first data set other than the m first data.

[0350] Optionally, the processing unit 1201 is configured to:

[0351] The m first data are quantized to obtain m second data, the m first data correspond to the m second data, the bit width of each first data is greater than the bit width of the second data corresponding to each first data, and the second data set includes the m second data and the serial number of the first data corresponding to each second data in the first data set.

[0352] Optionally, the detailed implementation process of the processing unit 1201 performing quantization processing on the m first data to obtain the m second data can be found in step 402 of the method 400 shown in FIG4 or step 802 of the method 800 shown in FIG8 , which will not be described in detail here.

[0353] Optionally, the type of the first data set is a matrix or a vector, and the processing unit 1201 is configured to:

[0354] Obtaining a fifth data set, where the dimension of the fifth data set is smaller than the dimension of the first data set;

[0355] At least one operation is performed on the fifth data set and the first data set to obtain a second data set.

[0356] Optionally, the detailed implementation process of the processing unit 1201 obtaining the fifth data set refers to the relevant content of step 402 of the method 400 shown in Figure 4 or step 802 of the method 800 shown in Figure 8, which will not be described in detail here.

[0357] Optionally, the processing unit 1201 performs at least one operation on the fifth data set and the first data set to obtain the second data set. For the detailed implementation process, please refer to step 402 of method 400 shown in Figure 4 or step 802 of method 800 shown in Figure 8, which will not be described in detail here.

[0358] Optionally, the first data set is obtained based on network parameters of the second AI model and / or network parameters of the first AI model, and the first AI model is obtained by training the second AI model based on training data.

[0359] Optionally, the apparatus 1200 further includes: a receiving unit 1203;

[0360] The sending unit 1202 is further configured to send a first data set;

[0361] A receiving unit 1203 is configured to receive a fourth data set, where the fourth data set is used to describe a group model corresponding to the group, where the group model is obtained based on a first data set sent by each client in the group;

[0362] The processing unit 1201 is further configured to adjust the network parameters of the first AI model based on the fourth data set.

[0363] Optionally, the detailed implementation process of the sending unit 1202 sending the first data set refers to the relevant content of step 411 of the method 400 shown in Figure 4 or step 812 of the method 800 shown in Figure 8, and will not be described in detail here.

[0364] Optionally, the detailed implementation process of the receiving unit 1203 receiving the fourth data set refers to the relevant content of step 411 of the method 400 shown in FIG4 or step 817 of the method 800 shown in FIG8 , which will not be described in detail here.

[0365] Optionally, the detailed implementation process of the processing unit 1201 adjusting the network parameters of the first AI model based on the fourth data set can be referred to the relevant content of step 417 of method 400 shown in Figure 4 or step 818 of method 800 shown in Figure 8, which will not be described in detail here.

[0366] In an embodiment of the present application, since the processing unit performs sparse processing on the first data set to obtain the second data set, the data volume of the second data set is smaller than the data volume of the first data set. In this way, when the second data set of each client is used for grouping, the data volume of the grouping operation is reduced, thereby improving the grouping efficiency.

[0367] 13 , an embodiment of the present application provides a schematic diagram of a device 1300 . The device 1300 may be the access node provided by any of the above embodiments, and includes at least one processor 1301 , an internal connection 1302 , a memory 1303 , and at least one transceiver 1304 .

[0368] The device 1300 is a hardware device that can be used to implement the functional modules in the device 1000 shown in Figure 10. For example, those skilled in the art will appreciate that the processing unit 1003 in the device 1000 shown in Figure 10 can be implemented by the at least one processor 1301 calling the code in the memory 1303, and the receiving unit 1001 and the sending unit 1002 in the device 1000 shown in Figure 10 can be implemented by the at least one transceiver 1304.

[0369] The device 1300 may also be used to implement the functions of the access node in any of the above embodiments.

[0370] The processor 1301 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0371] The internal connection 1302 may include a path for transmitting information between the components, and may be a single board or a bus.

[0372] The at least one transceiver 1304 is configured to communicate with other devices or a communication network.

[0373] The memory 1303 may be a read-only memory (ROM) or other static storage device capable of storing static information and instructions, a random access memory (RAM) or other dynamic storage device capable of storing information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0374] Among them, the memory 1303 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 1301. The processor 1301 is used to execute the application code stored in the memory 1303 and cooperate with at least one transceiver 1304, so that the device 1300 can implement the functions of the method of the present patent.

[0375] In a specific implementation, as an embodiment, the processor 1301 may include one or more CPUs, such as CPU0 and CPU1 in FIG13 .

[0376] In a specific implementation, as an example, the device 1300 may include multiple processors, such as the processor 1301 and the processor 1307 in FIG13 . Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0377] 14 , an embodiment of the present application provides a schematic diagram of a device 1400 . The device 1400 may be a management node provided in any of the above embodiments. The device 1400 includes at least one processor 1401 , an internal connection 1402 , a memory 1403 , and at least one transceiver 1404 .

[0378] The device 1400 is a hardware structure device that can be used to implement the functional modules in the device 1100 shown in Figure 11. For example, those skilled in the art can imagine that the processing unit 1102 in the device 1100 shown in Figure 11 can be implemented by the at least one processor 1401 calling the code in the memory 1403, and the receiving unit 1101 and the sending unit 1103 in the device 1100 shown in Figure 11 can be implemented by the at least one transceiver 1404.

[0379] The device 1400 may also be used to implement the functions of the management node in any of the above embodiments.

[0380] The processor 1401 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0381] The internal connection 1402 may include a path for transmitting information between the components, and may be a single board or a bus.

[0382] The at least one transceiver 1404 is configured to communicate with other devices or a communication network.

[0383] The memory 1403 may be a read-only memory (ROM) or other static storage device capable of storing static information and instructions, a random access memory (RAM) or other dynamic storage device capable of storing information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto. The memory may be independent and connected to the processor via a bus. The memory may also be integrated with the processor.

[0384] Among them, the memory 1403 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 1401. The processor 1401 is used to execute the application code stored in the memory 1403 and cooperate with at least one transceiver 1404, so that the device 1400 can implement the functions of the patent method.

[0385] In a specific implementation, as an embodiment, the processor 1401 may include one or more CPUs, such as CPU0 and CPU1 in FIG14 .

[0386] In a specific implementation, as an example, the device 1400 may include multiple processors, such as the processor 1401 and the processor 1407 in FIG14 . Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0387] 15 , an embodiment of the present application provides a schematic diagram of a device 1500 . The device 1500 may be a client provided by any of the above embodiments. The device 1500 includes at least one processor 1501 , an internal connection 1502 , a memory 1503 , and at least one transceiver 1504 .

[0388] The device 1500 is a hardware device that can be used to implement the functional modules in the device 1200 shown in Figure 12. For example, those skilled in the art will appreciate that the processing unit 1201 in the device 1200 shown in Figure 12 can be implemented by the at least one processor 1501 calling code in the memory 1503, and the sending unit 1202 and the receiving unit 1203 in the device 1200 shown in Figure 12 can be implemented by the at least one transceiver 1504.

[0389] The device 1500 can also be used to implement the functions of the client in any of the above embodiments.

[0390] The processor 1501 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0391] The internal connection 1502 may include a path for transmitting information between the components, and may be a single board or a bus.

[0392] The at least one transceiver 1504 is configured to communicate with other devices or a communication network.

[0393] The above-mentioned memory 1503 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this. The memory can exist independently and be connected to the processor through a bus. The memory can also be integrated with the processor.

[0394] Memory 1503 is used to store application code for executing the solution of the present application, and is controlled by processor 1501. Processor 1501 is used to execute the application code stored in memory 1503 and cooperate with at least one transceiver 1504, so that the device 1500 can implement the functions of the method of the present invention.

[0395] In a specific implementation, as an embodiment, the processor 1501 may include one or more CPUs, such as CPU0 and CPU1 in FIG15 .

[0396] In a specific implementation, as an example, the device 1500 may include multiple processors, such as processor 1501 and processor 1507 in FIG15 . Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0397] 16 , an embodiment of the present application provides a federated learning system 1600 . The federated learning system 1600 includes a management node 1601 , multiple access nodes, and multiple clients connected to each access node. The multiple access nodes include a first access node 1602 . A first client 1603 is one of n clients connected to the first access node 1602 , where n is an integer greater than 1.

[0398] The first client 1603 is configured to send a data set to the first access node 1602, where the data set is used to describe a first artificial intelligence AI model in the first client 1603;

[0399] The first access node 1602 is used to cluster data sets from n clients to obtain information of one or more first clusters, and to send the information of one or more first clusters to the management node 1601;

[0400] The management node 1601 is configured to obtain one or more first global cluster centers based on information of a first clusters, where the information of the a first clusters comes from some or all of the multiple access nodes, where a is an integer greater than 1, and the one or more first global cluster centers are used to indicate one or more groups of clients;

[0401] The management node 1601 is further configured to send one or more first global cluster centers to some or all of the multiple access nodes;

[0402] The first access node 1602 is further configured to obtain a group to which each of the n clients belongs based on part or all of the one or more first global cluster centers.

[0403] Optionally, the n data sets are first data sets from n clients;

[0404] The first access node 1602 is used to perform sparse processing on the n first data sets to obtain n second data sets, where the n first data sets correspond to the n second data sets. For each first data set, the data volume of the first data set is smaller than the data volume of the second data set corresponding to the first data set. Clustering operations are performed on the n second data sets to obtain information of one or more first clusters.

[0405] Optionally, the n data sets are second data sets from the n clients;

[0406] The first client is used to perform sparse processing on the first data set to obtain a second data set, the first data set is used to describe the first AI model, the data volume of the second data set is smaller than the data volume of the first data set, and the second data set is sent to the first access node 1602.

[0407] Optionally, the first data set includes multiple first data used to describe the first AI model, and the second data set includes m first data and sequence numbers of the m first data in the first data set, where m is an integer greater than 1, and the m first data are the first data in the first data set that meet the first condition;

[0408] The first access node 1602 is configured to perform a clustering operation based on the m first data and the sequence numbers of the m first data included in each of the n second data sets to obtain information of one or more first clusters.

[0409] Optionally, the first access node 1602 is used to cluster the n data sets based on the one or more first global cluster centers to obtain one or more second clusters, and the one or more first global cluster centers correspond to the one or more second clusters. For each second cluster, the second cluster includes a partial data set in the n data sets, and the partial data set and the first global cluster center corresponding to the second cluster meet the first similarity condition, and the client corresponding to each data set included in the second cluster is determined as a group.

[0410] Optionally, the information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies the second similarity condition with the center of the first cluster. The management node 1601 is used to perform clustering operations on the data sets included in the information of a first cluster to obtain one or more first global clusters, and the one or more first global cluster centers include one or more centers of the first global clusters.

[0411] Optionally, the information of the first cluster includes the center of the first cluster, and the management node 1601 is configured to obtain one or more first global cluster centers based on the weights corresponding to the a first clusters and the centers of the a first clusters.

[0412] Optionally, the one or more first global cluster centers do not meet the convergence condition,

[0413] The first access node 1602 is configured to cluster the n data sets based on the one or more first global cluster centers to obtain one or more third clusters;

[0414] The first access node 1602 is further configured to send information of one or more third clusters to the management node 1601;

[0415] The management node 1601 is further configured to obtain one or more second global cluster centers based on information of b third clusters, where the information of the b third clusters comes from some or all of the multiple access nodes, and b is an integer greater than 1, and send the one or more second global cluster centers to some or all of the multiple access nodes;

[0416] The first access node 1602 is further configured to obtain a group to which each client belongs based on the one or more second global cluster centers when the one or more second global cluster centers meet a convergence condition.

[0417] Optionally, the n clients include a plurality of second clients belonging to a first group, and the plurality of groups include the first group;

[0418] The first access point 1602 is further configured to receive multiple first data sets, where the multiple first data sets are data sets corresponding to multiple second clients and are used to describe the first AI model in the multiple second clients, and to send a third data set to the management node 1601, where the third data set is obtained based on the multiple first data sets.

[0419] The management node 1601 is further configured to obtain a group model corresponding to the first group based on multiple third data sets, where the multiple third data sets come from some or all of the multiple access nodes, and send a fourth data set to some or all of the multiple access nodes, where the fourth data set is used to describe the group model corresponding to the first group;

[0420] The first access node 1602 is further configured to send a fourth data set to multiple second clients.

[0421] In an embodiment of the present application, since the federated learning system includes multiple access nodes, each access node receives a data set sent by a respective accessed client, and each access node performs clustering operations on the received data set in parallel to obtain information of one or more first clusters. The first access node sends information of one or more first clusters to the management node. The amount of data in the information of the first cluster is relatively small, and the management node quickly obtains one or more first global cluster centers based on the received information of the first cluster. The first access node determines the group to which each client belongs based on the one or more first global cluster centers, thereby improving the grouping efficiency.

[0422] The information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0423] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0424] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A federated learning system, characterized in that: The federated learning system includes a management node, multiple access nodes, and multiple clients connected to each access node, wherein the multiple access nodes include a first access node, the first client is one of n clients connected to the first access node, and n is an integer greater than 1. The first client is used to send a data set to the first access node, where the data set is used to describe a first artificial intelligence AI model in the first client; The first access node is used to cluster the data sets from the n clients to obtain one or more pieces of information of the first cluster, and to send the one or more pieces of information of the first cluster to the management node; The management node is used to obtain one or more first global cluster centers based on information of a first clusters, where the information of the a first clusters comes from part or all of the multiple access nodes, a is an integer greater than 1, and the one or more first global cluster centers are used to indicate one or more groups of clients; The management node is further configured to send the one or more first global cluster centers to some or all of the multiple access nodes; The first access node is further configured to obtain a group to which each of the n clients belongs based on part or all of the one or more first global cluster centers.

2. The system according to claim 1, characterized in that The n data sets are first data sets from the n clients; The first access node is used to perform sparse processing on the n first data sets to obtain n second data sets, the n first data sets correspond to the n second data sets, for each first data set, the data volume of the first data set is smaller than the data volume of the second data set corresponding to the first data set, and clustering operation is performed on the n second data sets to obtain information of the one or more first clusters.

3. The system according to claim 1, characterized in that The n data sets are second data sets from the n clients; The first client is used to perform sparse processing on the first data set to obtain a second data set, the first data set is used to describe the first AI model, the data volume of the second data set is smaller than the data volume of the first data set, and the second data set is sent to the first access node.

4. The system according to claim 3, characterized in that The first data set includes multiple first data for describing the first AI model, the second data set includes m first data and sequence numbers of the m first data in the first data set, m is an integer greater than 1, and the m first data are the first data in the first data set that meet the first condition; The first access node is used to perform a clustering operation based on the m first data and the sequence numbers of the m first data included in each of the n second data sets to obtain information of the one or more first clusters.

5. The system according to any one of claims 1 to 4, characterized in that: The first access node is used to cluster the n data sets based on the one or more first global cluster centers to obtain one or more second clusters, the one or more first global cluster centers correspond to the one or more second clusters, for each second cluster, the second cluster includes a partial data set in the n data sets, the partial data set and the first global cluster center corresponding to the second cluster meet a first similarity condition, and the client corresponding to each data set included in the second cluster is determined as a group.

6. The system according to any one of claims 1 to 5, characterized in that: The information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies a second similarity condition with the center of the first cluster. The management node is used to perform clustering operations on the data sets included in the information of the a first clusters to obtain one or more first global clusters, and the one or more first global cluster centers include the centers of the one or more first global clusters.

7. The system according to any one of claims 1 to 4, characterized in that: The information of the first cluster includes the center of the first cluster, and the management node is used to obtain the one or more first global cluster centers based on the weights corresponding to the a first clusters and the centers of the a first clusters.

8. The system according to claim 7, characterized in that The one or more first global cluster centers do not satisfy the convergence condition, The first access node is used to cluster the n data sets based on the one or more first global clustering centers to obtain one or more third clusters; The first access node is further configured to send information of the one or more third clusters to the management node; The management node is further configured to obtain one or more second global cluster centers based on information of b third clusters, where the information of the b third clusters comes from some or all of the multiple access nodes, b is an integer greater than 1, and send the one or more second global cluster centers to some or all of the multiple access nodes; The first access node is further configured to obtain the group to which each client belongs based on part or all of the one or more second global clustering centers when the one or more second global clustering centers meet the convergence condition.

9. The system according to any one of claims 1 to 8, characterized in that: The n clients include a plurality of second clients belonging to a first group, and the one or more groups include the first group; The first access point is further configured to receive a plurality of first data sets, the plurality of first data sets being data sets corresponding to the plurality of second clients, the plurality of first data sets being used to describe a first AI model in the plurality of second clients, and to send a third data set to the management node, the third data set being obtained based on the plurality of first data sets; The management node is further configured to obtain a group model corresponding to the first group based on multiple third data sets, the multiple third data sets are from some or all of the multiple access nodes, and send a fourth data set to some or all of the multiple access nodes, the fourth data set is used to describe the group model corresponding to the first group; The first access node is further configured to send the fourth data set to the multiple second clients.

10. A grouping method, characterized in that: The method is applied to a first access node, the first access node is any access node among multiple access nodes included in a federated learning system, the federated learning system further includes a management node and n clients accessing the first access node, n is an integer greater than 1, and the method includes: Receiving n data sets from the n clients; Clustering the n data sets to obtain information of one or more first clusters; Sending information of one or more first clusters to the management node; receiving one or more first global cluster centers sent by the management node, where the one or more first global cluster centers are obtained by the management node based on information of a first clusters, where the information of the a first clusters comes from part or all of the multiple access nodes, where a is an integer greater than 1, and the one or more first global cluster centers are used to indicate one or more groups of clients; The group to which each client belongs is obtained based on part or all of the one or more first global cluster centers.

11. The method according to claim 10, characterized in that The obtaining the group to which each client belongs based on part or all of the one or more first global clustering centers includes: Clustering the n data sets based on part or all of the one or more first global cluster centers to obtain one or more second clusters, the one or more first global cluster centers corresponding to the one or more second clusters, for each second cluster, the second cluster includes a part of the n data sets, and the first global cluster center corresponding to the part of the data set and the second cluster satisfies a first similarity condition; The clients corresponding to each data set included in the second cluster are determined as a group.

12. The method according to claim 10 or 11, characterized in that The information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies a second similarity condition with the center of the first cluster.

13. The method according to claim 10 or 11, characterized in that: The information of the first cluster includes the center of the first cluster, the one or more first global cluster centers do not meet the convergence condition, and the obtaining the group to which each client belongs based on part or all of the one or more first global cluster centers includes: Clustering the n data sets based on the one or more first global cluster centers to obtain one or more third clusters; Sending information of the one or more third clusters to the management node; receiving one or more second global cluster centers sent by the management node, where the one or more second global cluster centers are used to indicate the one or more groups; When the one or more second global cluster centers satisfy the convergence condition, the group to which each client belongs is obtained based on part or all of the one or more second global cluster centers.

14. The method according to any one of claims 10 to 13, characterized in that: The n data sets are n first data sets from the n clients. For each client, the first data set of the client is used to describe a first artificial intelligence AI model in the client. The clustering of the n data sets to obtain information of one or more first clusters includes: Performing sparse processing on the n first data sets to obtain n second data sets, the n first data sets correspond to the n second data sets, and for each first data set, the data amount of the first data set is smaller than the data amount of the second data set corresponding to the first data set; A clustering operation is performed on the n second data sets to obtain information of the one or more first clusters.

15. The method according to any one of claims 10 to 14, characterized in that: The method further comprises: Receiving multiple first data sets sent by multiple clients in a first group, the n clients include the multiple clients, the multiple first data sets are used to describe a first AI model in the multiple clients, and the multiple groups include the first group; Sending a third data set to the management node, where the third data set is obtained based on the multiple first data sets; receiving a fourth data set sent by the management node, where the fourth data set is used to describe a grouping model corresponding to the first grouping; The fourth data set is sent to each client in the first group.

16. A grouping method, characterized in that: The method is applied to a management node in a federated learning system, the federated learning system includes multiple access nodes and multiple clients accessing each access node, the multiple access nodes include a first access node, and the method includes: Receive a first cluster of information, where the a first cluster of information comes from part or all of the multiple access nodes, a is an integer greater than 1, the first cluster of information sent by the first access node is obtained by clustering n data sets, n is an integer greater than 1, and the n data sets come from n clients accessing the first access node; Acquire one or more first global cluster centers based on the information of the a first clusters, wherein the one or more first global cluster centers are used to indicate one or more groups of the client; The one or more first global cluster centers are sent to part or all of the multiple access nodes, and the first access nodes are used to obtain the group to which each of the n clients belongs based on part or all of the one or more first global cluster centers.

17. The method according to claim 16, characterized in that The information of the first cluster includes at least one data set in the first cluster, and the at least one data set and the center of the first cluster satisfy a second similarity condition; The obtaining one or more first global cluster centers based on the information of the a first clusters includes: A clustering operation is performed on a data set including information of the a first clusters to obtain one or more first global clusters, and the one or more first global cluster centers include centers of the one or more first global clusters.

18. The method according to claim 16, characterized in that The information of the first cluster includes the center of the first cluster; The obtaining one or more first global cluster centers based on the information of the a first clusters includes: Based on the weights corresponding to the a first clusters and the centers of the a first clusters, the one or more first global cluster centers are obtained.

19. The method according to claim 18, characterized in that The one or more first global cluster centers do not satisfy a convergence condition, and the method further includes: receiving b third clustering information, where the b third clustering information comes from part or all of the multiple access nodes, b is an integer greater than 1, and the third clustering sent by the first access node is obtained by clustering the n data sets by the first access node based on the one or more first global clustering centers; Based on the information of the b third clusters, obtaining one or more second global cluster centers; The one or more second global cluster centers are sent to some or all of the plurality of access nodes.

20. The method according to any one of claims 16 to 19, characterized in that: The method further comprises: receiving a plurality of third data sets, the plurality of third data sets coming from part or all of the plurality of access nodes, the third data set sent by the first access node being used to describe a first AI module in a plurality of clients belonging to a first group, the n clients including the plurality of clients, and the plurality of groups including the first group; Acquire a grouping model corresponding to the first grouping based on multiple third data sets; A fourth data set is sent to part or all of the plurality of access nodes, where the fourth data set is used to describe the grouping model.

21. A grouping method, characterized in that: The method is applied to a client, the client includes training data, and the method includes: Perform model training based on the training data to obtain a first artificial intelligence AI model; Performing a sparse processing on the first data set to obtain a second data set, where the first data set is used to describe the first AI model, and the amount of data in the second data set is less than the amount of data in the first data set; The second data set is sent, where the second data set is used to request to determine the group to which the client belongs.

22. The method according to claim 21, characterized in that The first data set includes a plurality of first data for describing the first AI model, and the sparse processing of the first data set to obtain the second data set includes: Selecting m first data satisfying a first condition from the first data set, where m is an integer greater than or equal to 1; The second data set is acquired based on the m first data.

23. The method of claim 22, wherein: The first condition includes that the m first data are the m data with the largest absolute values ​​in the first data set, or the m first data are the m data with numerical values ​​in the first data set that are within a first range.

24. The method according to claim 22 or 23, characterized in that The second data set includes the m first data and the sequence numbers of the m first data in the first data set; or, The second data set includes m first bits and xm second bits, x is the number of first data included in the first data set, x is an integer greater than m, the m first bits correspond to the m first data, the xm second bits correspond to the xm first data, and the xm first data are the data in the first data set other than the m first data.

25. The method according to claim 22 or 23, characterized in that The acquiring the second data set based on the m first data includes: The m first data are quantized to obtain m second data, the m first data correspond to the m second data, the bit width of each first data is respectively greater than the bit width of the second data corresponding to each first data, and the second data set includes the m second data and the serial number of the first data corresponding to each second data in the first data set.

26. The method of claim 21, wherein: The type of the first data set is a matrix or a vector, and the sparse processing of the first data set to obtain the second data set includes: Acquire a fifth data set, wherein the dimension of the fifth data set is smaller than the dimension of the first data set; The fifth data set and the first data set are operated at least once to obtain the second data set.

27. The method according to any one of claims 21 to 26, characterized in that The first data set is obtained based on network parameters of the second AI model and / or network parameters of the first AI model, and the first AI model is obtained by training the second AI model based on the training data.

28. The method according to any one of claims 21 to 27, characterized in that The method further comprises: sending the first data set; receiving a fourth data set, the fourth data set being used to describe a group model corresponding to the group, the group model being obtained based on a first data set sent by each client in the group; Adjust network parameters of the first AI model based on the fourth data set.

29. A grouping device, characterized in that: The device is any one of a plurality of access nodes included in a federated learning system, the federated learning system further includes a management node and n clients accessing the device, where n is an integer greater than 1, and the device includes: A receiving unit, configured to receive n data sets from the n clients; A processing unit, configured to cluster the n data sets to obtain information of one or more first clusters; A sending unit, configured to send information of one or more first clusters to the management node; The receiving unit is further used to receive one or more first global clustering centers sent by the management node, where the one or more first global clustering centers are obtained by the management node based on information of a first clusters, where the information of the a first clusters comes from part or all of the multiple access nodes, a is an integer greater than 1, and the one or more first global clustering centers are used to indicate one or more groups of clients; The processing unit is configured to determine the group to which each client belongs based on part or all of the one or more first global cluster centers.

30. The device according to claim 29, characterized in that The processing unit is used for: Clustering the n data sets based on the one or more first global cluster centers to obtain one or more second clusters, the one or more first global cluster centers correspond to the one or more second clusters, and for each second cluster, the second cluster includes a partial data set in the n data sets, and the partial data set and the first global cluster center corresponding to the second cluster meet a first similarity condition; The clients corresponding to each data set included in the second cluster are determined as a group.

31. The device according to claim 29 or 30, characterized in that The information of the first cluster includes at least one data set in the first cluster, and the at least one data set satisfies a second similarity condition with the center of the first cluster.

32. The device according to claim 29 or 30, characterized in that The information of the first cluster includes the center of the first cluster, and the one or more first global cluster centers do not meet the convergence condition; The processing unit is further configured to cluster the n data sets based on the one or more first global clustering centers to obtain one or more third clusters; The sending unit is further configured to send the information of the one or more third clusters to the management node; The receiving unit is further used to receive one or more second global cluster centers sent by the management node, where the one or more second global cluster centers are used to indicate the one or more groups; The processing unit is further configured to obtain the group to which each client belongs based on the one or more second global clustering centers when the one or more second global clustering centers meet the convergence condition.

33. The device according to any one of claims 29 to 32, characterized in that The n data sets are n first data sets from the n clients, and for each client, the first data set of the client is used to describe a first AI model in the client, and the processing unit is further used to: Performing sparse processing on the n first data sets to obtain n second data sets, the n first data sets correspond to the n second data sets, and for each first data set, the data amount of the first data set is smaller than the data amount of the second data set corresponding to the first data set; A clustering operation is performed on the n second data sets to obtain information of the one or more first clusters.

34. The device according to any one of claims 29 to 33, characterized in that The receiving unit is further used to receive multiple first data sets sent by multiple clients in a first group, the n clients include the multiple clients, the multiple first data sets are used to describe the first AI model in the multiple clients, and the multiple groups include the first A group; The sending unit is further configured to send a third data set to the management node, where the third data set is obtained based on the multiple first data sets; The receiving unit is further used to receive a fourth data set sent by the management node, where the fourth data set is used to describe a grouping model corresponding to the first grouping; The sending unit is further configured to send the fourth data set to each client in the first group.

35. A grouping device, characterized in that: The device is applied to a management node in a federated learning system, the federated learning system includes multiple access nodes and multiple clients connected to each access node, the multiple access nodes include a first access node, and the device includes: a receiving unit, configured to receive a first cluster information, wherein the a first cluster information comes from part or all of the multiple access nodes, a is an integer greater than 1, the first cluster information sent by the first access node is obtained by clustering n data sets, n is an integer greater than 1, and the n data sets come from n clients accessing the first access node; A processing unit, configured to obtain one or more first global cluster centers based on information of the a first clusters, wherein the one or more first global cluster centers are used to indicate one or more groups of the client; A sending unit is used to send the one or more first global clustering centers to some or all of the multiple access nodes, and the first access node is used to determine the group to which each of the n clients belongs based on some or all of the one or more first global clustering centers.

36. The device according to claim 35, characterized in that The information of the first cluster includes at least one data set in the first cluster, and the at least one data set and the center of the first cluster satisfy a second similarity condition; The processing unit is used to perform clustering operation on the data set including the information of the a first clusters to obtain one or more first global clusters, and the one or more first global cluster centers include the centers of the one or more first global clusters.

37. The device according to claim 35, characterized in that The information of the first cluster includes the center of the first cluster; The processing unit is used to obtain the one or more first global cluster centers based on the weights corresponding to the a first clusters and the centers of the a first clusters.

38. The device according to claim 37, characterized in that The one or more first global cluster centers do not satisfy the convergence condition, The receiving unit is further configured to receive information of b third clusters, where the information of the b third clusters comes from part or all of the multiple access nodes, b is an integer greater than 1, and the third cluster sent by the first access node is obtained by clustering the n data sets by the first access node based on the one or more first global cluster centers; The processing unit is further used to obtain one or more second global cluster centers based on the information of the b third clusters; The sending unit is further configured to send the one or more second global clustering centers to part or all of the multiple access nodes.

39. The device according to any one of claims 35 to 38, characterized in that The receiving unit is further configured to receive multiple third data sets, where the multiple third data sets come from part or all of the multiple access nodes, and the third data set sent by the first access node is used to describe a first AI module in multiple clients belonging to a first group, where the n clients include the multiple clients, and the multiple groups include the first group; The processing unit is further configured to obtain a grouping model corresponding to the first grouping based on the multiple third data sets; The sending unit is further used to send a fourth data set to part or all of the multiple access nodes, where the fourth data set is used to describe the grouping model.

40. A grouping device, characterized in that: The device stores training data, and the device comprises: A processing unit, configured to perform model training based on the training data to obtain a first artificial intelligence AI model; The processing unit is further used to perform sparse processing on the first data set to obtain a second data set, the first data set is used to describe the first AI model, and the amount of data in the second data set is less than the amount of data in the first data set; The sending unit is used to send the second data set, where the second data set is used to request to determine the group to which the device belongs.

41. The device according to claim 40, characterized in that The first data set includes a plurality of first data for describing the first AI model, and the processing unit is configured to: Selecting m first data satisfying a first condition from the first data set, where m is an integer greater than or equal to 1; The second data set is acquired based on the m first data.

42. The device according to claim 41, characterized in that The first condition includes that the m first data are the m data with the largest absolute values ​​in the first data set, or the m first data are the m data with numerical values ​​in the first data set that are within a first range.

43. The device according to claim 41 or 42, characterized in that The second data set includes the m first data and the sequence numbers of the m first data in the first data set; or, The second data set includes m first bits and xm second bits, x is the number of first data included in the first data set, x is an integer greater than m, the m first bits correspond to the m first data, the xm second bits correspond to the xm first data, and the xm first data are the data in the first data set other than the m first data.

44. The device according to claim 41 or 42, characterized in that The processing unit is used for: The m first data are quantized to obtain m second data, the m first data correspond to the m second data, the bit width of each first data is respectively greater than the bit width of the second data corresponding to each first data, and the second data set includes the m second data and the serial number of the first data corresponding to each second data in the first data set.

45. The device according to claim 40, characterized in that The type of the first data set is a matrix or a vector, and the processing unit is used for: Acquire a fifth data set, wherein the dimension of the fifth data set is smaller than the dimension of the first data set; The fifth data set and the first data set are operated at least once to obtain the second data set.

46. ​​The device according to any one of claims 40 to 45, characterized in that The first data set is obtained based on network parameters of the second AI model and / or network parameters of the first AI model, and the first AI model is obtained by training the second AI model based on the training data.

47. The device according to any one of claims 40 to 46, characterized in that The device further comprises: a receiving unit; The sending unit is further used to send the first data set; The receiving unit is configured to receive a fourth data set, where the fourth data set is used to describe a group model corresponding to the group, where the group model is obtained based on a first data set sent by each client in the group; The processing unit is further used to adjust the network parameters of the first AI model based on the fourth data set.

48. A computing device, characterized in that Each device in the computing device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions so that the computing device executes the method according to any one of claims 10-28.

49. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 10 to 28 is implemented.

50. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 10 to 28 is implemented.

Images