Federal learning method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of data processing, in particular to a federal learning method and device, electronic equipment and a storage medium. The method comprises the following steps: a first member device initializes a plurality of clustering models, and sends the plurality of clustering models to a second member device, so that the second member device uses local data to respectively train the plurality of clustering models to obtain the membership degree of each clustering model; and receiving membership information from the second member equipment. And comparing the membership degree value with a predetermined first threshold value and a predetermined second threshold value, and distributing the data points provided by the second member equipment to different areas. And aggregating the plurality of clustering models based on the weight of each clustering model to obtain a global model, and sending the global model to the second member device, so that the second member device adjusts the global model to generate a local personalized model. According to the scheme, the data privacy can be protected, the generalization ability and the individuation degree of the model are improved, and the communication cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a federated learning method, device, electronic device and storage medium. Background Art

[0002] As a distributed machine learning framework, federated learning performs outstandingly in protecting data privacy, but it still faces multiple technical challenges. On the one hand, traditional algorithms are based on the assumption that client data is independent and identically distributed, but in actual applications, data is often not independent and identically distributed, which affects the model training effect. On the other hand, federated learning has high communication costs, and model parameters need to be frequently synchronized between the client and the server, which is particularly obvious when the network environment is poor. At the same time, the convergence and personalization of the model in non-data are also problems, which may be difficult to achieve the ideal state or unable to adapt to the characteristics of each client data. In addition, the privacy protection mechanism still needs to be strengthened to prevent the leakage of sensitive information during the model update process. Summary of the invention

[0003] The embodiments of the present application provide a federated learning method, device, electronic device and storage medium, which are used to improve the generalization ability and personalization of the model and reduce communication costs while protecting data privacy.

[0004] In a first aspect, an embodiment of the present application provides a federated learning method, which is performed by a federated learning system, the federated learning system including a first member device and a plurality of second member devices, and the method including:

[0005] The first member device initializes multiple clustering models and sends the multiple clustering models to the second member device, so that the second member device uses local data to train the multiple clustering models respectively to obtain the membership of each clustering model, where the membership is used to indicate the matching degree between the local data and the clustering model;

[0006] receiving membership information from a second member device, the membership information including a membership value between each clustering model and each data point in the local data set;

[0007] Comparing the membership value with a predetermined first threshold and a second threshold, allocating the data points provided by the second member device to different regions, where the different regions represent different degrees of association between the data points and the cluster center, and the first threshold and the second threshold are determined by optimizing the membership distribution;

[0008] Based on the weights of each clustering model, multiple clustering models are aggregated to obtain a global model, and the global model is sent to the second member device so that the second member device adjusts the global model to generate a local personalized model. The weight of each clustering model is determined based on the number of data points and membership in different regions.

[0009] In the above method, by combining the global model with local data, the second member device can adjust the global model to generate a local personalized model to better adapt to its local data characteristics. The communication process between the first member device and the second member device does not need to transmit the original data to the central server, but only transmits the clustering model, membership information and global model between devices, which greatly reduces the risk of data leakage. Through the weight aggregation of the clustering model, the computing power of each second member device can be fully utilized, reducing unnecessary communication overhead and communication costs.

[0010] Optionally, comparing the membership value with a predetermined first threshold and a second threshold to distribute the data points provided by the second member device to different areas specifically includes:

[0011] If the degree of membership of the data point to the first clustering model is less than or equal to the first threshold, the data point belongs to the irrelevant region of the first clustering model;

[0012] If the degree of membership of the data point to the first clustering model is greater than the first threshold and less than the second threshold, the data point belongs to the edge area of ​​the first clustering model;

[0013] If the degree of membership of a data point to the first clustering model is greater than or equal to a second threshold, the data point belongs to the core area of ​​the first clustering model.

[0014] In the above method, by assigning data points to different regions, the relationship between data points and clustering models can be understood more accurately. This helps to determine the weight of each clustering model according to the number and membership of data points in different regions when multiple clustering models are subsequently aggregated to form a global model, that is, based on reflecting the contribution of each clustering model in the global model, thereby improving the performance of the global model.

[0015] Optionally, the first threshold and the second threshold are determined in the following manner:

[0016] The first threshold and the second threshold are determined by minimizing the value of the objective function, where the objective function is used to measure the degree of optimization of the membership distribution.

[0017] In the above method, by minimizing the objective function to determine the threshold, it is possible to ensure a more reasonable distribution of membership, which helps to more accurately divide data points into different regions and improve the effect of subsequent clustering analysis.

[0018] Optionally, the above objective function is calculated in the following way:

[0019] Calculate the sum of the membership of the data points in the irrelevant area and the edge area to obtain the first sum of the membership;

[0020] Traversing each data point, if the membership is greater than or equal to the first threshold, calculating the membership difference between the maximum membership of the data point and its actual membership, and summing the membership differences to obtain a second membership sum;

[0021] Determine the number of data points in the edge area;

[0022] The value of the objective function is determined based on the first membership sum, the second membership sum, and the number of data points in the edge area.

[0023] In the above method, by calculating the first membership sum and the second membership sum, the membership distribution of the data points in the irrelevant area and the edge area can be quantified. This helps to evaluate the rationality of the current membership distribution and provide a basis for subsequent optimization. By determining the value of the objective function based on the first membership sum, the second membership sum and the number of data points in the edge area, it helps to balance the influence of the irrelevant area, the edge area and the core area on the clustering model.

[0024] Optionally, the weight of the above clustering model is determined in the following way:

[0025] Determine the number of data points in the core area and the number of data points in the boundary area;

[0026] If the number of data points in the core area and the number of data points in the boundary area are not zero, the weight of the core area is multiplied by the weight coefficient of the core area, and the weight of the boundary area is multiplied by the weight coefficient of the boundary area to obtain the weight of the clustering model. The weight of the core area is the average weight of the data points in the core area, and the weight of the boundary area is the weighted average of the sum of the membership weights of all data points in the boundary area.

[0027] If the number of data points in the core area is not zero and the number of data points in the boundary area is zero, the weight of the core area is used as the weight of the clustering model;

[0028] If the number of data points in the core area is zero and the number of data points in the boundary area is not zero, the weight of the boundary area is used as the weight of the clustering model.

[0029] In the above method, by considering the number of data points in the core area and the boundary area and their membership, the importance of different data points in the clustering model can be more accurately reflected. This helps to give more important data points higher weights when aggregating multiple clustering models, thereby improving the performance of the global model.

[0030] In a second aspect, an embodiment of the present application provides a federated learning method, the federated learning method is performed by a federated learning system, the federated learning system includes a first member device and multiple second member devices, and the method includes:

[0031] The second member device receives the multiple clustering models from the first member device, and uses the local data to train the multiple clustering models respectively to obtain the membership degree of each clustering model, where the membership degree is used to indicate the matching degree between the local data set and the clustering model;

[0032] Sending membership information to the first member device, where the membership information includes a membership value between each clustering model and each data point in the local data set;

[0033] A global model is received from a first member device, and the global model is adjusted to generate a local personalized model, wherein the global model is obtained by aggregating multiple clustering models based on the number of data points in different regions, the degree of membership, and the weight of each region by the first member device.

[0034] In the above method, by sending membership information instead of complete local data, the second member device can also protect the privacy and security of local data. The second member device does not need to train the model from scratch, but can make adjustments based on the global model, which can save a lot of computing resources and time. By making adjustments based on the global model, the second member device can retain the generalization ability of the global model while combining local data characteristics for personalized optimization.

[0035] Optionally, the above-mentioned adjustment of the global model to generate a local personalized model specifically includes:

[0036] Calculate a new weight of each data point according to the degree of membership of the data point to each cluster and the weight of each cluster model, where the data point comes from a local data set of the second member device;

[0037] Based on the new weights and local data set, the global model is adjusted to generate a local personalized model.

[0038] In the above method, by using the local data set to adjust the global model, the second member device can generate a personalized model that is more in line with the characteristics of the local data, thereby improving the accuracy and practicality of the model.

[0039] In a third aspect, an embodiment of the present application provides a federated learning system, the system comprising a first member device and a plurality of second member devices;

[0040] The first member device is used to initialize multiple clustering models and send the multiple clustering models to the second member device;

[0041] The second member device is used to use local data to train multiple clustering models respectively to obtain the membership degree of each clustering model, where the membership degree is used to indicate the matching degree between the local data and the clustering model;

[0042] The first member device is further used to receive membership information from the second member device, the membership information including the membership of each clustering model and the local data; compare the membership with the first threshold and the second threshold, and distribute the data points provided by the second member device to different areas, where the different areas represent different degrees of association between the data points and the cluster center, and the first threshold and the second threshold are determined by optimizing the membership distribution; aggregate multiple clustering models based on the weights of each clustering model to obtain a global model, and send the global model to the second member device;

[0043] The second member device is also used to adjust the global model to generate a local personalized model.

[0044] In a fourth aspect, an embodiment of the present application provides a federated learning device, which is applied to a first member device, and includes:

[0045] A processing module, used to initialize multiple clustering models;

[0046] A communication module, used to send multiple clustering models to the second member device, so that the second member device uses local data to train the multiple clustering models respectively to obtain the membership of each clustering model, where the membership is used to indicate the matching degree between the local data and the clustering model;

[0047] The communication module is further used to receive membership information from the second member device, where the membership information includes a membership value between each clustering model and each data point in the local data set;

[0048] The processing module is further used to compare the membership value with a predetermined first threshold and a second threshold, and distribute the data points provided by the second member device to different areas, where the different areas represent different degrees of association between the data points and the cluster center, and the first threshold and the second threshold are determined by optimizing the membership distribution;

[0049] The processing module is also used to aggregate multiple clustering models based on the weights of each clustering model to obtain a global model;

[0050] The communication module is also used to send the global model to the second member device so that the second member device adjusts the global model to generate a local personalized model. The weight of each clustering model is determined based on the number of data points and membership in different regions.

[0051] In a fifth aspect, an embodiment of the present application provides a federated learning device, which is applied to a second member device, and includes:

[0052] A communication module, configured to receive a plurality of clustering models from a first member device;

[0053] A processing module, used to use local data to train multiple clustering models respectively to obtain the membership of each clustering model, where the membership is used to indicate the matching degree between the local data set and the clustering model;

[0054] The communication module is further used to send membership information to the first member device, where the membership information includes a membership value between each clustering model and each data point in the local data set;

[0055] The communication module is further used to receive the global model from the first member device;

[0056] The processing module is also used to adjust the global model to generate a local personalized model. The global model is obtained by aggregating multiple clustering models based on the number of data points, membership degrees and weights of each area in different areas of the first member device.

[0057] In the sixth aspect, an embodiment of the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the processor implements any one of the federated learning methods in the first to second aspects.

[0058] In the seventh aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any one of the federated learning methods in the first aspect to the second aspect is implemented.

[0059] In an eighth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which is executed by a processor to implement any one of the federated learning methods in the first to second aspects above.

[0060] The technical effects brought about by any one of the implementation methods in the third to eighth aspects can refer to the technical effects brought about by the corresponding implementation methods in the first to fifth aspects, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flowchart of a federated learning method provided in an embodiment of the present application;

[0062] Figure 2 A schematic diagram of a federated learning method implemented in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of the structure of a federated learning device provided in an embodiment of the present application;

[0064] Figure 4 A schematic diagram of the structure of another federated learning device provided in an embodiment of the present application;

[0065] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0067] The application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. In the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more.

[0068] Optional, such as Figure 1 As shown, a flow chart of a federated learning method provided in an embodiment of the present application. The federated learning method is executed by a federated learning system. The federated learning system includes a first member device and multiple second member devices. The method flow chart may specifically include the following operations:

[0069] Step S101: A first member device initializes multiple clustering models.

[0070] In an optional embodiment, the first member device may set a plurality of clustering models with different parameters, wherein the goal of each clustering model is to group data points into different categories or clusters, so that data points within the same cluster are similar to each other, while data points between different clusters are quite different.

[0071] In this way, by setting multiple clustering models with different parameters, data can be grouped in different ways.

[0072] Step S102: The first member device sends multiple clustering models to the second member device.

[0073] Step S103: The second member device uses local data to train multiple clustering models respectively to obtain the membership degree of each clustering model.

[0074] Among them, the membership degree is used to indicate the matching degree between local data and clustering model.

[0075] In an optional embodiment, the second member device may apply its local data to each received clustering model. For each local data sample, calculate its distance to all cluster centers, select the minimum distance as the loss of the sample, and select the cluster with the minimum distance as the cluster to which the data sample belongs. Calculate the membership of each data point to each cluster. The membership can be in the form of a matrix, i.e., a membership matrix, in which each element represents the degree to which a data point belongs to a cluster.

[0076] It can be understood that selecting the smallest distance as the loss of the sample indicates that the sample has the highest similarity with a certain cluster center.

[0077] Step S104: The second member device sends membership information to the first member device.

[0078] The membership information includes the membership value of each clustering model and each data point in the local data set.

[0079] Step S105: The first member device compares the degree of membership with the first threshold and the second threshold, and distributes the data points provided by the second member device to different areas.

[0080] Different regions represent different degrees of association between data points and cluster centers, and the first threshold and the second threshold are determined by optimizing the membership distribution.

[0081] In an optional embodiment, the first member device receives the membership information sent by the second member device, performs ternary clustering through shadow set theory, and allocates the data points provided by the second member device to different areas. Specifically, the membership value can be compared with a predetermined first threshold and a second threshold, and the data points provided by the second member device can be allocated to different areas. If the membership of the data point to the first clustering model is less than or equal to the first threshold, and the correlation between the data point and the first clustering model is low, then the data point belongs to an irrelevant area of ​​the first clustering model. If the membership of the data point to the first clustering model is greater than the first threshold and less than the second threshold, and the correlation between the data point and the first clustering model is moderate, then the data point belongs to the edge area of ​​the first clustering model. If the membership of the data point to the first clustering model is greater than or equal to the second threshold, and the data point is highly correlated with the first clustering model, then the data point belongs to the core area of ​​the first clustering model.

[0082] In this way, by comparing the membership value with the predetermined first threshold and second threshold, the data points can be assigned to different regions. Since different regions represent different degrees of association between the data points and the cluster center, further data analysis is helpful.

[0083] The following describes how to determine the first threshold and the second threshold:

[0084] The first threshold and the second threshold are determined by minimizing the value of the objective function, where the objective function is used to measure the degree of optimization of the membership distribution.

[0085] For example, the objective function is calculated as follows:

[0086] Calculate the membership sum of the data points in the irrelevant area and the edge area to obtain a first membership sum. Traverse each data point, if the membership is greater than or equal to the first threshold, calculate the membership difference between the maximum membership of the data point and its actual membership, and sum the membership differences to obtain a second membership sum. Determine the number of data points in the edge area. Determine the value of the objective function based on the first membership sum, the second membership sum, and the number of data points in the edge area.

[0087] The objective function can satisfy the following formula:

[0088]

[0089] Where α represents the first threshold, β represents the second threshold, K represents the kth cluster, t represents the tth round of training, and π i Indicates the i-th second member device. Indicates that the data point provided by the i-th second member device in the t-th round of cluster training belongs to the cluster center c k The degree of membership. It represents the sum of the membership degrees of data points whose membership degrees are less than or equal to the second threshold β. Indicates the number of data points in the edge area.

[0090] In order to find the best α and β, we need to minimize the objective function

[0091] Right now:

[0092] By adjusting the values ​​of α and β and calculating the value of the objective function, the combination of α and β that minimizes the objective function is found.

[0093] In this way, by continuously adjusting the threshold and calculating the value of the objective function, the optimal threshold setting can be found, which helps to ensure that the data points are correctly divided into different areas, thereby improving the effect of clustering analysis.

[0094] Step S106: The first member device aggregates multiple clustering models based on the weights of the clustering models to obtain a global model.

[0095] Among them, the weight of each clustering model is determined based on the number of data points and membership in different regions.

[0096] In an optional embodiment, the number of data points in the core area and the number of data points in the boundary area are determined. If the number of data points in the core area and the number of data points in the boundary area are not zero, the weight of the core area is multiplied by the weight coefficient of the core area based on the kernel, and the weight of the boundary area is multiplied by the weight coefficient of the boundary area to obtain the weight of the clustering model, the weight of the core area is the average of the weights of the data points in the core area, and the weight of the boundary area is the weighted average of the sum of the membership weights of all data points in the boundary area. If the number of data points in the core area is not zero and the number of data points in the boundary area is zero, the weight of the core area is used as the weight of the clustering model. If the number of data points in the core area is zero and the number of data points in the boundary area is not zero, the weight of the boundary area is used as the weight of the clustering model.

[0097] For example, for the kth cluster model c in the t+1 round of clustering training k , its weight It can be determined using the following formula:

[0098]

[0099] in, Indicates the core area c in the tth round of clustering training 0 The number of data points. Indicates the boundary region F in the tth round of clustering training r The number of data points. Indicates the core area c in the tth round of clustering training 0 The weight of . Indicates the boundary region F in the tth round of clustering training r The weight of . Indicates the core area. It represents the model weight of the kth cluster model in the tth round of cluster training. is the weight coefficient of the core area. is the weight coefficient of the boundary area. It can be preset by the user or dynamically adjusted according to the application scenario.

[0100] Weight of core area It is the core area Weight of inner data points The average value satisfies the following formula:

[0101]

[0102] in, Indicates the core area c in the tth round of clustering training 0 The weight of . Indicates the core area The number of data points in .

[0103] Weight of the boundary area It is a border area The membership weights of all data points in The weighted average of and satisfies the following formula:

[0104]

[0105] in, Indicates that in the tth round of clustering training, the data point provided by the i-th second member device belongs to the cluster center c k The degree of membership.

[0106] Right now:

[0107] When there are data points in both the core area and the boundary area,

[0108] When only the core area has data points,

[0109] When only the boundary region has data points,

[0110] In this way, based on the weight of each clustering model, multiple clustering models are aggregated to obtain a global model, which can better reflect the overall distribution and characteristics of the data.

[0111] Step S107: The first member device sends the global model to the second member device.

[0112] Step S108: The second member device adjusts the global model to generate a local personalized model.

[0113] In each iteration, the second member device can calculate the new weight of each data point based on the degree of membership of the data point to each cluster and the weight of each cluster model. The second member device fine-tunes or personalizes the global model based on the new weight and the local data set, where the data point comes from the local data set of the second member device.

[0114] Specifically, the weight of the data point It is through the weights of different cluster models and data point π i Membership to the cluster Obtained by weighted summation. Satisfies the following formula:

[0115]

[0116] in, Represents the data point π after the t+1th iteration i The weight of . Represents the data point π after the t+1th iteration i For cluster c k The degree of membership. Indicates cluster c after the t+1th iteration k The weight of . k is the total number of clusters.

[0117] like Figure 2 As shown, the present application provides a schematic diagram of implementing a federated learning method. Figure 2 In the example, the first member device is the central server. The second member device includes client 1, client 2, and client 3. The central server initializes multiple clustering models and sends multiple clustering models to client 1, client 2, and client 3. Client 1, client 2, and client 3 use local data to train multiple clustering models to obtain the membership of each clustering model. Client 1, client 2, and client 3 send membership information to the central server respectively. Among them, the membership is in the form of a matrix. The central server receives the membership information from client 1, client 2, and client 3. The central server performs ternary clustering through shadow set theory to obtain cluster 1, cluster 2...cluster k. The membership value is compared with a predetermined first threshold and a second threshold, and the data points provided by client 1, client 2, and client 3 are allocated to different areas. The central server aggregates multiple clustering models based on the weights of each clustering model to obtain a global model, and sends the global model to client 1, client 2, and client 3. Client 1, client 2, and client 3 adjust the global model to generate a local personalized model.

[0118] Figure 3 A structural diagram of a federated learning device provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the apparatus is applied to a first member device, and includes: a processing module 301 and a communication module 302 .

[0119] Processing module 301, used to initialize multiple clustering models;

[0120] The communication module 302 is used to send multiple clustering models to the second member device, so that the second member device uses local data to train the multiple clustering models respectively to obtain the membership of each clustering model, where the membership is used to indicate the matching degree between the local data and the clustering model;

[0121] The communication module 302 is further used to receive membership information from the second member device, where the membership information includes a membership value between each clustering model and each data point in the local data set;

[0122] The processing module 301 is further used to compare the membership value with a predetermined first threshold and a second threshold, and allocate the data points provided by the second member device to different areas, where the different areas represent different degrees of association between the data points and the cluster center, and the first threshold and the second threshold are determined by optimizing the membership distribution;

[0123] The processing module 301 is also used to aggregate multiple clustering models based on the weights of each clustering model to obtain a global model;

[0124] The communication module 302 is further used to send the global model to the second member device so that the second member device adjusts the global model to generate a local personalized model. The weight of each clustering model is determined based on the number of data points and membership in different regions.

[0125] Optionally, the membership value is compared with a predetermined first threshold and a second threshold, and the data points provided by the second member device are allocated to different areas. The processing module 301 is specifically used for:

[0126] If the degree of membership of the data point to the first clustering model is less than or equal to the first threshold, the data point belongs to the irrelevant region of the first clustering model;

[0127] If the degree of membership of the data point to the first clustering model is greater than the first threshold and less than the second threshold, the data point belongs to the edge area of ​​the first clustering model;

[0128] If the degree of membership of a data point to the first clustering model is greater than or equal to a second threshold, the data point belongs to the core area of ​​the first clustering model.

[0129] Optionally, the first threshold and the second threshold are determined in the following manner, and the processing module 301 is further configured to:

[0130] The first threshold and the second threshold are determined by minimizing the value of the objective function, where the objective function is used to measure the degree of optimization of the membership distribution.

[0131] Optionally, the objective function is calculated in the following manner, and the processing module 301 is further used for:

[0132] Calculate the sum of the membership of the data points in the irrelevant area and the edge area to obtain the first sum of the membership;

[0133] Traversing each data point, if the membership is greater than or equal to the first threshold, calculating the membership difference between the maximum membership of the data point and its actual membership, and summing the membership differences to obtain a second membership sum;

[0134] Determine the number of data points in the edge area;

[0135] The value of the objective function is determined based on the first membership sum, the second membership sum, and the number of data points in the edge area.

[0136] Optionally, the weight of the clustering model is determined in the following manner, and the processing module 301 is further used for:

[0137] Determine the number of data points in the core area and the number of data points in the boundary area;

[0138] If the number of data points in the core area and the number of data points in the boundary area are not zero, the weight of the core area is multiplied by the weight coefficient of the core area, and the weight of the boundary area is multiplied by the weight coefficient of the boundary area to obtain the weight of the clustering model. The weight of the core area is the average weight of the data points in the core area, and the weight of the boundary area is the weighted average of the sum of the membership weights of all data points in the boundary area.

[0139] If the number of data points in the core area is not zero and the number of data points in the boundary area is zero, the weight of the core area is used as the weight of the clustering model;

[0140] If the number of data points in the core area is zero and the number of data points in the boundary area is not zero, the weight of the boundary area is used as the weight of the clustering model.

[0141] Figure 4 A structural diagram of a federated learning device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the apparatus is applied to a second member device, and includes: a communication module 401 and a processing module 402 .

[0142] The communication module 401 is used to receive multiple clustering models from a first member device;

[0143] Processing module 402, used to use local data to train multiple clustering models respectively to obtain the membership of each clustering model, where the membership is used to indicate the matching degree between the local data set and the clustering model;

[0144] The communication module 401 is further used to send membership information to the first member device, where the membership information includes a membership value between each clustering model and each data point in the local data set;

[0145] The communication module 401 is further configured to receive the global model from the first member device;

[0146] The processing module 402 is further used to adjust the global model to generate a local personalized model. The global model is obtained by aggregating multiple clustering models based on the number of data points in different regions, the degree of membership and the weight of each region by the first member device.

[0147] Optionally, the global model is adjusted to generate a local personalized model, and the processing module 402 is specifically used to:

[0148] Calculate a new weight of each data point according to the degree of membership of the data point to each cluster and the weight of each cluster model, where the data point comes from a local data set of the second member device;

[0149] Based on the new weights and local data set, the global model is adjusted to generate a local personalized model.

[0150] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0151] At least one processor 501, and a memory 502 connected to the at least one processor 501. The specific connection medium between the processor 501 and the memory 502 is not limited in the embodiment of the present application. Figure 5 In the example, the processor 501 and the memory 502 are connected via a bus 500. The bus 500 is Figure 5 The connections between other components are shown in bold lines, and are not intended to be limiting. The bus 500 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 501 can also be called a controller, and there is no limitation on the name.

[0152] In the embodiment of the present application, the memory 502 stores instructions that can be executed by at least one processor 501. The at least one processor 501 can execute the database maintenance method discussed above by executing the instructions stored in the memory 502. The processor 501 can implement Figure 3 or Figure 4 The functions of each module in the device shown.

[0153] Among them, the processor 501 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 502 and calling data stored in the memory 502, the various functions of the device and processing data, the device can be monitored as a whole.

[0154] In one possible design, the processor 501 may include one or more processing units, and the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a driver interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0155] Processor 501 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the maintenance method of the database disclosed in the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.

[0156] The memory 502 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 502 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 502 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 502 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0157] By designing and programming the processor 501, the code corresponding to the database maintenance method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 3 The database maintenance method of the embodiment shown. How to design and program the processor 501 is a technology well known to those skilled in the art and will not be described in detail here.

[0158] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0159] The embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the database maintenance method in the above embodiment.

[0160] The present application also provides a federated learning system, which may include a first member device and a plurality of second member devices. The operations performed by the first member device and the second member device may refer to Figure 1 Related description of the illustrated embodiment.

[0161] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0162] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0163] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps of the functions specified in one or more boxes. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A federated learning method, characterized in that: The federated learning method is performed by a federated learning system, the federated learning system includes a first member device and a plurality of second member devices, and the method includes: The first member device initializes multiple clustering models, and sends the multiple clustering models to the second member device, so that the second member device uses local data to train the multiple clustering models respectively to obtain the membership of each clustering model, where the membership is used to indicate the matching degree between the local data and the clustering model; receiving membership information from the second member device, the membership information including a membership value between each clustering model and each data point in the local data set; Comparing the membership value with a predetermined first threshold and a second threshold, allocating the data points provided by the second member device to different regions, the different regions representing different degrees of association between the data points and the cluster center, the first threshold and the second threshold being determined by optimizing the membership distribution; The multiple clustering models are aggregated based on the weights of each clustering model to obtain a global model, and the global model is sent to the second member device so that the second member device adjusts the global model to generate a local personalized model, wherein the weights of each clustering model are determined based on the number of data points and membership degrees in different regions.

2. The method according to claim 1, characterized in that The step of comparing the membership value with a predetermined first threshold and a second threshold, and allocating the data points provided by the second member device to different areas specifically includes: If the degree of membership of a data point to the first clustering model is less than or equal to a first threshold, the data point belongs to an irrelevant region of the first clustering model; If the degree of membership of the data point to the first clustering model is greater than the first threshold and less than the second threshold, the data point belongs to the edge area of ​​the first clustering model; If the degree of membership of the data point to the first clustering model is greater than or equal to the second threshold, the data point belongs to the core area of ​​the first clustering model.

3. The method according to claim 2, characterized in that The first threshold and the second threshold are determined in the following manner: The first threshold and the second threshold are determined by minimizing a value of an objective function, where the objective function is used to measure the degree of optimization of the membership distribution.

4. The method according to claim 3, characterized in that The objective function is calculated in the following way: Calculate the membership sum of the data points in the irrelevant area and the edge area to obtain the first membership sum; Traversing each data point, if the membership is greater than or equal to the first threshold, calculating the membership difference between the maximum membership of the data point and its actual membership, and summing the membership differences to obtain a second membership sum; Determine the number of data points in the edge area; The value of the objective function is determined based on the first membership sum, the second membership sum, and the number of data points in the edge area.

5. The method according to claim 1, characterized in that The weights of the clustering model are determined in the following way: Determine the number of data points in the core area and the number of data points in the boundary area; If the number of data points in the core area and the number of data points in the boundary area are not zero, the weight of the core area is multiplied by the weight coefficient of the core area, and the weight of the boundary area is multiplied by the weight coefficient of the boundary area to obtain the weight of the clustering model, wherein the weight of the core area is the average of the weights of the data points in the core area, and the weight of the boundary area is the weighted average of the sum of the membership weights of all data points in the boundary area; If the number of data points in the core area is not zero and the number of data points in the boundary area is zero, the weight of the core area is used as the weight of the clustering model; If the number of data points in the core area is zero, and the number of data points in the boundary area is not zero, the weight of the boundary area is used as the weight of the clustering model.

6. A federated learning method, characterized in that: The federated learning method is performed by a federated learning system, the federated learning system includes a first member device and a plurality of second member devices, and the method includes: The second member device receives multiple clustering models from the first member device, and uses local data to train the multiple clustering models to obtain membership of each clustering model, where the membership is used to indicate the degree of matching between the local data set and the clustering model; Sending membership information to the first member device, the membership information including a membership value between each clustering model and each data point in the local data set; A global model is received from a first member device, and the global model is adjusted to generate a local personalized model, wherein the global model is obtained by aggregating the multiple clustering models based on the number of data points in different regions, the degree of membership, and the weight of each region by the first member device.

7. The method according to claim 6, characterized in that The adjusting the global model to generate a local personalized model specifically includes: Calculating a new weight of each data point according to the degree of membership of the data point to each cluster and the weight of each cluster model, wherein the data point comes from the local data set of the second member device; According to the new weight and the local data set, the global model is adjusted to generate a local personalized model.

8. A federated learning system, characterized in that: The system includes a first member device and a plurality of second member devices; The first member device is used to initialize multiple clustering models and send the multiple clustering models to the second member device; The second member device is used to use local data to train the multiple clustering models respectively to obtain the membership of each clustering model, where the membership is used to indicate the matching degree between the local data and the clustering model; The first member device is further used to receive membership information from the second member device, the membership information including the membership of each clustering model and local data; compare the membership with a first threshold and a second threshold, and allocate data points provided by the second member device to different areas, wherein the different areas represent different degrees of association between the data points and the cluster center, and the first threshold and the second threshold are determined by optimizing the membership distribution; aggregate the multiple clustering models based on the weights of the clustering models to obtain a global model, and send the global model to the second member device; The second member device is further used to adjust the global model to generate a local personalized model.

9. A federated learning device, characterized in that: The device is applied to a first member device, and includes: A processing module, used to initialize multiple clustering models; A communication module, used for sending the multiple clustering models to the second member device, so that the second member device uses local data to train the multiple clustering models respectively to obtain the membership of each clustering model, where the membership is used to indicate the matching degree between the local data and the clustering model; The communication module is further used to receive membership information from the second member device, wherein the membership information includes a membership value between each clustering model and each data point in the local data set; The processing module is further used to compare the membership value with a predetermined first threshold and a second threshold, and allocate the data points provided by the second member device to different areas, wherein the different areas represent different degrees of association between the data points and the cluster center, and the first threshold and the second threshold are determined by optimizing the membership distribution; The processing module is further used to aggregate the multiple clustering models based on the weights of the clustering models to obtain a global model; The communication module is also used to send the global model to the second member device so that the second member device adjusts the global model to generate a local personalized model, and the weight of each clustering model is determined based on the number of data points and membership degrees in different regions.

10. A federated learning device, characterized in that: The device is applied to a second member device, and includes: A communication module, configured to receive a plurality of clustering models from the first member device; A processing module, used to use local data to train the multiple clustering models respectively to obtain the membership of each clustering model, wherein the membership is used to indicate the matching degree between the local data set and the clustering model; The communication module is further used to send membership information to the first member device, wherein the membership information includes a membership value between each clustering model and each data point in the local data set; The communication module is further used to receive the global model from the first member device; The processing module is further used to adjust the global model to generate a local personalized model, where the global model is obtained by aggregating the multiple clustering models based on the number of data points in different regions, the degree of membership and the weight of each region by the first member device.

11. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to cause the computer to execute the method according to any one of claims 1 to 7.

13. A computer program product, characterized in that When the computer program product is called by a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images