A client screening supplement method based on federated learning

Abstract

The present invention discloses a client screening and supplementing method based on federated learning, comprising: a parameter server receives clients that have met the local update round under a predetermined time constraint as the first part of an aggregated subset; the distribution of sample labels of each client is measured by an indicator, and on the basis of the first part of the aggregated subset, the encrypted transmission of the indicator information is realized by combining the random summation between the clients and the asymmetric encryption technology, and the average indicator information of the first part of the aggregated subset is calculated and uploaded to the parameter server; the parameter server sends the average indicator information as query information to the remaining clients, selects the remaining clients whose own indicator information is less than or equal to the average indicator information as supplementary clients, and combines and supplements with the clients of the first part to form a complete aggregated subset. The present invention can effectively improve the convergence efficiency of global model update under the federated learning framework.

Description

Technical Field

[0001] The present invention relates to the field of computer science and technology, and in particular to a client screening and supplementation method based on federated learning. Background Art

[0002] The data heterogeneity between different data sets in federated learning is mainly reflected in the imbalance of data distribution. The deviation caused by the imbalance of label distribution is the main problem faced by federated learning, which is mainly manifested in the uneven distribution of the number of samples and labels in different clients. If the distribution of sample labels in the aggregated subset is not controlled and only the sample size is increased, the global model update direction will be offset due to the serious imbalance in the number of different sample labels, reducing the update quality of the global model. Therefore, in response to the above problems, it is necessary to design corresponding screening and supplementation strategies to alleviate the impact on the model convergence efficiency.

[0003] However, how to design a client screening and supplementation strategy suitable for the federated learning framework still faces certain challenges: first, the client scale in the federated learning framework is large, and the process of screening and evaluating each client is likely to prolong the duration of the supplementation phase, thereby reducing the interaction efficiency in the federated learning framework; second, federated learning has high requirements for data security and privacy protection, so it is impossible to directly evaluate the client's contribution from the data information obtained, which makes the design of the screening index subject to the constraints of privacy protection. Therefore, under the premise of ensuring data privacy security, how to select a suitable client subset to participate in the update of the global model within a limited waiting time is the key to the design of the screening and supplementation strategy. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a client screening supplement method based on federated learning, which evaluates the data quality of each client from the perspective of label distribution while ensuring data privacy and security, so as to further optimize the data combination quality of the aggregated subset.

[0005] To solve the above technical problems, the embodiments of the present invention provide the following solutions:

[0006] A client screening supplement method based on federated learning includes the following steps:

[0007] The parameter server receives the clients that have satisfied the local update round under the predetermined time constraint as the first part of the aggregated subset;

[0008] The distribution of sample labels of each client is measured through indicators. Based on the aggregated subset in the first part, the encrypted transmission of indicator information is achieved through the combination of random summation between clients and asymmetric encryption technology. The average indicator information of the aggregated subset in the first part is calculated and uploaded to the parameter server.

[0009] The parameter server sends the average index information as query information to the remaining clients, selects the remaining clients whose own index information is less than or equal to the average index information as supplementary clients, and combines and supplements them with the clients in the first part to form a complete aggregation subset.

[0010] Preferably, federated learning adopts a topology architecture centered on a parameter server. Based on the idea that data is not transmitted out, model gradients or model parameters are used instead of source data to be transmitted between data sources, and the parameter server is used to update and optimize the global aggregation model.

[0011] Preferably, the measuring the distribution of sample labels of each client by an indicator includes:

[0012] The indicator w uses the independent and identically distributed sample label distribution ratio as a reference, and calculates the difference between the local data label ratio distribution of each client and it, and uses the calculation result as a measure of the sample label distribution of each client. The calculation formula is as follows:

[0013]

[0014] Among them, C represents the classification problem of C categories, y represents the label space, and p k represents the distribution ratio between each label in the sample data set of client k, and p represents the distribution ratio between each label in the independent and identically distributed data set.

[0015] Preferably, the communication architecture centered on the parameter server is converted into a point-to-point transmission communication mode between clients. The process of calculating the average indicator information by combining random addition and asymmetric encryption technology between clients is as follows:

[0016] Client 1 generates a random number locally, adds it to the indicator information w1 that measures its own label distribution, and encrypts the calculation result with the public key of client 2 and sends it to client 2.

[0017] Client 2 uses its local private key to decrypt the data obtained from client 1, adds it to its own indicator information w2, encrypts the calculation result using the public key of client 3, and then sends it to client 3;

[0018] Client 3 continues to perform the decryption calculation as in the previous step and continues to send the encryption result to the next client;

[0019] After the final result is sent to the last client, the last client decrypts the received data, adds it to its own indicator information, and then encrypts the calculation result with the public key of client 1 and sends the encrypted result to client 1.

[0020] At this moment, client 1 receives the sum of the indicator information of all clients, decrypts the result with its own private key, and finally calculates the average value and sends it to the parameter server.

[0021] Preferably, the remaining clients to be screened and supplemented are only used to receive the average indicator information of the first part of the aggregated subset, and compare and judge with the screening threshold locally in parallel, without uploading their own indicator information.

[0022] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0023] In an embodiment of the present invention, a client screening and supplementation method based on federated learning is proposed, which evaluates the quality of local data sets of different clients from the perspective of measuring label distribution deviation to screen the remaining clients, so that after each client receives the average indicator query information from the parameter server, it can compare with the screening threshold in parallel locally and decide whether it can be called a supplementary client, thereby avoiding the impact on the efficiency of federated learning interaction caused by the large client scale; in terms of privacy protection, through the combination of random addition and asymmetric encryption technology between clients, the encrypted transmission of indicator information is realized, effectively avoiding the leakage of label distribution deviation information. Therefore, under the premise of ensuring data privacy security, the present invention optimizes the aggregate subset combination from the quality of the sample combination in the aggregate subset, effectively improving the convergence efficiency of global model update under the federated learning framework. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a flowchart of a client screening and supplementing method based on federated learning provided by an embodiment of the present invention;

[0026] Figure 2It is a schematic diagram of calculating average index information by combining random summation between clients and asymmetric encryption technology provided by an embodiment of the present invention.

[0027] As shown in the figure, in order to clearly implement the structure of the embodiment of the present invention, specific structures and devices are marked in the figure, but this is only for illustrative purposes and is not intended to limit the present invention to the specific structure, device and environment. According to specific needs, ordinary technicians in this field can adjust or modify these devices and environments, and the adjustments or modifications made are still included in the scope of protection of the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The embodiment of the present invention provides a client screening supplement method based on federated learning, such as Figure 1 As shown, the method comprises the following steps:

[0030] Step 1: The parameter server receives clients that have satisfied the local update round under the predetermined time constraint as the first part of the aggregated subset.

[0031] In an embodiment of the present invention, federated learning adopts a client-server architecture. The framework adopts a topology architecture centered on a parameter server. Based on the idea that data is not transmitted out, model gradients or model parameters are used instead of source data to be transmitted between data sources, and the parameter server realizes the update and optimization of the global aggregation model.

[0032] Step 2: Measure the distribution of sample labels of each client through indicators. Based on the aggregated subset in the first part, realize the encrypted transmission of indicator information through the combination of random addition between clients and asymmetric encryption technology, and calculate the average indicator information of the aggregated subset in the first part and upload it to the parameter server.

[0033] Step 2.1: The distribution of sample labels of each client is measured by indicators, including:

[0034] The indicator w uses the independent and identically distributed sample label distribution ratio as a reference, and calculates the difference between the local data label ratio distribution of each client and it, and uses the calculation result as a measure of the sample label distribution of each client. The calculation formula is as follows:

[0035]

[0036] Among them, C represents the classification problem of C categories, y represents the label space, and p k represents the distribution ratio between each label in the sample data set of client k, and p represents the distribution ratio between each label in the independent and identically distributed data set.

[0037] Step 2.2: Convert the communication architecture centered on the parameter server into a point-to-point transmission communication method between clients, and calculate the average indicator information by combining random summation between clients and asymmetric encryption technology. The schematic diagram is as follows Figure 2 The process is as follows:

[0038] (1) Client 1 generates a random number locally and adds it to the indicator information w1 that measures its own label distribution. It also encrypts the calculation result with the public key of client 2 and sends it to client 2.

[0039] (2) Client 2 uses its local private key to decrypt the data obtained from client 1, adds it to its own indicator information w2, encrypts the calculation result using the public key of client 3, and then sends it to client 3;

[0040] (3) Client 3 continues to perform the decryption calculation as in the previous step and continues to send the encryption result to the next client;

[0041] (4) After the final result is sent to the last client, the last client decrypts the received data, adds it to its own indicator information, and then encrypts the calculation result with the public key of client 1 and sends the encrypted result to client 1;

[0042] (5) At this moment, client 1 receives the sum of the indicator information of all clients, decrypts the result with its own private key, and finally calculates the average value and sends it to the parameter server.

[0043] Step 3: The parameter server sends the average indicator information calculated based on the first part of the aggregated subset as query information to the remaining clients, and starts further screening the remaining clients. The remaining clients whose own indicator information is less than or equal to the average indicator information are selected as supplementary clients, uploaded to the parameter server, and combined and supplemented with the clients in the first part to form a complete aggregated subset.

[0044] Among them, the remaining clients to be screened and supplemented are only used to receive the average indicator information of the first part of the aggregated subset, and compare and judge with the screening threshold locally in parallel, without uploading their own indicator information.

[0045] In summary, the client screening and supplement method based on federated learning provided by the present invention evaluates the degree of label distribution deviation of local data sets of different clients from the distribution information of the labels, and on the basis of the aggregated subset in the first part, selects the remaining clients with less than or equal to the average index information as selected clients to supplement the aggregated subset of the first stage. While expanding the sample size, the label distribution deviation in the sample combination is reduced by lowering the index value of the aggregated subset, so as to select better clients to optimize the data combination quality in the aggregated subset, thereby effectively evaluating the data quality of the local data set to achieve the purpose of screening the remaining clients, and effectively improving the convergence efficiency of the global model update under the federated learning framework.

[0046] It should be noted that the references to "one embodiment", "embodiment", "exemplary embodiment", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0047] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0048] As used herein, the term "nominal / nominal" refers to an expected or target value for a characteristic or parameter of a component or process operation set during the design phase of a production or manufacturing process, as well as a range of values ​​above and / or below the expected value. The range of values ​​may be due to slight variations in the manufacturing process or tolerances. As used herein, the term "approximately" indicates a value of a given quantity that may vary based on a particular technology node associated with the subject semiconductor device. Based on a particular technology node, the term "approximately" may indicate a value of a given quantity that varies, for example, within 5%-15% of the value (e.g., ±5%, ±10%, or ±15% of the value).

[0049] It will be understood that the meaning of “on,” “over,” and “above” in this disclosure should be interpreted in the broadest manner, so that “on” means not only “directly on” something, but also includes the meaning of being “on” something with intervening features or layers therebetween, and “on” or “over” means not only “on” or “above” something, but also includes the meaning of being “on” or “above” something with no intervening features or layers therebetween.

[0050] Additionally, spatially relative terms such as "under," "beneath," "lower," "above," "upper," and the like may be used herein for descriptive convenience to describe the relationship of one element or feature to another element or features, as shown in the accompanying drawings. Spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the accompanying drawings. The device may be oriented in other ways, and the spatially relative descriptors used herein may be similarly interpreted accordingly.

[0051] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0052] A person skilled in the art will appreciate that all or part of the steps in the above-mentioned embodiment method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc.

[0053] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A client screening and supplementation method based on federated learning, characterized in that: The following steps are involved: The parameter server receives the clients that have satisfied the local update round under the predetermined time constraint as the first part of the aggregated subset; The distribution of sample labels of each client is measured through indicators. Based on the aggregated subset in the first part, the encrypted transmission of indicator information is achieved through the combination of random summation between clients and asymmetric encryption technology. The average indicator information of the aggregated subset in the first part is calculated and uploaded to the parameter server. The distribution of sample labels of each client is measured by indicators, including: The indicator w uses the independent and identically distributed sample label distribution ratio as a reference, and calculates the difference between the local data label ratio distribution of each client and it, and uses the calculation result as a measure of the sample label distribution of each client. The calculation formula is as follows: Among them, C represents the classification problem of C categories, y represents the label space, and p k represents the distribution ratio between each label in the sample data set of client k, and p represents the distribution ratio between each label in the independent and identically distributed data set; The communication architecture centered on the parameter server is converted into a point-to-point transmission communication method between clients. The process of calculating the average indicator information by combining random summation between clients and asymmetric encryption technology is as follows: Client 1 generates a random number locally, adds it to the indicator information w1 that measures its own label distribution, and encrypts the calculation result with the public key of client 2 and sends it to client 2. Client 2 uses its local private key to decrypt the data obtained from client 1, adds it to its own indicator information w2, encrypts the calculation result using the public key of client 3, and then sends it to client 3; Client 3 continues to perform the decryption calculation as in the previous step and continues to send the encryption result to the next client; After the final result is sent to the last client, the last client decrypts the received data, adds it to its own indicator information, and then encrypts the calculation result with the public key of client 1 and sends the encrypted result to client 1. At this moment, client 1 receives the sum of the indicator information of all clients, decrypts the result with its own private key, and finally calculates the average value and sends it to the parameter server; The parameter server sends the average index information as query information to the remaining clients, selects the remaining clients whose own index information is less than or equal to the average index information as supplementary clients, and combines and supplements them with the clients in the first part to form a complete aggregation subset.

2. The client screening and supplementation method based on federated learning according to claim 1, characterized in that: Federated learning adopts a topology architecture centered on a parameter server. Based on the idea that data is not transmitted out, model gradients or model parameters are used instead of source data to be transmitted between data sources, and the parameter server is used to update and optimize the global aggregation model.

3. The client screening and supplementation method based on federated learning according to claim 1 is characterized in that: The remaining clients to be screened and supplemented are only used to receive the average indicator information of the first part of the aggregated subset, and compare and judge with the screening threshold locally in parallel, without uploading their own indicator information.

Images