Federal learning method and device based on random quantization, electronic equipment and medium

Abstract

The application discloses a federated learning method and device based on random quantization, electronic equipment and medium. Through the application of the technical solution, the client can first perform random quantization processing on the model data before transmitting the model data to the server, thereby obtaining quantization data with smaller data quantity relative to the model data. So that the subsequent purpose of global model aggregation is realized by transmitting the quantization data to the server. Thus, on the one hand, the communication overhead generated by the client in the related art federated learning when synchronizing the model data to the server is reduced. On the other hand, the problem that the user privacy is leaked once the original model data is intercepted by a malicious user is also avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to data processing technology, and particularly relates to a federated learning method based on random quantization, a device, an electronic equipment and a medium. BACKGROUND

[0002] Federated learning is a popular distributed machine learning paradigm in which multiple clients can collaborate to train a machine learning model while protecting data privacy by exchanging only intermediate learning results. Due to its advantages such as efficient decentralized model training and data privacy, federated learning has been applied to various privacy-sensitive fields, including healthcare, finance and smart cities.

[0003] However, the federated learning method in the related art still has certain disadvantages. For example, it is vulnerable to attacks by malicious users during the model aggregation process. Moreover, since the decentralized clients need to frequently transmit model data updates to the server to achieve global convergence, this also increases the communication overhead of the system. SUMMARY

[0004] Embodiments of the present application provide a federated learning method based on random quantization, a device, an electronic equipment and a medium. Thus, the problem that federated learning is vulnerable to attacks by malicious users during the model aggregation process in the related art is solved.

[0005] According to an aspect of the embodiments of the present application, a federated learning method based on random quantization is provided, which includes:

[0006] The multiple clients respectively perform local model training on the initial model issued by the server to obtain trained model data;

[0007] The client uses a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data;

[0008] Each client transmits the quantization data to the server, so that the server performs global model updating according to the received multiple quantization data.

[0009] Optionally, in another embodiment based on the above method of the present application, the client uses a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data, which includes:

[0010] The client uses a random quantization module to perform random quantization processing on the model data based on the quantization interval issued by the server to obtain quantization data corresponding to the model data.

[0011] Optionally, in another embodiment based on the above method of the application, the client uses a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data, including:

[0012] The client uses a random quantization module to perform random quantization processing on the model parameters to obtain a first discrete grid corresponding to the model parameters.

[0013] Or,

[0014] The client uses a random quantization module to perform random quantization processing on the gradient value to obtain a second discrete grid corresponding to the gradient value.

[0015] Optionally, in another embodiment based on the above method of the application, the server performs global model updating according to the received multiple quantization data, including:

[0016] The server calculates a quantization data average value based on the multiple quantization data.

[0017] The server performs global model updating on the local model based on the quantization data average value.

[0018] Optionally, in another embodiment based on the above method of the application, after the server performs global model updating according to the received multiple quantization data, it further includes:

[0019] The server determines the accuracy rate of the client training the local model using the quantization data transmitted by the client.

[0020] Based on the size relationship between the accuracy rate and the preset threshold value, an updated quantization interval is generated and sent to the client, so that the client subsequently performs random quantization processing on the model data based on the updated quantization interval.

[0021] According to another aspect of the embodiments of the application, a federal learning device based on random quantization is provided, including:

[0022] The sending module is configured to perform local model training on the initial model sent by the server by multiple clients to obtain trained model data.

[0023] The processing module is configured to use a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data.

[0024] The aggregation module is configured to transmit the quantization data to the server by each client, so that the server performs global model updating according to the received multiple quantization data.

[0025] According to still another aspect of the embodiments of the present application, an electronic device is provided, comprising:

[0026] a memory for storing executable instructions; and

[0027] a display for executing the executable instructions with the memory to complete the operations of any of the above-described federated learning methods based on random quantization.

[0028] According to still another aspect of the embodiments of the present application, a computer-readable storage medium is provided for storing computer-readable instructions, which when executed perform the operations of any of the above-described federated learning methods based on random quantization.

[0029] In the present application, a plurality of clients respectively perform local model training on an initial model issued by a server to obtain trained model data; the clients use a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data; and each client transmits the quantization data to the server so that the server performs global model updating based on the received plurality of quantization data.

[0030] By applying the technical solutions of the present application, the model data can be first randomly quantized by the client before being transmitted to the server, thereby obtaining quantization data with a smaller data volume relative to the model data. This enables subsequent transmission of the quantization data to the server to achieve the purpose of global model aggregation. Thus, on the one hand, the communication overhead generated by the client in synchronizing the model data with the server in the federated learning of the related art is reduced. On the other hand, the problem of user privacy being leaked if the original model data is intercepted by a malicious user is also avoided.

[0031] The technical solutions of the present application will be described in further detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings, which form a part of the specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the present application.

[0033] The present application can be more clearly understood with reference to the following detailed description in conjunction with the accompanying drawings, in which:

[0034] Figure 1 A federated learning method based on random quantization is provided for the present application;

[0035] Figure 2 A system architecture diagram of the federated learning method in the related art is provided for the present application;

[0036] Figure 3 A flow chart of a federated learning method based on random quantization proposed in the present application;

[0037] Figure 4 A structural schematic diagram of an electronic device proposed in the present application;

[0038] Figure 5 A structural schematic diagram of an electronic device proposed in the present application. DETAILED DESCRIPTION

[0039] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement, numerical expressions, and numerical values of the components and steps set forth in these embodiments are not limiting to the scope of the present application unless specifically stated otherwise.

[0040] At the same time, it should be understood that, for the convenience of description, the sizes of various parts shown in the drawings are not drawn in accordance with the actual proportional relationship.

[0041] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way limiting to the scope of the application or its applications or uses.

[0042] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the specification, where appropriate.

[0043] It should be noted that similar reference numbers and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0044] In addition, the technical solutions of various embodiments of the present application can be combined with each other, but it must be based on the fact that a person of ordinary skill in the art can realize it, and when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor is it within the scope of protection claimed by the present application.

[0045] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative positional relationship, motion condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications will also change accordingly.

[0046] The following will be described in conjunction with Figures 1-3A method for federated learning based on random quantization according to an example embodiment of the present application will be described. It should be noted that the following application scenarios are only shown for the purpose of facilitating the understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this respect. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0047] The present application also proposes a method, device, electronic equipment and medium for federated learning based on random quantization.

[0048] Figure 1 A flowchart of a method for federated learning based on random quantization according to an embodiment of the present application is schematically shown. As shown in Figure 1 The method comprises:

[0049] S101, a plurality of clients respectively perform local model training on an initial model issued by a server to obtain trained model data.

[0050] S102, the client uses a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data.

[0051] S103, each client transmits the quantization data to the server, so that the server performs global model updating according to the received plurality of quantization data.

[0052] In the related art, federated learning is a popular distributed machine learning paradigm in which multiple entities can collaborate to train a machine learning model while protecting data privacy by exchanging only intermediate learning results. Due to its advantages of efficient decentralized model training and data privacy, federated learning has been applied to various privacy-sensitive fields, including healthcare, finance, and smart cities.

[0053] Further, traditional machine learning algorithms require users to upload local data to a cloud server with high computing power for centralized training, which leads to uncontrollable data flow and sensitive data leakage. Federated learning technology was proposed in 2016, allowing users to protect user privacy during machine learning while sharing training data without local data aggregation.

[0054] In one way, federated learning can be used to train a model by clients (such as tablets, mobile phones, and Internet of Things devices) under the coordination of a central server (such as a service provider), where the client is responsible for training local data to obtain a local model. The central server is responsible for weighted aggregation of the gradients or models uploaded by the clients to obtain a global model, and after multiple iterations, a model close to the result of centralized machine learning is finally obtained, effectively reducing the many privacy risks brought by local data aggregation in traditional machine learning.

[0055] As an example, as shown in Figure 2 The process of federated learning can include the following steps, taking the model data as the gradient value for example:

[0056] 1) Global model synchronization, the client downloads the initial model w m from the center server.

[0057] 2) Client training on the initial model, local update, client i trains the local data to obtain the locally generated gradient (local model update of the mth communication of the i th client), each party uploads the trained gradient to the center server.

[0058] 3) Global model update, the center server receives the data of each party and performs a weighted aggregation operation to update the global model (that is, the global model update of the mth communication).

[0059] Further, as can be seen from the above steps, although the original training data is saved locally on the client, federated learning is still vulnerable to data attacks because the transmitted local gradient can leak information about the original data. For example, an attacker can access the local gradient and try to recover the original data.

[0060] In other words, the gradient information and other model data uploaded by the client are vulnerable to various attacks, and when the number of users increases and the model becomes more complex, frequent interaction with the server will inevitably result in a large amount of communication overhead.

[0061] Based on the above problems, the present application proposes a federated learning method based on random quantization, the scheme of which is that the client first performs random quantization processing on the model data before transmitting it to the server, thereby obtaining quantization data with smaller data volume relative to the model data. To achieve the purpose of global model aggregation by transmitting the quantization data to the server.

[0062] Further, the present application will be described in detail in combination with Figure 3 the scheme:

[0063] Step 1, a plurality of clients respectively perform local model training on the initial model issued by the server to obtain trained model data. Then go to step 2a or step 2b.

[0064] Step 2a, the client uses a random quantization module to perform random quantization processing on the model parameters based on the quantization interval issued by the server, to obtain a first discrete grid corresponding to the model parameters. Then go to step 3.

[0065] In one mode, the present application does not specifically limit the model data transmitted by the client to the server. For example, it can be a gradient value, or a model parameter, etc.

[0066] Specifically, for model data (whether it is a gradient or a model parameter), it can usually be represented by a floating-point number, so there is usually a large communication overhead in transmitting model data.

[0067] In summary, regardless of which parameter the model data is, the embodiments of the present application need to be randomly quantized before being transmitted to the server. Thus, on the one hand, it ensures that the amount of quantized data transmitted is small, and on the other hand, it also avoids the problem of user security risks caused by the theft of original model data by malicious users.

[0068] Further, compared with ordinary quantization, random quantization determines the specific quantization value according to probability. Random quantization first determines the quantization value according to the size of the quantization interval, and then for a certain original data, the closer it is to a certain quantization value, the more likely it is to be quantized to this point. On the contrary, the farther away from a quantization endpoint, the less likely it is to be quantized to this point.

[0069] Therefore, compared with ordinary quantization, it also leads to a better protection effect on model data due to the increase in randomness.

[0070] In one mode, after obtaining the trained model data in the client, the embodiments of the present application can perform random quantization processing on a certain sub-data (such as a certain model parameter or a local gradient value). Or all model data (such as all model parameters or the entire gradient value) can be randomly quantized.

[0071] Step 2b, the client uses the random quantization module to perform random quantization processing on the gradient value based on the quantization interval issued by the server, to obtain a second discrete grid corresponding to the gradient value. Then go to step 3.

[0072] In one mode, taking the local gradient value as an example, the process of using the random quantization module to perform random quantization processing on the gradient value by the present application is as follows:

[0073] Wherein, the client can use the random quantization module Q(·) to quantize the local gradient to to reduce the size of the transmitted data and protect data privacy.

[0074] It can be understood that since the gradient of each neuron of the learning model is element-wise, the local gradient of the client i in the mth round can be represented as:

[0075]

[0076] wherein, is the local gradient of the client i at the j-th neuron in the m-th round, K is the total number of neurons of the local model w of the client.

[0077] Further, the present application defines δ as the quantization interval of the random quantization function Q(·), each local gradient falls into the interval [I u δ, I u+1 δ) interval, wherein (3 / 2, the value is 1), (3 / 2, the value is 2) is the integral exponent of the interval.

[0078] In summary, the discrete grid corresponding to the random quantization (quantization interval δ) can be represented as c u = I u δ and c u+1 = I u+1 δ, respectively.

[0079] Finally, for its result after random quantization is given as:

[0080]

[0081] wherein, w.p. represents probability, sign() represents extracting the sign of.

[0082] It should be noted that the random quantization function Q(·) is element-wise, that is, the quantization of the vector at the client i in the m-th round is according to the above equation to quantize all elements of each neuron j. Unlike the classical quantization that maps a real number to the nearest discrete value, the random quantization (with a predefined probability distribution) randomly generates one of the two endpoints of the interval into which the input falls.

[0083] Step 3, each client transmits the quantized data to the server respectively.

[0084] In one way, the original model data is generally represented by a 32-bit floating point number. However, after random quantization, the client only needs to transmit a discrete grid , which reduces the communication overhead of model data synchronization in federated learning.

[0085] Step 4, the server calculates the average value of the quantized data based on the plurality of quantized data.

[0086] ​Step 5, the server updates the global model based on the quantized data average value.

[0087] In one way, after the local training process of each client with the added random quantization module ends, each client i uses the random quantization module to quantize the training data of the trained model and sends the quantized data to the server for global aggregation.

[0088] Further, the server can update the global model according to the equation It can be understood that, since the added random quantization module for the client has low complexity (only part of the model data can be quantized) and does not need to interact with other clients, the communication overhead in the federated learning process can be reduced.

[0089] It can be understood that, for an attacker, even if the quantized data is intercepted, the malicious user can only know the range of the original model data , and the original data is not directly leaked.

[0090] Step 6, the server determines the accuracy of the client in training the local model using the quantized data transmitted by the client. Then go to step 7a or step 7b.

[0091] In one way, the embodiments of the present application can select a set of quantization intervals from the server before the client trains the local model.

[0092] Further, after the training process ends and the model data is obtained, the server can select an updated quantization interval set for the client according to the accuracy of the calculation derived from the model data. To dynamically adjust the random quantization interval, select a quantization interval that balances privacy protection, accuracy, and communication overhead, and achieve the purpose of balancing the three.

[0093] For example, when the accuracy of the client in training the local model is determined to be low (i.e., less than a preset threshold), an updated quantization interval with a smaller numerical range relative to the original quantization interval can be fed back to the client. To make the client subsequently quantize the model data from the updated quantization interval with a smaller numerical range, the data precision of the quantized data obtained by the client will also be relatively more accurate. Conversely, the server feeds back an updated quantization interval with a larger numerical range relative to the original quantization interval to the client.

[0094] Step 7a, if the server determines that the accuracy of the client training the local model is less than the preset threshold, the server generates an updated quantization interval with a smaller numerical range, and sends the updated quantization interval to the client.

[0095] Step 8a, the client subsequently performs random quantization processing on the model data based on the updated quantization interval with a smaller numerical range.

[0096] Step 7b, if the server determines that the accuracy of the client training the local model is not less than the preset threshold, the server generates an updated quantization interval with a larger numerical range, and sends the updated quantization interval to the client.

[0097] Step 8b, the client subsequently performs random quantization processing on the model data based on the updated quantization interval with a larger numerical range.

[0098] In this application, a plurality of clients respectively train the initial model sent by the server to obtain trained model data; the client uses a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data; each client transmits the quantization data to the server, so that the server updates the global model according to the received plurality of quantization data.

[0099] By applying the technical solution of the present application, the client can first perform random quantization processing on the model data before transmitting it to the server, thereby obtaining a quantization data with a smaller data size relative to the model data. This enables subsequent transmission of the quantization data to the server to achieve the purpose of global model aggregation. On the one hand, it reduces the communication overhead generated by the client synchronizing model data to the server in the federated learning of related technologies. On the other hand, it also avoids the problem of user privacy being leaked if the original model data is intercepted by a malicious user.

[0100] Optionally, in another embodiment based on the above method of the present application, the client uses a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data, including:

[0101] The client uses a random quantization module to perform random quantization processing on the model data based on the quantization interval sent by the server to obtain quantization data corresponding to the model data.

[0102] Optionally, in another embodiment based on the above method of the present application, the client uses a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data, including:

[0103] The client uses a random quantization module to perform random quantization processing on the model parameters to obtain a first discrete grid corresponding to the model parameters.

[0104] Or,

[0105] The client uses a random quantization module to perform random quantization processing on the gradient value to obtain a second discrete grid corresponding to the gradient value.

[0106] Optionally, in another embodiment based on the above method of the application, the server performs global model updating according to the received plurality of quantization data, comprising:

[0107] The server calculates the average value of the quantization data based on the plurality of quantization data;

[0108] The server performs global model updating on the local model based on the average value of the quantization data.

[0109] Optionally, in another embodiment based on the above method of the application, after the server performs global model updating according to the received plurality of quantization data, further comprising:

[0110] The server determines the accuracy of the client training the local model using the quantization data transmitted by the client;

[0111] Based on the size relationship between the accuracy and the preset threshold, an updated quantization interval is generated and sent to the client, so that the client subsequently performs random quantization processing on the model data based on the updated quantization interval.

[0112] In combination with the above, the application designs a federated learning framework capable of reducing communication overhead, reducing time complexity, and avoiding signal interaction between clients. By performing random quantization processing on the trained model data, the performance is guaranteed, and the security and communication efficiency of federated learning are realized with minimal implementation overhead (time complexity and signal).

[0113] By applying the technical solution of the application, the client can first perform random quantization processing on the model data before transmitting it to the server, thereby obtaining a quantization data with smaller data quantity relative to the model data. This enables subsequent transmission of the quantization data to the server to achieve the purpose of global model aggregation. Thus, on the one hand, the communication overhead generated by the client synchronizing model data to the server in the related art federated learning is reduced. On the other hand, the problem of user privacy being leaked if the original model data is intercepted by a malicious user is also avoided.

[0114] Optionally, in another embodiment of the application, asFigure 4 The application also provides a federated learning device based on random quantization. The device comprises:

[0115] The sending module 201 is configured to perform local model training on the initial model issued by the server by the plurality of clients respectively, to obtain trained model data;

[0116] The processing module 202 is configured to perform random quantization processing on the model data by the random quantization module of the client, to obtain quantization data corresponding to the model data;

[0117] The aggregation module 03 is configured to transmit the quantization data to the server by each client respectively, so that the server performs global model updating according to the plurality of received quantization data.

[0118] By applying the technical solution of the application, the model data can be first quantized by the client before being transmitted to the server, so as to obtain quantization data with smaller data quantity relative to the model data. The subsequent purpose of global model aggregation can be achieved by transmitting the quantization data to the server. Thus, on the one hand, the communication overhead generated by the client in synchronizing the model data to the server in the federated learning of the related art is reduced. On the other hand, the problem of user privacy leakage caused by interception of the original model data by malicious users is avoided.

[0119] In another embodiment of the application, the processing module 202 is configured to:

[0120] The client performs random quantization processing on the model data by the random quantization module based on the quantization interval issued by the server, to obtain quantization data corresponding to the model data.

[0121] In another embodiment of the application, the processing module 202 is configured to:

[0122] The client performs random quantization processing on the model parameters by the random quantization module, to obtain a first discrete grid corresponding to the model parameters.

[0123] Or,

[0124] The client performs random quantization processing on the gradient value by the random quantization module, to obtain a second discrete grid corresponding to the gradient value.

[0125] In another embodiment of the application, the processing module 202 is configured to:

[0126] The server calculates the average value of the quantization data based on the plurality of quantization data;

[0127] The server performs global model updating on the local model based on the quantized data average value.

[0128] In another embodiment of the present application, the processing module 202 is configured to:

[0129] The server determines the accuracy of the client in training the local model using the quantized data transmitted by the client;

[0130] Based on the size relationship between the accuracy and the preset threshold, an updated quantization interval is generated, and the updated quantization interval is issued to the client, so that the client subsequently performs random quantization processing on the model data based on the updated quantization interval.

[0131] Figure 5 is a logical structure block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 300 can be an electronic device.

[0132] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, is also provided, which instructions are executable by an electronic device processor to complete the above-mentioned federated learning method based on random quantization, which method comprises: a plurality of clients respectively performing local model training on an initial model issued by a server to obtain trained model data; the client uses a random quantization module to perform random quantization processing on the model data to obtain quantized data corresponding to the model data; each client transmits the quantized data to the server, so that the server performs global model updating according to the received multiple quantized data.

[0133] Alternatively, the above instructions can also be executed by the processor of the electronic device to complete other steps involved in the above-mentioned exemplary embodiments. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0134] In an exemplary embodiment, an application program / computer program product is also provided, comprising one or more instructions executable by a processor of an electronic device to complete the above-mentioned federated learning method based on random quantization, which method comprises: a plurality of clients respectively performing local model training on an initial model issued by a server to obtain trained model data; the client uses a random quantization module to perform random quantization processing on the model data to obtain quantized data corresponding to the model data; each client transmits the quantized data to the server, so that the server performs global model updating according to the received multiple quantized data.

[0135] Optionally, the above instructions can also be executed by the processor of the electronic device to complete other steps involved in the above exemplary embodiments.

[0136] Figure 5 An example of the electronic device 300 is shown in FIG. 3. Those skilled in the art can understand that the schematic diagram of the electronic device 300 is only an example and does not constitute a limitation on the electronic device 300. The electronic device 300 can include more or fewer components than those shown in the figure, or combine certain components, or include different components, for example, the electronic device 300 can also include an input / output device, a network access device, a bus, etc. Figure 5 The electronic device 300 shown in FIG. 3 is only an example and does not constitute a limitation on the electronic device 300. The electronic device 300 can include more or fewer components than those shown in the figure, or combine certain components, or include different components, for example, the electronic device 300 can also include an input / output device, a network access device, a bus, etc.

[0137] The processor 302 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor 302 can also be any conventional processor. The processor 302 is the control center of the electronic device 300, and is connected to all parts of the electronic device 300 through various interfaces and lines.

[0138] The memory 301 can be used to store computer readable instructions 303. The processor 302 realizes various functions of the electronic device 300 by running or executing the computer readable instructions or modules stored in the memory 301, and calling the data stored in the memory 301. The memory 301 can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data created according to the use of the electronic device 300, etc. In addition, the memory 301 can include a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, a read-only memory (ROM), a random access memory (RAM) or other non-volatile / volatile storage devices.

[0139] The modules integrated in the electronic device 300, if implemented in the form of software function modules and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by computer readable instructions instructing relevant hardware, and the computer readable instructions can be stored in a computer readable storage medium. When the computer readable instructions are executed by a processor, the steps of the above-mentioned various method embodiments can be implemented.

[0140] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the application encompass any and all variations, uses, or adaptations of the application following the general principles thereof and including such modifications and changes as come within the scope of the application. The specification and examples are to be considered exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0141] It is to be understood that the application is not limited to the precise details of design and construction that have been described and illustrated herein, and that various modifications and changes can be made without departing from the scope of the application. The scope of the application is limited only by the claims that follow.

Claims

1. A federated learning method based on random quantization, characterized in that, The method comprises the following steps: a plurality of clients respectively perform local model training on an initial model issued by a server to obtain trained model data; the client uses a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data; each client transmits the quantization data to the server so that the server performs global model updating based on the received plurality of quantization data; wherein the quantization value is determined according to the size of the quantization interval, the farther the distance from the quantization endpoint, the smaller the probability of quantizing to the quantization endpoint, and the random quantization with a predefined probability distribution randomly generates one of the two endpoints of the interval into which the input model data falls; after the server performs global model updating based on the received plurality of quantization data, the server further determines the accuracy of the local model training performed by the client using the quantization data transmitted by the client, generates an updated quantization interval based on the size relationship between the accuracy and a preset threshold, and issues the updated quantization interval to the client so that the client subsequently performs random quantization processing on the model data based on the updated quantization interval.

2. The method of claim 1, wherein, the client uses a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data, which comprises: the client uses a random quantization module to perform random quantization processing on the model data based on the quantization interval issued by the server to obtain quantization data corresponding to the model data.

3. The method of claim 1 or 2, wherein, the client uses a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data, which comprises: the client uses a random quantization module to perform random quantization processing on the model parameters to obtain a first discrete grid corresponding to the model parameters. or the client uses a random quantization module to perform random quantization processing on the gradient value to obtain a second discrete grid corresponding to the gradient value.

4. The method of claim 1, wherein, the server performs global model updating based on the received plurality of quantization data, which comprises: the server calculates the average value of the quantization data based on the plurality of quantization data; the server performs global model updating on the local model based on the average value of the quantization data.

5. A federated learning device based on random quantization, characterized in that, The method comprises the following steps: a sending module configured to perform local model training on an initial model issued by a server by a plurality of clients to obtain trained model data; a processing module configured to use a random quantization module to perform random quantization processing on the model data to obtain quantization data corresponding to the model data; an aggregation module configured to transmit the quantization data to the server by each client so that the server performs global model updating based on the received plurality of quantization data; the device is further configured to determine the quantization value according to the size of the quantization interval, the farther the distance from the quantization endpoint, the smaller the probability of quantizing to the quantization endpoint, and the random quantization with a predefined probability distribution randomly generates one of the two endpoints of the interval into which the input model data falls; The server determines an accuracy of the client training the local model by using the quantization data transmitted by the client; Based on the size relationship between the accuracy and a preset threshold, an updated quantization interval is generated, and the updated quantization interval is issued to the client, so that the client subsequently performs random quantization processing on the model data based on the updated quantization interval.

6. An electronic device, comprising: Comprise: a memory for storing executable instructions; and, a processor for executing the executable instructions with the memory to complete the operation of the federated learning method based on random quantization in any one of claims 1-4.

7. A computer readable storage medium storing computer readable instructions, wherein, The instructions are executed to perform the operation of the federated learning method based on random quantization in any one of claims 1-4.

Images