Data processing method and device, equipment and storage medium
By encoding the original gradient data of floating point type into integer types and performing multi-level encryption, the problems of high computational complexity and high communication cost in federated machine learning scenarios are solved, efficient gradient aggregation and model updates are achieved, and performance bottlenecks are reduced.
Patent Information
- Application Number
- CN202411648586.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-18
- Publication Date
- 2025-03-14
AI Technical Summary
The existing encryption methods lead to high computational complexity and high communication costs in federated machine learning scenarios, which easily lead to performance bottlenecks.
By encoding the original gradient data of floating point type into coded gradient data of integer type and performing multi-level encryption, a second encryption gradient is generated, sent to adjacent second parties for gradient aggregation, and finally the model parameters are updated.
Improves encryption efficiency, reduces bandwidth consumption, reduces communication costs, avoids performance bottlenecks, and ensures privacy protection of gradient data.
Smart Images
Figure CN119622759A_ABST
Abstract
Description
Background Art
[0002] Each participant in federated machine learning scenarios such as government affairs, finance, and smart transportation obtains model gradients through local iterative training, and then encrypts the gradient information using encryption technologies such as homomorphic encryption, differential privacy, or secret sharing. The encrypted gradient is sent to the aggregation server, which aggregates the multiple encrypted gradients received and sends the aggregated gradient to each participant, allowing each participant to use the aggregated gradient to update the model.
[0003] Since this encryption method has high computational complexity and high communication cost, it is easy to produce performance bottlenecks. Therefore, this application proposes a new data processing method for federated machine learning scenarios. Summary of the invention
[0004] The embodiments of the present application provide a data processing method, apparatus, device and storage medium to solve the problems of high computational complexity, high communication cost and easy performance bottleneck caused by the current encryption method.
[0005] In a first aspect, an embodiment of the present application provides a data processing method, which is applied to multiple parties and includes:
[0006] The first participant is used to encode the original gradient data of floating point type in this round of iterative training into encoded gradient data of integer type, send the second encrypted gradient generated by multi-level encryption of the encoded gradient data to the adjacent second participant for gradient aggregation, and update the model parameters according to the target aggregated gradient obtained by the last second participant aggregation;
[0007] Each second participant except the first participant is used to aggregate and encrypt the dense gradient data of the previous participant and the first encrypted gradient of the second participant to obtain the target aggregated gradient, and update the model parameters according to the target aggregated gradient obtained by the last second participant aggregation; wherein the dense gradient information is the second encrypted gradient of the first participant or the target aggregated gradient generated by the previous second participant.
[0008] Optionally, encoding the original gradient data of the floating point type in the current round of iterative training into encoded gradient data of the integer type includes:
[0009] According to a preset precision value, the original gradient data of the floating point type is shifted left to generate shifted gradient data, and the shifted gradient data is rounded down to determine the number of paddings;
[0010] Based on a preset precision value, a preset base value and a number of padding bits, the original gradient data of the floating point type is converted into the encoded gradient data of the integer type.
[0011] Optionally, the second encrypted gradient is generated by performing the following operations:
[0012] Based on the homomorphic encryption public key, homomorphically encrypt the encoded gradient data converted into an integer type to generate the first encrypted gradient;
[0013] The first encrypted gradient is secondary encrypted based on a communication public key to generate the second encrypted gradient, wherein the communication public key is generated based on the task number and the task flow node number of the federated learning task.
[0014] Optionally, the aggregating and encrypting the encrypted gradient data of the previous participant and the first encrypted gradient of the second participant to obtain a target aggregated gradient includes:
[0015] Decrypting the encrypted gradient data of the previous participant based on the communication private key associated with the user private key of the second participant to obtain intermediate gradient data, where the intermediate gradient data is the first encrypted gradient of the first participant or the candidate aggregated gradient aggregated by the previous second participant;
[0016] Performing a semi-homomorphic addition operation on the intermediate gradient data and the first encrypted gradient of the second participant to obtain a candidate aggregate gradient;
[0017] The candidate aggregate gradient is secondary encrypted based on the communication public key to generate the target aggregate gradient.
[0018] Optionally, updating the model parameters according to the target aggregate gradient obtained by the last second participant aggregation includes:
[0019] Decrypting the target aggregated gradient obtained by the last second participant aggregation based on the communication private key associated with the user private key of the second participant to obtain the intermediate gradient data;
[0020] The intermediate gradient data is homomorphically decrypted, and based on a preset precision value, a preset base value and a number of padding bits, the integer type encoded aggregate gradient is inversely encoded into a floating point type original aggregate gradient, and the model parameters are updated according to the original aggregate gradient.
[0021] In a second aspect, an embodiment of the present application further provides a data processing device, which is applied to multiple parties, including:
[0022] The first participant unit is used to encode the original gradient data of the floating point type in this round of iterative training into the encoded gradient data of the integer type, send the second encrypted gradient generated by multi-level encryption of the encoded gradient data to the adjacent second participant for gradient aggregation, and update the model parameters according to the target aggregated gradient obtained by the last second participant aggregation;
[0023] The second participant unit is composed of second participants except the first participant, which aggregate and encrypt the dense gradient data of the previous participant and the first encrypted gradient of the second participant to obtain a target aggregate gradient, and update the model parameters according to the target aggregate gradient obtained by the last second participant aggregation; wherein the dense gradient information is the second encrypted gradient of the first participant or the target aggregate gradient generated by the previous second participant.
[0024] Optionally, the first participant unit is used to:
[0025] According to a preset precision value, the original gradient data of the floating point type is shifted left to generate shifted gradient data, and the shifted gradient data is rounded down to determine the number of paddings;
[0026] Based on a preset precision value, a preset base value and a number of padding bits, the original gradient data of the floating point type is converted into the encoded gradient data of the integer type.
[0027] Optionally, the first participant unit generates the second encrypted gradient by performing the following operations:
[0028] Based on the homomorphic encryption public key, homomorphically encrypt the encoded gradient data converted into an integer type to generate the first encrypted gradient;
[0029] The first encrypted gradient is secondary encrypted based on a communication public key to generate the second encrypted gradient, wherein the communication public key is generated based on the task number and the task flow node number of the federated learning task.
[0030] Optionally, the second participant unit is used to:
[0031] Decrypting the encrypted gradient data of the previous participant based on the communication private key associated with the user private key of the second participant to obtain intermediate gradient data, where the intermediate gradient data is the first encrypted gradient of the first participant or the candidate aggregated gradient aggregated by the previous second participant;
[0032] Performing a semi-homomorphic addition operation on the intermediate gradient data and the first encrypted gradient of the second participant to obtain a candidate aggregate gradient;
[0033] The candidate aggregate gradient is secondary encrypted based on the communication public key to generate the target aggregate gradient.
[0034] Optionally, the second participant unit is used to:
[0035] Decrypting the target aggregated gradient obtained by the last second participant aggregation based on the communication private key associated with the user private key of the second participant to obtain the intermediate gradient data;
[0036] The intermediate gradient data is homomorphically decrypted, and based on a preset precision value, a preset base value and a number of padding bits, the integer type encoded aggregate gradient is inversely encoded into a floating point type original aggregate gradient, and the model parameters are updated according to the original aggregate gradient.
[0037] In a third aspect, an embodiment of the present application further provides a computer device, comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of any one of the above-mentioned data processing methods.
[0038] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of any one of the above-mentioned data processing methods.
[0039] The beneficial effects of this application are as follows:
[0040] The embodiment of the present application provides a data processing method, apparatus, device and storage medium, which is applied to multiple participants, wherein the first participant is used to encode the original gradient data of the floating point type of the current round of iterative training into the encoded gradient data of the integer type, unify the data format, avoid floating point errors, solve the integer encoding problem for floating point data in the federated learning task, and then when the encoded gradient data is encrypted at multiple levels, the encryption efficiency can be improved and the bandwidth consumption can be reduced. After that, the generated second encrypted gradient is sent to the adjacent second participant for gradient aggregation, and the model parameters are updated according to the target aggregated gradient obtained by the last second participant aggregation;
[0041] Each second participant, except the first participant, is used to aggregate and encrypt the secret gradient data of the previous participant and the first encrypted gradient of the second participant to obtain the target aggregated gradient, and update the model parameters according to the target aggregated gradient obtained by the last second participant; wherein the secret gradient data is the second encrypted gradient of the first participant or the target aggregated gradient generated by the previous second participant. The participants in the federated learning task perform gradient aggregation based on the secret gradient data of the previous participant and their own first encrypted gradient, which not only protects the privacy of the gradient data, but also reduces the communication overhead and communication cost because only the aggregated gradient needs to be transmitted. The progressive aggregation method allows all participants to participate in the gradient aggregation process, avoiding the concentration of all gradients to a central node, sharing the computing resources occupied during aggregation and the risk of being attacked, thereby reducing the performance bottleneck of federated learning.
[0042] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0044] Figure 1A A schematic diagram of a flow chart of data processing of original gradient data of this round of iterative training by multiple participants provided in an embodiment of the present application;
[0045] Figure 1B A logical schematic diagram of data processing of original gradient data of this round of iterative training by multiple participants provided in an embodiment of the present application;
[0046] Figure 1C A schematic diagram of ciphertext data provided in an embodiment of the present application;
[0047] Figure 1D A schematic diagram of generating a communication public key and a communication private key based on a task number and a task flow node number provided in an embodiment of the present application;
[0048] Figure 2 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;
[0049] Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;
[0050] Figure 4 Schematic diagram of the structure of a computing device in an embodiment of the present application. DETAILED DESCRIPTION
[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.
[0052] The following is a brief introduction to the design concept of the embodiment of the present application:
[0053] Each participant in federated machine learning scenarios such as government affairs, finance, and smart transportation obtains model gradients through local iterative training, and then encrypts the gradient information using encryption technologies such as homomorphic encryption, differential privacy, or secret sharing. The encrypted gradient is sent to the aggregation server, which aggregates the multiple encrypted gradients received and sends the aggregated gradient to each participant, allowing each participant to use the aggregated gradient to update the model.
[0054] Since this encryption method has high computational complexity and high communication cost, it is easy to produce performance bottlenecks. In view of this, the embodiments of the present application provide a data processing method, device, equipment and storage medium. The method is applied to multiple participants, wherein the first participant is used to encode the original gradient data of the floating-point type of this round of iterative training into the encoded gradient data of the integer type, unify the data format, avoid floating-point errors, and solve the integer encoding problem for floating-point data in the federated learning task. Then, when the encoded gradient data is encrypted at multiple levels, the encryption efficiency can be improved and the bandwidth consumption can be reduced. After that, the generated second encrypted gradient is sent to the adjacent second participant for gradient aggregation, and the model parameters are updated according to the target aggregated gradient obtained by the last second participant aggregation;
[0055] Each second participant, except the first participant, is used to aggregate and encrypt the secret gradient data of the previous participant and the first encrypted gradient of the second participant to obtain the target aggregated gradient, and update the model parameters according to the target aggregated gradient obtained by the last second participant; wherein the secret gradient data is the second encrypted gradient of the first participant or the target aggregated gradient generated by the previous second participant. The participants in the federated learning task perform gradient aggregation based on the secret gradient data of the previous participant and their own first encrypted gradient, which not only protects the privacy of the gradient data, but also reduces the communication overhead and communication cost because only the aggregated gradient needs to be transmitted. The progressive aggregation method allows all participants to participate in the gradient aggregation process, avoiding the concentration of all gradients to a central node, sharing the computing resources occupied during aggregation and the risk of being attacked, thereby reducing the performance bottleneck of federated learning.
[0056] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.
[0057] Next, combine Figure 1A-1B The schematic diagram shown specifically introduces the process of multiple participants performing data processing on the original gradient data of this round of iterative training.
[0058] S101: The first participant encodes the original gradient data of the floating point type of this round of iterative training into integer type encoded gradient data, sends the second encrypted gradient generated by multi-level encryption of the encoded gradient data to the adjacent second participant for gradient aggregation, and updates the model parameters according to the target aggregated gradient obtained by the last second participant aggregation.
[0059] The first participant shifts the original floating-point gradient data W to the left according to the preset precision value p, generates shifted gradient data, and rounds down the shifted gradient data to determine the number of padding e. For example, the formula Z = (W × p) ÷ b e As shown, based on the preset precision value p, the preset base value b and the number of padding e, the original gradient data W of the floating point type is converted into the encoded gradient data Z of the integer type to adapt to the requirement that the input parameter is a large integer in the semi-homomorphic addition mechanism of the Paillier protocol.
[0060] After converting floating-point data to integer data, the data format can be unified, simplifying the subsequent data processing and transmission process. Floating-point numbers are prone to precision loss and rounding errors during calculations, while integer data does not have these problems. In scenarios that require precise calculations and transmissions, using integer types can avoid these errors. Moreover, floating-point numbers usually contain more redundant information, while integer data is more compact and easier to compress, reducing unnecessary transmission data and thus reducing the bandwidth required for transmission, which is particularly important in environments with limited network bandwidth.
[0061] Perform the following operations on the encoded gradient data to generate the second encrypted gradient: Based on the homomorphic encryption public key R of the Paillier protocol pk , homomorphically encrypt the encoded gradient data Z1 converted to integer type to generate the first encrypted gradient [Z1]. Since many encryption and transmission protocols support integer type data, converting floating point type data to integer type data can improve compatibility with other systems and avoid the possible representation differences of floating point numbers on different platforms; then based on the communication public key T pk Perform secondary encryption on the first encrypted gradient [Z1] to generate a second encrypted gradient [[Z1]].
[0062] Furthermore, under the premise of ensuring security, in order to reduce the number of data bits encrypted by the public key as much as possible and improve the calculation and transmission rate, the present application discloses the preset precision value p and the preset base value b, based on the communication public key T pk Perform secondary encryption on the first encrypted gradient [Z1], the preset precision value p, the preset base value b and the number of padding e to generate Figure 1C The second party uses the communication private key T skBy decrypting the ciphertext data, the first encrypted gradient and encoding parameters encrypted by the first party can be obtained, and then the number of padding e can be determined based on the preset precision value p and the preset base value b in the encoding parameters. Then, the preset precision value p, the preset base value b and the number of padding e can be used to generate the encoded gradient data Z2, thus efficiently completing the encoding process of the original gradient data in the federated learning process.
[0063] Moreover, the public key mechanism algorithm based on the national secret SM9 encrypts the coding parameters and the first encrypted gradient [Z1], [Z2], reducing the public key parameters. When the attacker only obtains the preset precision value p and the preset base value b, it is impossible to decrypt and obtain the correct number of padding e, and thus it is impossible to recover the original gradient data Z1, Z2.
[0064] The communication public key is generated based on the task ID of the federated learning task and the task flow node ID. Specifically, based on the national secret SM9 algorithm, the task ID of the federated learning task and the task flow node ID are concatenated to obtain Figure 1D The communication public key T shown pk With the communication private key T sk . After the communication public key T pk With the communication private key T sk While saving it locally, the communication public key T pk With the communication private key T sk Sent to each second party.
[0065] Each participant searches for the corresponding user private key U in the local public and private key library based on the national secret SM2 algorithm based on their respective user IDs. sk , and then as the formula [T sk ]=SM2(T sk ,U sk ), based on the respective user private key U sk Encrypt the communication private key T separately sk , the public communication key T pk The encrypted communication private key [T sk ] is saved in the security domain database of each node. In this way, after the system or network abnormality of each participant is restored, the task ID and the task flow node ID are used in the local database or requested from other nodes, and the communication private key T sk Recover data and encoding parameters after secure decryption.
[0066] This application combines the national secret SM2 algorithm and the national secret SM9 algorithm with the unique task information shared by all participants in the federated learning task, and performs multi-level encryption only on the encoded gradient data, thereby reducing the types of data to be encrypted, reducing communication costs, protecting the privacy of gradient data, and solving security issues such as access control and computational recovery of encrypted gradient data for federated learning tasks.
[0067] S102: Each second participant except the first participant is used to aggregate and encrypt the dense gradient data of the previous participant and the first encrypted gradient of the second participant to obtain a target aggregated gradient, and update the model parameters according to the target aggregated gradient obtained by the last second participant aggregation; wherein the dense gradient information is the second encrypted gradient of the first participant or the target aggregated gradient generated by the previous second participant.
[0068] Taking a second participant as an example, the process of generating a target aggregation gradient by executing aggregation and encryption steps is introduced.
[0069] The second participant is based on the user U of the second participant. sk The communication private key T associated with the private key sk , decrypt the previous participant's encrypted gradient data [[R1]] to obtain the intermediate gradient data, which is the first encrypted gradient [Z1] of the first participant or the candidate aggregated gradient [R1] aggregated by the previous second participant; then perform a semi-homomorphic addition operation on the intermediate gradient data and the second participant's first encrypted gradient [Z2] to obtain the candidate aggregated gradient [R2]; finally, perform secondary encryption on the candidate aggregated gradient based on the communication public key to generate the target aggregated gradient [[R2]].
[0070] By analogy, the last second participant is based on the dense gradient data of the previous participant [[R n-1 ]] and its first encryption gradient [Z n ] performs semi-homomorphic addition operations to aggregate the gradient data of all participants and obtain the candidate aggregate gradient [R n ]; Finally, the candidate aggregate gradient is encrypted twice based on the communication public key to generate the target aggregate gradient [[R n ]], n is the total number of parties.
[0071] Finally, the second participant generates the target aggregation gradient [[R n ]] is sent to the server, and then sent to each participant through the server, so that each participant can respectively calculate the target aggregation gradient [[R n ]]Update the model.
[0072] Taking a second participant as an example, the process of updating the model parameters is introduced. Based on the user private key U skThe associated communication private key T sk , decrypt the target aggregate gradient obtained by the last second participant aggregation [[R n ]], obtain the intermediate gradient data; then perform homomorphic decryption on the intermediate gradient data, and convert the integer type encoded aggregate gradient [R n ] The original aggregate gradient R is inversely encoded as a floating point type n , and according to the original aggregation gradient R n Update model parameters.
[0073] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a structural schematic diagram of a data processing device applied to multiple participants. Figure 2 As shown, the data processing device 200 may include:
[0074] The first participant unit 201 is used to encode the original gradient data of the floating point type in this round of iterative training into the encoded gradient data of the integer type, send the second encrypted gradient generated by multi-level encryption of the encoded gradient data to the adjacent second participants for gradient aggregation, and update the model parameters according to the target aggregated gradient obtained by the last second participant aggregation;
[0075] The second participant unit 202 is composed of each second participant except the first participant, which aggregates and encrypts the dense gradient data of the previous participant and the first encrypted gradient of the second participant to obtain a target aggregate gradient, and updates the model parameters according to the target aggregate gradient obtained by the last second participant aggregation; wherein the dense gradient information is the second encrypted gradient of the first participant or the target aggregate gradient generated by the previous second participant.
[0076] Optionally, the first participant unit 201 is used to:
[0077] According to a preset precision value, the original gradient data of the floating point type is shifted left to generate shifted gradient data, and the shifted gradient data is rounded down to determine the number of paddings;
[0078] Based on a preset precision value, a preset base value and a number of padding bits, the original gradient data of the floating point type is converted into the encoded gradient data of the integer type.
[0079] Optionally, the first participant unit 201 generates the second encrypted gradient by performing the following operations:
[0080] Based on the homomorphic encryption public key, homomorphically encrypt the encoded gradient data converted into an integer type to generate the first encrypted gradient;
[0081] The first encrypted gradient is secondary encrypted based on a communication public key to generate the second encrypted gradient, wherein the communication public key is generated based on the task number and the task flow node number of the federated learning task.
[0082] Optionally, the second participant unit 202 is used to:
[0083] Decrypting the encrypted gradient data of the previous participant based on the communication private key associated with the user private key of the second participant to obtain intermediate gradient data, where the intermediate gradient data is the first encrypted gradient of the first participant or the candidate aggregated gradient aggregated by the previous second participant;
[0084] Performing a semi-homomorphic addition operation on the intermediate gradient data and the first encrypted gradient of the second participant to obtain a candidate aggregate gradient;
[0085] The candidate aggregate gradient is secondary encrypted based on the communication public key to generate the target aggregate gradient.
[0086] Optionally, the second participant unit 202 is used to:
[0087] Decrypting the target aggregated gradient obtained by the last second participant aggregation based on the communication private key associated with the user private key of the second participant to obtain the intermediate gradient data;
[0088] The intermediate gradient data is homomorphically decrypted, and based on a preset precision value, a preset base value and a number of padding bits, the integer type encoded aggregate gradient is inversely encoded into a floating point type original aggregate gradient, and the model parameters are updated according to the original aggregate gradient.
[0089] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.
[0090] After introducing the data processing method and apparatus according to the exemplary embodiment of the present application, next, a computer device according to another exemplary embodiment of the present application is introduced.
[0091] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.
[0092] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a computer device, see Figure 3 As shown, the computer device 300 may include at least a processor 301 and a memory 302. The memory 302 stores program codes, and when the program codes are executed by the processor 301, the processor 301 executes the steps of any one of the above data processing methods.
[0093] In some possible implementations, the computing device according to the present application may include at least one processor and at least one memory. The memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the data processing method according to various exemplary implementations of the present application described above in this specification. For example, the processor may execute the following steps: Figure 1A Follow the steps shown in .
[0094] Refer to the following Figure 4 hereinafter, a computing device 400 according to this embodiment of the present application is described. Figure 4 The computing device 400 is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0095] like Figure 4 As shown, the computing device 400 is in the form of a general computing device. The components of the computing device 400 may include but are not limited to: at least one processing unit 401, at least one storage unit 402, and a bus 403 connecting different system components (including the storage unit 402 and the processing unit 401).
[0096] Bus 403 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.
[0097] The storage unit 402 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 4021 and / or a cache memory unit 4022 , and may further include a read-only memory (ROM) 4023 .
[0098] The storage unit 402 may also include a program / utility 4025 having a set (at least one) of program modules 4024, such program modules 4024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0099] The computing device 400 may also communicate with one or more external devices 404 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the computing device 800, and / or any device that enables the computing device 400 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 405. In addition, the computing device 400 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 406. As shown, the network adapter 406 communicates with other modules for the computing device 400 via a bus 403. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computing device 400, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0100] Based on the same inventive concept as the above method embodiments, various aspects of the data processing method provided by the present application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the data processing method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device can execute the following steps: Figure 1A Follow the steps shown in .
[0101] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0102] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0103] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A data processing method, characterized in that: The method is applied to multiple parties, including: The first participant is used to encode the original gradient data of floating point type in this round of iterative training into encoded gradient data of integer type, send the second encrypted gradient generated by multi-level encryption of the encoded gradient data to the adjacent second participant for gradient aggregation, and update the model parameters according to the target aggregated gradient obtained by the last second participant aggregation; Each second participant except the first participant is used to aggregate and encrypt the dense gradient data of the previous participant and the first encrypted gradient of the second participant to obtain the target aggregated gradient, and update the model parameters according to the target aggregated gradient obtained by the last second participant aggregation; wherein the dense gradient information is the second encrypted gradient of the first participant or the target aggregated gradient generated by the previous second participant.
2. The method according to claim 1, characterized in that The step of encoding the original gradient data of the floating point type of the current round of iterative training into encoded gradient data of the integer type includes: According to a preset precision value, the original gradient data of the floating point type is shifted left to generate shifted gradient data, and the shifted gradient data is rounded down to determine the number of paddings; Based on a preset precision value, a preset base value and a number of padding bits, the original gradient data of the floating point type is converted into the encoded gradient data of the integer type.
3. The method according to claim 1, characterized in that The second encrypted gradient is generated by performing the following operations: Based on the homomorphic encryption public key, homomorphically encrypt the encoded gradient data converted into an integer type to generate the first encrypted gradient; The first encrypted gradient is secondary encrypted based on a communication public key to generate the second encrypted gradient, wherein the communication public key is generated based on the task number and the task flow node number of the federated learning task.
4. The method according to claim 1, characterized in that The aggregating and encrypting the encrypted gradient data of the previous participant and the first encrypted gradient of the second participant to obtain a target aggregated gradient includes: Decrypting the encrypted gradient data of the previous participant based on the communication private key associated with the user private key of the second participant to obtain intermediate gradient data, where the intermediate gradient data is the first encrypted gradient of the first participant or the candidate aggregated gradient aggregated by the previous second participant; Performing a semi-homomorphic addition operation on the intermediate gradient data and the first encrypted gradient of the second participant to obtain a candidate aggregate gradient; The candidate aggregate gradient is secondary encrypted based on a communication public key to generate the target aggregate gradient.
5. The method according to any one of claims 1 to 4, characterized in that: The updating of the model parameters according to the target aggregated gradient obtained by the last second participant aggregation includes: Decrypting the target aggregated gradient obtained by the last second participant aggregation based on the communication private key associated with the user private key of the second participant to obtain the intermediate gradient data; The intermediate gradient data is homomorphically decrypted, and based on a preset precision value, a preset base value and a number of padding bits, the integer type encoded aggregate gradient is inversely encoded into a floating point type original aggregate gradient, and the model parameters are updated according to the original aggregate gradient.
6. A data processing device, characterized in that: The device is applied to multiple parties, including: The first participant unit is used to encode the original gradient data of the floating point type in this round of iterative training into the encoded gradient data of the integer type, send the second encrypted gradient generated by multi-level encryption of the encoded gradient data to the adjacent second participant for gradient aggregation, and update the model parameters according to the target aggregated gradient obtained by the last second participant aggregation; The second participant unit is composed of second participants except the first participant, which aggregate and encrypt the dense gradient data of the previous participant and the first encrypted gradient of the second participant to obtain a target aggregate gradient, and update the model parameters according to the target aggregate gradient obtained by the last second participant aggregation; wherein the dense gradient information is the second encrypted gradient of the first participant or the target aggregate gradient generated by the previous second participant.
7. The device according to claim 6, characterized in that The first participant unit is used for: According to a preset precision value, the original gradient data of the floating point type is shifted left to generate shifted gradient data, and the shifted gradient data is rounded down to determine the number of paddings; Based on a preset precision value, a preset base value and a number of padding bits, the original gradient data of the floating point type is converted into the encoded gradient data of the integer type.
8. The device according to claim 6, characterized in that The first participant unit generates the second encrypted gradient by performing the following operations: Based on the homomorphic encryption public key, homomorphically encrypt the encoded gradient data converted into an integer type to generate the first encrypted gradient; The first encrypted gradient is secondary encrypted based on a communication public key to generate the second encrypted gradient, wherein the communication public key is generated based on the task number and the task flow node number of the federated learning task.
9. The device according to claim 6, characterized in that The second participant unit is used for: Decrypting the encrypted gradient data of the previous participant based on the communication private key associated with the user private key of the second participant to obtain intermediate gradient data, where the intermediate gradient data is the first encrypted gradient of the first participant or the candidate aggregated gradient aggregated by the previous second participant; Performing a semi-homomorphic addition operation on the intermediate gradient data and the first encrypted gradient of the second participant to obtain a candidate aggregate gradient; The candidate aggregate gradient is secondary encrypted based on a communication public key to generate the target aggregate gradient.
10. The device according to any one of claims 6 to 9, characterized in that: The second participant unit is used for: Decrypting the target aggregated gradient obtained by the last second participant aggregation based on the communication private key associated with the user private key of the second participant to obtain the intermediate gradient data; The intermediate gradient data is homomorphically decrypted, and based on a preset precision value, a preset base value and a number of padding bits, the integer type encoded aggregate gradient is inversely encoded into a floating point type original aggregate gradient, and the model parameters are updated according to the original aggregate gradient.
11. A computer device, characterized in that: It comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 5.
12. A computer-readable storage medium, characterized in that: The method comprises a program code, and when the program code is run on a computer device, the program code is used to make the computer device execute the steps of the method according to any one of claims 1 to 5.