A method, device, equipment and medium for breakpoint privacy protection
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 新奥新智科技有限公司
- Filing Date
- 2021-12-17
- Publication Date
- 2026-06-02
Smart Images
Figure CN116340959B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for breakpoint privacy protection. Background Technology
[0002] With the rapid development of deep learning theory and technology, the advantages of artificial intelligence in various industries are gradually becoming apparent, while the protection of privacy in deep learning is becoming increasingly important. Joint learning is a distributed learning paradigm for distributed datasets that protects data privacy. A classic joint learning framework includes a central node and different local participants. Training is achieved by each participant uploading, distributing, and updating model parameters, requiring consistency among all participants during training. Through joint learning, each participant can train an accurate global model based on its local privacy data, achieving connectivity between participants while protecting their individual privacy.
[0003] However, in practice, due to the large number of participants, it's easy for some participants to miss a training round due to various reasons. Currently, to avoid leaking information about each participant, most methods use differential privacy to encrypt joint learning. While encryption can mask differences between real data by adding noise, it cannot control privacy loss and cannot address the issue of a participant missing a training round. Summary of the Invention
[0004] In view of this, embodiments of this disclosure provide a method, apparatus, device, and medium for breakpoint privacy protection to solve the problem that the prior art cannot control privacy loss during model training and cannot achieve a balance between model privacy and availability when a participant does not participate in the current joint learning model training.
[0005] A first aspect of this disclosure provides a method for breakpoint privacy protection, comprising:
[0006] Data from each participant is acquired, and a corresponding local model is trained based on the data from each participant, and the model parameters corresponding to the local model are acquired respectively.
[0007] Noise is added to the corresponding model parameters using a preset encryption algorithm to obtain the encrypted value corresponding to each model parameter;
[0008] Each encrypted value is uploaded to the central node, and the central node performs an aggregation operation on each encrypted value to obtain the corresponding aggregation parameters. Based on the aggregation parameters, the mean value of the model parameters corresponding to each participant is calculated.
[0009] By using a preset privacy loss calculation mechanism, based on the average model parameters, the privacy loss caused by the central node accessing the model parameters corresponding to each participant is calculated, and the total privacy loss value corresponding to each participant is obtained.
[0010] The total privacy loss is compared with a preset privacy loss threshold. If the total privacy loss is less than the preset privacy loss threshold, the number of non-participating parties that did not participate in the training of the current joint learning model is calculated based on the total privacy loss and the preset privacy loss threshold. The state of the non-participating parties is a breakpoint.
[0011] According to the preset encryption algorithm, noise corresponding to the number of non-participating parties is added to the corresponding model parameters.
[0012] A second aspect of this disclosure provides an apparatus for breakpoint privacy protection, comprising:
[0013] Training module, noise addition module, aggregation module, statistics module, calculation module, noise supplementation module;
[0014] The training module is used to acquire data from each participant, train corresponding local models based on the data from each participant, and acquire model parameters corresponding to the local models respectively.
[0015] The noise addition module is used to add noise to the corresponding model parameters using a preset encryption algorithm to obtain the encrypted value corresponding to each model parameter.
[0016] The aggregation module is used to upload each encrypted value to the central node, and the central node performs an aggregation operation on each encrypted value to obtain the corresponding aggregation parameters, so as to calculate the average model parameter of each participant based on the aggregation parameters.
[0017] The statistics module is used to calculate the privacy loss caused by the central node accessing the model parameters of each participant based on the average model parameters using a preset privacy loss calculation mechanism, and to obtain the total privacy loss value for each participant.
[0018] The calculation module is used to compare the total privacy loss value with a preset privacy loss threshold. If the total privacy loss value is less than the preset privacy loss threshold, the module calculates the number of non-participating parties that did not participate in the training of the current joint learning model based on the total privacy loss value and the preset privacy loss threshold. The state of the non-participating parties is a breakpoint.
[0019] The noise supplementation module is used to supplement the corresponding model parameters with noise corresponding to the number of non-participating parties according to the preset encryption algorithm.
[0020] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.
[0021] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.
[0022] The beneficial effects of this disclosed embodiment compared to the prior art are as follows: By training the local model corresponding to each participant using the data obtained from each participant, the corresponding model parameters can be obtained; by adding noise interference to the model parameters through a preset encryption algorithm to obtain the encrypted value corresponding to the noise, the difference between the encrypted value and the model parameters can be masked, making it impossible for attackers to obtain the real model parameters based on the query results, and without needing to care about the attacker's background knowledge; by aggregating each encrypted value through the central node, the mean value of the model parameters corresponding to each participant can be determined, and the current model corresponding to each participant can be updated accordingly; by using a preset privacy loss calculation mechanism, the total privacy loss value during the model training process can be calculated, and by comparing it with a preset privacy loss threshold, the privacy protection effect of the model can be effectively controlled; when there are non-participating parties that have not participated in the current joint learning model training, the noise corresponding to the non-participating party at the breakpoint is added to the model parameters corresponding to each participant, thereby achieving a balance between the model's privacy and usability. This not only provides a strong privacy protection effect for the privacy data of the participants, but also saves privacy budget, while ensuring the service quality of the model. Attached Figure Description
[0023] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0024] Figure 1 This is a schematic diagram of a joint learning architecture according to an embodiment of this disclosure;
[0025] Figure 2 This is a flowchart illustrating a method for protecting privacy based on breakpoints provided in an embodiment of this disclosure;
[0026] Figure 3 This is a flowchart illustrating another method for breakpoint privacy protection provided in this embodiment of the disclosure;
[0027] Figure 4 This is a schematic diagram of the structure of a device for breakpoint privacy protection provided in an embodiment of this disclosure;
[0028] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure;
[0029] Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of this disclosure. Detailed Implementation
[0030] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.
[0031] Federation learning refers to the comprehensive utilization of multiple AI (Artificial Intelligence) technologies, under the premise of ensuring data security and user privacy, to collaboratively explore the value of data and foster new intelligent business forms and models based on joint modeling. Federation learning has at least the following characteristics:
[0032] (1) Participating nodes control their own data in a weakly centralized joint training mode to ensure data privacy and security in the process of co-creating intelligence.
[0033] (2) In different application scenarios, various model aggregation optimization strategies are established by using screening and / or combination of AI algorithms and privacy-preserving computing to obtain high-level and high-quality models.
[0034] (3) Under the premise of ensuring data security and user privacy, based on multiple model aggregation optimization strategies, obtain methods to improve the performance of the federated learning engine. The performance methods can be improved by solving problems such as parallel computing architecture, information interaction under large-scale cross-domain networks, intelligent perception, and anomaly handling mechanisms.
[0035] (4) Obtain the needs of multiple users in various scenarios, determine the true contribution of each joint participant through a mutual trust mechanism, and allocate incentives accordingly.
[0036] Based on the above approach, an AI technology ecosystem based on collaborative learning can be established, fully leveraging the value of industry data and promoting the implementation of scenarios in vertical fields.
[0037] A method, apparatus, device, and medium for breakpoint privacy protection according to embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.
[0038] Figure 1 This is a schematic diagram of a joint learning architecture according to an embodiment of this disclosure. Figure 1 As shown, the architecture of joint learning may include a server (central node) 101 and participants 102, 103 and 104.
[0039] In the joint learning process, a basic model can be established through server 101, which then sends this model to participants 102, 103, and 104 with whom it has established a communication connection. Alternatively, any participant can establish the basic model and upload it to server 101, which then sends it to other participants with whom it has established a communication connection. Participants 102, 103, and 104 construct models based on the downloaded basic structure and model parameters, train the models using local data, obtain updated model parameters, and encrypt and upload these updated model parameters to server 101. Server 101 aggregates the model parameters sent by participants 102, 103, and 104 to obtain global model parameters, which are then transmitted back to participants 102, 103, and 104. Participants 102, 103, and 104 iterate on their respective models based on the received global model parameters until the models converge, thus achieving model training. During the joint learning process, the data uploaded by participants 102, 103, and 104 are model parameters; local data is not uploaded to server 101, and all participants can share the final model parameters. Therefore, collaborative modeling can be achieved while ensuring data privacy. It should be noted that the number of participants is not limited to the three mentioned above, but can be set as needed; this disclosure does not impose any limitations on this. Since existing technologies cannot control privacy loss during model training, a balance between model privacy and usability cannot be achieved when a participant does not participate in the current joint learning model training. Therefore, a method is needed to control privacy loss and achieve privacy protection when a participant does not participate in the joint learning model training.
[0040] Figure 2 This is a flowchart illustrating a method for protecting privacy based on breakpoints, as provided in an embodiment of this disclosure. Figure 2 A method for breakpoint-oriented privacy protection can be derived from Figure 1 Server execution oriented towards breakpoint privacy protection. For example... Figure 2 As shown, this method for breakpoint privacy protection includes:
[0041] S201, acquire data from each participant, train corresponding local models based on the data from each participant, and acquire model parameters corresponding to each local model.
[0042] S202, noise is added to the corresponding model parameters using a preset encryption algorithm to obtain the encrypted value corresponding to each model parameter.
[0043] S203, each encrypted value is uploaded to the central node, and the central node performs an aggregation operation on each encrypted value to obtain the corresponding aggregation parameters, so as to calculate the average model parameter of each participant based on the aggregation parameters.
[0044] S204, using a preset privacy loss calculation mechanism, based on the average model parameters, calculate the privacy loss caused by the central node accessing the model parameters corresponding to each participant, and obtain the total privacy loss value corresponding to each participant.
[0045] S205, compare the total privacy loss value with the preset privacy loss threshold. If the total privacy loss value is less than the preset privacy loss threshold, calculate the number of non-participating parties that did not participate in the training of the current joint learning model based on the total privacy loss value and the preset privacy loss threshold.
[0046] S206, according to the preset encryption algorithm, noise corresponding to the number of non-participating parties is added to the corresponding model parameters.
[0047] Specifically, in the context of breakpoint privacy protection, the server obtains the privacy data of each participant, and then trains a local model for each participant based on this data, thereby obtaining the model parameters from each participant's local model. The server adds noise to the model parameters using a pre-defined encryption algorithm, resulting in an encrypted value for each parameter. This encrypted value is then uploaded to a central node, which aggregates the values to obtain aggregated parameters. The mean value of the model parameters for each participant is then calculated based on these aggregated parameters. Using a pre-defined privacy loss calculation mechanism and the calculated mean value of the model parameters for each participant, the server can calculate the privacy loss incurred by the central node accessing the model parameter locks for each participant, obtaining the total privacy loss for each participant. This allows for control of the iterative process during the joint learning model training. This application, by introducing a pre-defined privacy loss calculation mechanism, calculates the privacy loss incurred by the central node accessing the training data, thereby controlling and guiding the entire analysis process.
[0048] The server compares the total privacy loss calculated by the preset privacy loss calculation mechanism with the preset privacy loss threshold to determine in real time whether the total privacy loss during the joint learning model training process exceeds the preset privacy loss threshold. It should be noted that the preset privacy loss calculation mechanism in this embodiment is a MomentsAccountant (MA) mechanism.
[0049] If the total privacy loss is determined to be less than a preset privacy loss threshold, it indicates that there are non-participating parties who did not participate in the training of the current joint learning model. The current total privacy loss does not achieve a balance between the model's privacy and usability. Therefore, it is necessary to calculate the number of non-participating parties among the participants based on the total privacy loss and the preset privacy loss threshold. It should be noted that the status of non-participating parties in this application is a breakpoint. After calculating the number of non-participating parties, the server will add noise corresponding to the number of non-participating parties to the model parameters corresponding to each participant according to a preset encryption algorithm to ensure that the model's privacy and usability are balanced when non-participating parties exist during the training of the current joint learning model.
[0050] According to the technical solution provided in this disclosure, by acquiring the data from each participant, the local model corresponding to each participant can be trained, thereby obtaining the corresponding model parameters. A preset encryption algorithm is used to add noise interference to the model parameters to obtain encrypted values corresponding to the noise, thus masking the difference between the encrypted values and the model parameters. This prevents attackers from obtaining the true model parameters based on the query results and eliminates the need to consider the attacker's background knowledge. By aggregating the encrypted values through a central node, the average model parameters corresponding to each participant can be determined, thereby updating the current model for each participant. A preset privacy loss calculation mechanism is used to calculate the total privacy loss during model training. By comparing this with a preset privacy loss threshold, the privacy protection effect of the model can be effectively controlled. When there are non-participating parties that are not involved in the current joint learning model training, the noise corresponding to the non-participating party at the breakpoint is added to the model parameters corresponding to each participant, thereby achieving a balance between model privacy and usability. This not only provides strong privacy protection for the participants' private data but also saves on privacy budgets while ensuring the service quality of the model.
[0051] In some embodiments, noise is added to the model parameters corresponding to each participant using a preset encryption algorithm to obtain the encrypted value corresponding to each model parameter, specifically including:
[0052] Calculate the gradient corresponding to each model parameter, and then clip the gradient corresponding to each model parameter.
[0053] According to the preset encryption algorithm, noise is added to the total gradient value corresponding to each parameter of the clipped model through a noise addition sub-protocol.
[0054] Based on the total gradient value after adding noise, the encrypted values corresponding to each model parameter are obtained.
[0055] Specifically, after obtaining the model parameters corresponding to each participant, the server calculates the gradient corresponding to each model parameter based on the model parameters corresponding to each participant, and then clips the gradient corresponding to each model parameter. Then, according to the preset encryption algorithm, noise is added to the total gradient value corresponding to each model parameter after clipping through the noise addition sub-protocol, and the encrypted value corresponding to each model parameter is obtained based on the total gradient value corresponding to each model parameter after adding noise.
[0056] According to the technical solution provided in this disclosure, noise is added to the model parameters corresponding to each participant by a preset encryption algorithm to encrypt the privacy data of each participant. This can mask the difference between the encrypted value and the model parameters, making it impossible for attackers to obtain the real model parameters based on the query results, and it does not require the attacker to care about the background knowledge they possess.
[0057] In some embodiments, before adding noise to the total gradient value corresponding to each trimmed model parameter according to a preset encryption algorithm and a noise addition sub-protocol, the method further includes:
[0058] Determine the total number of participants and the total amount of noise added corresponding to the current joint learning model training;
[0059] Based on the total number of participants and the total amount of noise to be added, calculate the noise to be added for the model parameters of each participant.
[0060] Specifically, when adding noise to the model parameters of each participant, the server calculates the total noise added during the joint learning model training process, determines the total number of participants corresponding to the current joint learning model training, and then calculates the noise to be added to the model parameters of each participant based on the ratio of the total noise added to the total number of participants.
[0061] According to the technical solution provided in this disclosure, the noise to be added for each participant is calculated by the ratio of the total noise added to the total number of participants. This makes it easier to determine the noise to be added for each non-participant in the subsequent training process of the joint learning model when there are non-participants, thereby controlling privacy loss.
[0062] In some embodiments, the number of non-participating parties that did not participate in the training of the current joint learning model is calculated based on the total privacy loss value and a preset privacy loss threshold, specifically including:
[0063] The privacy loss difference corresponding to the current joint learning model training is determined based on the total privacy loss value and the preset privacy loss threshold.
[0064] The number of non-participants who did not participate in the training of the current joint learning model is determined by the ratio between the privacy loss difference and the noise to be added for each participant.
[0065] Specifically, the server can determine the privacy loss difference corresponding to the current joint learning model training based on the difference between the preset privacy loss threshold and the total privacy loss value. Then, based on the ratio between the privacy loss difference and the noise to be added for each non-participating party, the server can calculate the number of non-participating parties that did not participate in the current joint learning model training.
[0066] According to the technical solution provided in this disclosure, when the total privacy loss value of the current joint learning model training does not reach the preset privacy loss threshold, the number of non-participating parties is determined by the ratio of the privacy loss difference to the noise to be added corresponding to each non-participating party. Thus, the privacy loss difference corresponding to the current joint learning model training can be supplemented based on the calculated number of non-participating parties and the noise to be added corresponding to each non-participating party, thereby ensuring a balance between the privacy and availability of the model and guaranteeing the service quality of the model.
[0067] In some embodiments, after adding noise corresponding to the number of non-participating parties to the corresponding model parameters according to a preset encryption algorithm, the method further includes:
[0068] Obtain the encrypted values corresponding to each model parameter after noise supplementation, and upload the encrypted values after noise supplementation to the central node;
[0069] The encrypted values after noise supplementation are aggregated through the central node to obtain the aggregated parameters after noise supplementation.
[0070] Based on the aggregate parameters after noise supplementation, calculate the mean values of the model parameters for each participant after noise supplementation;
[0071] Update the local model for each participant based on the mean of the model parameters for each participant after adding noise.
[0072] Specifically, the server, based on a preset encryption algorithm, adds noise corresponding to the number of non-participants to the corresponding model parameters, obtaining encrypted values for each model parameter after noise addition. These encrypted values are then uploaded to the central node, which re-aggregates the noise-added encrypted values to obtain the corresponding aggregated parameters. The number of participants is calculated based on the difference between the total number of participants and the number of non-participants. Then, based on the ratio between the aggregated parameters after noise addition and the number of participants, the mean value of the model parameters for each participant after noise addition is calculated. Finally, the local model for each participant is updated based on the mean value of the model parameters for each participant after noise addition.
[0073] According to the technical solution provided in this disclosure, by calculating the aggregated participants corresponding to the model parameters after noise supplementation, the mean value of the model parameters corresponding to each participant is determined, thereby updating the local model corresponding to each participant based on the mean value of the model parameters, completing one training update of this model, and making the performance of the local model corresponding to each participant better.
[0074] In some embodiments, after comparing the total privacy loss with a preset privacy loss threshold, the method further includes:
[0075] If the total privacy loss is determined to be greater than or equal to the preset privacy loss threshold, then the training of the current joint learning model is stopped.
[0076] Specifically, after comparing the total privacy loss value with the preset privacy loss threshold, if the server determines that the total privacy loss value is greater than or equal to the preset privacy loss threshold, it means that the total privacy loss value of the joint learning model training has reached the expected level. At this time, the privacy and usability of the model have reached a balance, so the current joint model training is stopped to avoid excessive noise addition to the model, which would prevent the model from reaching a usable level.
[0077] According to the technical solution provided in this disclosure, by comparing the total privacy loss value with the preset privacy loss threshold in real time, the balance between the privacy and availability of the model can be accurately controlled, thereby improving the service quality of the model.
[0078] In some embodiments, the preset encryption algorithm is a differential privacy algorithm.
[0079] According to the technical solution provided in this disclosure, by combining differential privacy algorithm with privacy loss calculation mechanism, it is possible to ensure that privacy loss reaches a preset privacy loss threshold. Furthermore, differential privacy masks the differences between real data by adding noise interference. Differential privacy query can ensure that the result remains unchanged when data is added or deleted. Attackers cannot obtain real data based on the query result and do not need to care about the background knowledge possessed by the attacker.
[0080] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.
[0081] Figure 3 This is a flowchart illustrating another method for breakpoint privacy protection provided in this disclosure. Figure 3 The server trains a local model for each participant based on the data obtained from each participant, and obtains the model parameters for each local model. Then, it adds noise to the obtained model parameters using a differential privacy algorithm to obtain an encrypted value for each model parameter. The encrypted values for each model parameter are then uploaded to the central node, where the central node performs an aggregation operation on the encrypted values to obtain the corresponding aggregate parameters. The central node also calculates the mean of the model parameters for each participant and sends the mean of the model parameters to each participant so that each participant can update its own local model based on the mean of the model parameters.
[0082] The server uses the MA mechanism to statistically analyze the privacy loss during model training and determines whether the total privacy loss exceeds a preset privacy loss threshold. If not, it sends the result to each participant through the central node to determine whether the number of participants equals the total number of participants. When the number of participants is less than the total number of participants, it calculates the privacy loss difference between the preset privacy loss threshold and the total privacy loss. Based on the ratio of the privacy loss difference to the noise to be added for each participant, it determines the number of non-participants and adds noise corresponding to the number of non-participants to the model parameters for each participant, thereby achieving a balance between the privacy and usability of the model.
[0083] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.
[0084] Figure 4 This is a schematic diagram of a device for breakpoint privacy protection provided in an embodiment of this disclosure.
[0085] like Figure 4 As shown, the device for breakpoint privacy protection includes:
[0086] Training module 401 is configured to acquire data from each participant, train corresponding local models based on the data from each participant, and acquire the model parameters corresponding to the local models respectively.
[0087] The noise addition module 402 is configured to add noise to the corresponding model parameters using a preset encryption algorithm to obtain the encrypted value corresponding to each model parameter;
[0088] The aggregation module 403 is configured to upload each encrypted value to the central node, and the central node performs an aggregation operation on each encrypted value to obtain the corresponding aggregation parameters, so as to calculate the average model parameter of each participant based on the aggregation parameters.
[0089] The statistics module 404 is configured to use a preset privacy loss calculation mechanism to calculate the privacy loss generated by the central node accessing the model parameters of each participant based on the mean of the model parameters, and obtain the total privacy loss value for each participant.
[0090] The calculation module 405 is configured to compare the total privacy loss value with a preset privacy loss threshold. If the total privacy loss value is less than the preset privacy loss threshold, the number of non-participating parties that did not participate in the training of the current joint learning model is calculated based on the total privacy loss value and the preset privacy loss threshold. The state of the non-participating parties is a breakpoint.
[0091] The noise supplementation module 406 is configured to supplement the corresponding model parameters with noise corresponding to the number of non-participating parties according to a preset encryption algorithm.
[0092] According to the technical solution provided in this disclosure, the noise addition module adds noise interference to the model parameters corresponding to each participant, which can mask the difference between the encrypted value and the model parameters, making it impossible for attackers to obtain the real model parameters based on the query results, and without needing to care about the background knowledge possessed by the attackers; the total privacy loss during the model training process is statistically analyzed through the MA mechanism, which can effectively control the privacy protection effect of the model; the noise supplementation module can add noise supplementation corresponding to non-participating parties to the model parameters corresponding to each participant, thereby achieving a balance between the privacy and availability of the model and improving the service quality of the model.
[0093] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.
[0094] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. For example... Figure 5 As shown, the electronic device 5 of this embodiment includes: a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, it implements the steps in the various method embodiments described above. Alternatively, when the processor 501 executes the computer program 503, it implements the functions of each module / unit in the various device embodiments described above.
[0095] For example, computer program 503 may be divided into one or more modules / units, which are stored in memory 502 and executed by processor 501 to perform the present disclosure. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 503 in electronic device 5.
[0096] Electronic device 5 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 5 may include, but is not limited to, processor 501 and memory 502. Those skilled in the art will understand that... Figure 5 This is merely an example of electronic device 5 and does not constitute a limitation on electronic device 5. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.
[0097] Processor 501 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.
[0098] The memory 502 can be an internal storage unit of the electronic device 5, such as a hard disk or RAM. The memory 502 can also be an external storage device of the electronic device 5, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 502 can include both internal and external storage units of the electronic device 5. The memory 502 is used to store computer programs and other programs and data required by the electronic device. The memory 502 can also be used to temporarily store data that has been output or will be output.
[0099] Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of this disclosure. Figure 6As shown, the computer-readable storage medium stores a computer program 601, which, when executed by a processor, implements the steps of the above method.
[0100] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0101] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0102] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.
[0103] In the embodiments provided in this disclosure, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.
[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0105] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0106] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.
[0107] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.
Claims
1. A method for breakpoint privacy protection, characterized in that, include: Data from each participant is acquired, and a corresponding local model is trained based on the data from each participant, and the model parameters corresponding to the local model are acquired respectively. Noise is added to the corresponding model parameters using a preset encryption algorithm to obtain the encrypted value corresponding to each model parameter; Each encrypted value is uploaded to the central node, and the central node performs an aggregation operation on each encrypted value to obtain the corresponding aggregation parameters. Based on the aggregation parameters, the mean value of the model parameters corresponding to each participant is calculated. By using a preset privacy loss calculation mechanism, based on the average model parameters, the privacy loss caused by the central node accessing the model parameters corresponding to each participant is calculated, and the total privacy loss value corresponding to each participant is obtained. The total privacy loss is compared with a preset privacy loss threshold. If the total privacy loss is less than the preset privacy loss threshold, the number of non-participating parties that did not participate in the training of the current joint learning model is calculated based on the total privacy loss and the preset privacy loss threshold. The state of the non-participating parties is a breakpoint. According to the preset encryption algorithm, noise corresponding to the number of non-participating parties is added to the corresponding model parameters.
2. The method according to claim 1, characterized in that, The step of adding noise to the corresponding model parameters using a preset encryption algorithm to obtain encrypted values for each model parameter specifically includes: Calculate the gradient corresponding to each model parameter, and then clip the gradient corresponding to each model parameter. Based on the preset encryption algorithm, noise is added to the total gradient value corresponding to each parameter of the trimmed model through a noise addition sub-protocol. Based on the total gradient value after adding noise, the encrypted value corresponding to each model parameter is obtained.
3. The method according to claim 2, characterized in that, Before adding noise to the total gradient value corresponding to each model parameter after pruning according to the preset encryption algorithm and noise addition sub-protocol, the method further includes: Determine the total number of participants and the total amount of noise added corresponding to the current joint learning model training; Based on the total number of participants and the total noise addition value, calculate the noise to be added corresponding to the model parameters of each participant.
4. The method according to claim 1, characterized in that, The step of calculating the number of non-participating parties who did not participate in the current joint learning model training based on the total privacy loss value and the preset privacy loss threshold specifically includes: Based on the total privacy loss value and the preset privacy loss threshold, determine the privacy loss difference corresponding to the current joint learning model training; The number of non-participants who did not participate in the training of the current joint learning model is determined based on the ratio between the privacy loss difference and the noise to be added for each participant.
5. The method according to claim 1, characterized in that, After adding noise corresponding to the number of non-participating parties to the corresponding model parameters according to the preset encryption algorithm, the method further includes: The encrypted values corresponding to each model parameter after noise supplementation are obtained, and the encrypted values after noise supplementation are uploaded to the central node; The central node performs an aggregation operation on the encrypted value after noise supplementation to obtain the aggregation parameters after noise supplementation. Based on the aggregated parameters after noise supplementation, calculate the mean values of the model parameters corresponding to each participant after noise supplementation; The local model corresponding to each participant is updated based on the mean of the model parameters corresponding to each participant after the noise is added.
6. The method according to claim 1, characterized in that, After comparing the total privacy loss value with the preset privacy loss threshold, the method further includes: If the total privacy loss is determined to be greater than or equal to the preset privacy loss threshold, then the training of the current joint learning model is stopped.
7. The method according to any one of claims 1 to 6, characterized in that, The preset encryption algorithm is a differential privacy algorithm.
8. A device for protecting privacy at breakpoints, characterized in that, include: Training module, noise addition module, aggregation module, statistics module, calculation module, noise supplementation module; The training module is used to acquire data from each participant, train corresponding local models based on the data from each participant, and acquire model parameters corresponding to the local models respectively. The noise addition module is used to add noise to the corresponding model parameters using a preset encryption algorithm to obtain the encrypted value corresponding to each model parameter. The aggregation module is used to upload each encrypted value to the central node, and the central node performs an aggregation operation on each encrypted value to obtain the corresponding aggregation parameters, so as to calculate the average model parameter of each participant based on the aggregation parameters. The statistics module is used to calculate the privacy loss caused by the central node accessing the model parameters of each participant based on the average model parameters using a preset privacy loss calculation mechanism, and to obtain the total privacy loss value for each participant. The calculation module is used to compare the total privacy loss value with a preset privacy loss threshold. If the total privacy loss value is less than the preset privacy loss threshold, the module calculates the number of non-participating parties that did not participate in the training of the current joint learning model based on the total privacy loss value and the preset privacy loss threshold. The state of the non-participating parties is a breakpoint. The noise supplementation module is used to supplement the corresponding model parameters with noise corresponding to the number of non-participating parties according to the preset encryption algorithm.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.