A differential private federated learning method and device providing adaptive gradient

Abstract

Embodiments of the present application provide a self-adaptive gradient differential private federated learning method and device, which is applied to a federated learning architecture, and comprises the following steps: a training model and training parameters are initialized by an aggregation server and then distributed to all terminal devices, and a quantization value list is obtained through initial training feedback of the terminal devices; the number of iterations is determined, and each iteration comprises: the terminal device updates a local training set, adaptively collects training data, determines a differential budget required for local training, generates updated parameters, calculates a local cost function value, and uploads the local cost function value, the number of training data samples and the updated parameters to the aggregation server; the aggregation server performs terminal device filtering, determines a terminal device set satisfying a quantity, performs weighted aggregation, updates the quantization value list, and distributes the quantization value list to all terminal devices; and an output model is obtained after a preset number of iterations is completed. Based on the above processing, the convergence rate of federated learning when applying differential privacy can be improved.

Description

Technical Field

[0001] This invention relates to the field of privacy protection technology, and in particular to a differential private federated learning method and apparatus that provides adaptive gradients. Background Technology

[0002] Federated learning is a machine learning technique that effectively avoids privacy breaches caused by data centralization by jointly training algorithms using local sample data from multiple distributed edge devices. However, potential risks such as model inversion attacks and member inference attacks can significantly increase the probability of sensitive personal information leakage. Therefore, ensuring the security of data stored on distributed clients has become a challenge and a hot topic in federated training. Currently, some research has applied differential privacy to optimizers and sample perturbations to protect data privacy. However, this often leads to a decrease in convergence rate, especially in non-independent and identically distributed (non-iID) environments.

[0003] Model inversion attacks aim to obtain information about the model's training or test data through the model's predicted output. Membership inference attacks aim to train a binary classifier to determine whether data comes from the target model's training or test data, given black-box access to the model. To resist these attacks in federated learning, some studies have applied differential privacy to add perturbations to the aggregation and distribution federated learning models to increase the difficulty of adversary inference. However, directly applying differential privacy may increase the total number of rounds in federated aggregation and reduce the convergence rate.

[0004] Non-independent and identically distributed (IOD) refers to a set of random variables where each variable has a different probability distribution, and these random variables are independent of each other. In federated learning architectures, this means that data from different clients will have different label and data distributions. Compared to federated learning architectures in IOD environments, federated learning architectures in non-independent and identically distributed (IOD) environments suffer from significant convergence problems. Lower update frequencies lead to a substantial decrease in the performance of the learning model, or even failure to converge. Higher update frequencies, on the other hand, greatly increase the communication and computational costs of training. This situation is further exacerbated by applying differential privacy.

[0005] Differential privacy defines a rigorous attack model to provide a quantitative description of privacy leakage. It focuses on protecting the statistical characteristics of the data distribution and protecting individual privacy while ensuring the utility of statistical data. A key technique for achieving differential privacy lies in the perturbation of superimposed noise. The level of privacy protection is directly related to the sensitivity of the function and the privacy budget. Executing a query function on adjacent datasets (obtained by adding or deleting a row from one dataset to obtain another dataset) to which differential privacy is applied can achieve the effect of obtaining the same output with almost identical probabilities. On the other hand, differential privacy is unaffected by the attacker's background knowledge; the magnitude of the background knowledge will not affect the performance of differential privacy. This means that even if an attacker has obtained sensitive information about all records except for one specific record, that record will still not be disclosed. Summary of the Invention

[0006] The purpose of this invention is to provide an adaptive gradient differential private federated learning method and apparatus to improve the convergence rate of federated learning training when applying differential privacy. The specific technical solution is as follows:

[0007] Firstly, to achieve the above objectives, embodiments of the present invention provide a data processing method, which is applied to adaptive differential private parameter perturbation during local training on any terminal device in federated learning. The method includes multiple iterative processes, divided into a first execution round and subsequent iteration rounds.

[0008] The first execution round includes:

[0009] The aggregation server initializes the training model, defining model details such as layers and corresponding units, and randomly generates initial model weights. The aggregation server also determines the local loss function, initial local learning rate, initial local training batch size, initial local training epochs, and initial local pruning threshold, and distributes these, along with model information and test data, to all terminal devices. All terminal devices select training data from their local training sets and perform stochastic gradient descent for the specified initial training epochs. The system performs SGD and generates the first round of local model weights. All terminal devices generate updated parameters based on the initial model weights and the first round of model weights. All terminal devices calculate the local cost function value of the test data based on the first round of local model weights. All terminal devices upload the number of training data samples, the updated parameters, and the local cost function value to the aggregation server. The aggregation server weights and aggregates the updated parameters received from all terminal devices in the first execution round according to the number of training data samples, and calculates the global cost function value of the test data. Based on the local cost function value and the global cost function value, the aggregation server generates quantized values ​​for all terminal devices, determines their initial degree of non-independent and identically distributed (ICD), and distributes these values, along with the aggregated model weights, to all terminal devices.

[0010] Subsequent iterations include:

[0011] The aggregation server initializes the global training epochs, determines the number of terminal devices to be aggregated in each epoch, the aggregation optimizer, the global learning rate, the local learning rate for this epoch, the local training batches, the number of local training epochs, and the local pruning threshold, and distributes these along with the test data to all terminal devices. All terminal devices update their local training sets and adaptively select training data. Based on the quantization values ​​from the previous epoch, the learning rate for this epoch, and the local training data, all terminal devices determine the differential budget required for local training. All terminal devices perform Differentially-Private Stochastic Gradient Descent (DP-SGD) for the specified number of training epochs and generate new local model weights. All terminal devices generate updated parameters based on the aggregated model weights and the new epoch model weights. All terminal devices calculate the local cost function value of the test data based on the new epoch local model weights. All terminal devices upload the number of training data samples, the updated parameters, and the local cost function value to the aggregation server.

[0012] The aggregation server executes a terminal device filtering algorithm on the received feedback from the terminal devices. Based on the number of terminal devices required in each round, it filters out a set of terminal devices that meet the required number. After filtering, it performs weighted aggregation on the updated parameters in the set of terminal devices according to the number of training data samples, and calculates the global cost function value of the test data. Based on the local cost function value and the global cost function value, the aggregation server generates quantized values ​​for the terminal devices in the set, updates their initial non-independent and identically distributed degree, and sends the updated list of quantized values ​​to all terminal devices.

[0013] Repeat the aforementioned number of global training rounds to complete this training.

[0014] Optionally, determining the differential budget required for local training includes: determining the target terminal device's quantization value from the list of updated quantization values ​​issued by the aggregation server in the previous round; and determining the target terminal learning rate from the local learning rate and the terminal device's quantization value issued by the aggregator in this round. Specifically, the larger the updated quantization value of a terminal device, the smaller the local training data sample size, the greater the difference between the terminal device and the globally optimal model, the more severe the client drift and non-independent identically distributed phenomenon, the larger the required target terminal learning rate, the higher the required perturbation level, and the smaller the required differential budget.

[0015] Secondly, to achieve the above objectives, embodiments of the present invention provide a data processing method applied to the terminal device filtering described in the first aspect. The method includes: a first determining module, which, upon receiving a local cost function value of a terminal device, generates a quantized value for the terminal device based on the local cost function value and a global cost function value, and updates and stores the quantized value of the current round number in a preset storage module; upon receiving the number of training data samples of a terminal device, records its feedback time and generates a ratio of the number of training data samples to the feedback time; based on the quantized value update of the current round number and the ratio of the number of training data samples to the feedback time, determines a preset proportional coefficient, calculates the quality of the terminal device, and maps it to a specified interval; and a second determining module, which, based on the number of terminal devices to be aggregated in each round, presets a storage module with an upper limit of that number to store the set of terminal devices that meet the requirements of the aggregation server after filtering.

[0016] Furthermore, the quality of this terminal device in this round can be assessed through a quality function. Assessment, in which This represents the quantization update of the current round number. This indicates the ranking that the terminal device reports to the aggregation server in the current round. This represents the preset scaling factor. To ensure that the denominator is not zero To ensure that the quality function is mapped to the specified interval.

[0017] Optionally, the feedback time is a specific time counting module, including: determining the theoretical latency from the aggregation server to each terminal device; determining the average computation time for the number of training samples under the number of local training rounds; and multiplying the theoretical latency from the aggregation server to each terminal device and the average computation time for the number of training samples under the number of local training rounds by a preset scaling factor to determine the feedback time. .

[0018] Optionally, the storage module is specifically divided into two storage stages: in the first storage stage, the storage module will store data according to the preset upper limit size. Store in sequence Each terminal device index and its quality function value The pair, and based on the mass function value The process involves sorting the data; in the second storage stage, a preset quality threshold for the required terminal devices is established. When a new terminal device feedback is received, the quality function value calculated from that terminal device feedback is compared with the quality function value of the last sorted pair in the storage module. In comparison, if the quality function value calculated by the new terminal device is greater than the quality function value of the last sorted pair in the storage module... The quality function value and its terminal device index are grouped together and inserted into the storage module and reordered; otherwise, the terminal device feedback is discarded. When the quality function value of the last sorted pair in the storage module... Not less than the preset required terminal device quality threshold At this point, the aggregation server stops receiving new feedback from terminal devices and completes the filtering of the terminal device set.

[0019] Thirdly, to achieve the above objectives, embodiments of the present invention provide a data sampling device. This device is applied to the adaptive selection of training data described in the first aspect. The method includes: when the number of training data samples and the number of local training rounds are received from each terminal device, determining the initial sample acquisition window size for each terminal device. Based on the number of local training rounds, the change in the acquisition window size will be specifically divided into two stages: in the first acquisition stage, the acquisition window size remains unchanged according to a preset delay round; in the second acquisition stage, each terminal device will determine the generated acquisition window size based on the indices of a start pointer and an end pointer, wherein the start pointer will increment automatically at the same preset interval, and the end pointer will increment automatically at a given proportional coefficient. The lower bound of the product of the difference between the current training data sample size and the previous training data sample size is incremented.

[0020] Optionally, determining the preset delay rounds includes: determining the training data sample size from each terminal device; determining the expected number of samplings for each training sample; determining the sampling rate of the training samples; and determining the preset delay rounds based on the training data sample size, the expected number of samplings for each training sample, and the sampling rate of the training samples.

[0021] This invention also provides an electronic device, including a processor, an internal bus, a network interface, and a memory, and may also include other hardware required for business operations. The processor, network interface, and memory communicate with each other through the internal bus. The memory is used to store computer programs. When the processor executes the program stored in the memory, it implements the data processing method steps described in any of the first, second, or third aspects above.

[0022] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the data processing method steps described in any of the first, second, or third aspects above.

[0023] This invention also provides a computer program product containing instructions that, when run on a computer, causes the computer to perform any of the data processing methods described above.

[0024] This invention provides a data processing method in which, in each round, the aggregation server receives the number of training data samples, update parameters, and local cost function values ​​from the terminal devices. Based on the local and global cost function values, the aggregation server updates and distributes quantized values ​​to all terminal devices. The global cost function value is calculated by selecting a set of terminal devices that meet the required number of terminal devices per round, and then weighting and aggregating the update parameters in the set of terminal devices according to the number of training data samples to calculate the global cost function value for the test data. Based on the quantized values ​​distributed by the aggregation server in each round, the differential budget for local training on the terminal devices is determined, adaptive data sampling is performed, and training is completed. After training, a new round of training data samples, update parameters, and local cost function values ​​are uploaded. The adaptive data sampling determines the sample collection window for each terminal device based on the number of training data samples and the number of local training rounds for each terminal device. After the aggregation server completes the preset number of training rounds, the target model is obtained.

[0025] Based on the above processing, the quantization value of the target terminal device can be determined based on the label distribution and quantity distribution of the training data for different terminal devices. A privacy budget is then allocated to the target terminal device according to its quantization value, and noise is added to the model weights output by the target terminal device. Compared to existing technologies where each target terminal adds noise to its output model weights using the same differential budget, the technical solution provided in this invention determines the differential budget for target terminal devices with different label and quantity distributions. Noise is added to the model weights output by the terminal device according to this differential budget. For extreme cases of label and data distribution in the training data, this improves the effectiveness of the model weights output by the terminal device. For more reasonable label and data distributions, it improves the security of the model weights output by the terminal device. Overall, after completing model aggregation for a preset number of global training rounds, the technical solution provided in this invention, compared to existing technologies, can improve the effectiveness of the aggregated model and increase the training convergence rate while ensuring the security level of the aggregated model. Attached Figure Description

[0026] Figure 1 This is a flowchart of a data processing method provided in an embodiment of the present invention;

[0027] Figure 2 This is a flowchart of a data processing method provided in an embodiment of the present invention;

[0028] Figure 3 This is a flowchart of a data processing method provided in an embodiment of the present invention;

[0029] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0030] Exemplary examples will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of methods and apparatus consistent with some aspects of this specification as detailed in the appended claims. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this invention.

[0031] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0032] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0033] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0034] The purpose of this invention is to provide an adaptive gradient differential private federated learning method, see [link to relevant documentation]. Figure 1 As shown, the method may include the following steps:

[0035] S101: The aggregation server initializes the training model and randomly generates initial model weights. Determine the number of global training rounds. Global learning rate Local loss function Local learning rate Local training batches and local training rounds And distributed to all terminal devices .

[0036] S102: Determine if this is the initialization wheel:

[0037] S103: If this is the initialization round, all terminal devices select training data from the local training set, perform stochastic gradient descent for the specified number of training rounds, and generate model update parameters. Calculate local cost Upload to the server. Then proceed to step S109.

[0038] S104: If not an initialization round, the aggregation server determines the number of terminal devices to be aggregated in each round. and aggregation optimizer.

[0039] S105: Terminal device updates local training set It also adaptively collects training data.

[0040] S106: The terminal device uses the quantization value described in the previous round. Learning rate in this round Based on the local training data, determine the differential budget required for local training. And add noise to the training model to generate perturbations.

[0041] S107: The terminal device performs differential private stochastic gradient descent to generate updated parameters. Calculate the local cost function value The training data sample count and update parameters are then uploaded to the aggregation server.

[0042] S108: The aggregation server performs terminal device filtering, selecting a set of terminal devices that meets the required number.

[0043] S109: Aggregation server performs weighted aggregation Update the list of quantized values And distribute it to the terminal device, among which .

[0044] Among them, the current quantization value of a terminal device The larger the value, the more severe the client drift phenomenon of the terminal device, and the more severe the non-independent and identically distributed phenomenon of its data. This relates to determining the differential budget required for local training. The lower the value. This is the quantization value of the terminal device in this round. The differential budget required for determining local training. It shows a negative correlation.

[0045] S1010: Determine if the global training rounds have been reached. If the total number of training rounds has not been reached Iteration steps S104-S109.

[0046] S1011: If the global training rounds are reached... Training complete.

[0047] In a preferred embodiment, the selected initial training model is a CNN model, which has two convolutional layers (the first convolutional layer is...). The kernel, the second convolutional layer is The kernel is followed by each convolutional layer. The max pool is activated by ReLU), and there are two fully connected layers (the first fully connected layer has 32 units and the second fully connected layer has 10 units, and the first fully connected layer is activated by ReLU).

[0048] In step S108, the aggregation server performs terminal device filtering, selecting a set of terminal devices that meets the required quantity. For details, please refer to [link to specific steps]. Figure 2 :

[0049] S201: The aggregation server has a storage module that presets the number of terminal devices to be aggregated in the current round. and terminal device quality threshold .

[0050] S202: Determine if the current number of received feedbacks exceeds the storage module's upper limit. .

[0051] S203: The aggregation server sequentially indexes the terminal devices. and its mass function value In pairs Stored in the storage module. Among them, This represents the quantization update of the current round number. This indicates the ranking that the terminal device reports to the aggregation server in the current round. This represents the preset scaling factor. To ensure that the denominator is not zero To ensure that the quality function is mapped to the specified interval.

[0052] S204: The aggregation server will process the group Based on the mass function value Sort the data and proceed to step S209.

[0053] S205: Upon receiving feedback from a new terminal device, the calculated quality function value will be... The quality function value of the last sorted pair in the storage module Compare them.

[0054] S206: Determine if the calculated mass function value Is it greater than the quality function value of the last group in the sort? .

[0055] S207: If the calculated mass function value The quality function value of the last pair in the sorted group is not greater than the quality function value of the last pair in the sorted group. If the terminal device fails to respond, the feedback from that terminal device will be discarded, and the system will wait for feedback from a new terminal device.

[0056] S208: If the calculated mass function value The quality function value of the last pair in the sorting is greater than Then insert a new pair into the storage module. And reorder them.

[0057] S209: Determine the quality function value of the last sorted item in the storage module. Is it greater than the threshold? If the last quality function value in the storage module is sorted Less than the threshold Then the iteration steps are S205-S209.

[0058] S2010: If the last sorted quality function value in the storage module z is greater than the threshold Then the selection of the terminal device set is completed.

[0059] In step S105, the aggregation server performs terminal device filtering, selecting a set of terminal devices that meets the required quantity. For details, please refer to [link to steps]. Figure 3 :

[0060] S301: Determine the initial sample acquisition window size for each terminal device. .

[0061] S302: Determine whether the current round number is the preset delay round number.

[0062] For each terminal device, increasing the accumulated training data will lead to sample backlog, wasted computing power, and low sampling rate of new features. A preset delay epoch ensures sufficient learning of the accumulated sample features.

[0063] S303: If the current number of rounds is less than the preset delay number of rounds, the size of the acquisition window remains unchanged, and step S305 is performed.

[0064] S304: If the current round number reaches the preset delay round number, the terminal device determines the size of the generated acquisition window based on the indices of the start pointer and the end pointer.

[0065] The starting pointer will increment at the same preset interval, and the ending pointer will increment by a given proportional coefficient. The lower bound of the product of the difference between the current training data sample size and the previous training data sample size is incremented.

[0066] S305: The terminal device is performing this round of sample collection.

[0067] This invention also provides an electronic device, such as... Figure 4 As shown, at the hardware level, the electronic device includes a processor S401, an internal bus S402, a network interface S403, and a memory S404. The processor S401, network interface S403, and memory S404 communicate with each other via the internal bus S402.

[0068] The memory S404 is used to store computer programs;

[0069] The processor S401, when executing the program stored in the memory S404, implements any of the data processing methods described above.

[0070] The network interface S403 is used for communication between the aforementioned electronic devices and other devices.

[0071] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described data processing methods.

[0072] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the data processing methods described in the above embodiments.

[0073] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0074] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0075] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A differential private federated learning method providing adaptive gradients, characterized in that, Includes the following steps: The aggregation server initializes the training model, randomly generates initial model weights, determines the global training epochs, global learning rate, local loss function, local learning rate, local training batches, and local training epochs, and distributes these to all terminal devices. Determine if this is the initialization round; When the initialization round is determined, all terminal devices select training data from the local training set, perform stochastic gradient descent for the number of local training rounds, calculate the model update parameters and local cost function values, and upload them to the aggregation server. The aggregation server performs aggregation, establishes a list of quantified values ​​for terminal devices, and sends it to the terminal devices. When it is determined to be a non-initialized round, the aggregation server plans the number of terminal devices to be aggregated and the aggregation optimizer for each round; The terminal device updates its local training set and adaptively collects the training data; Based on the quantization value, the local learning rate, and the local training data from the previous round, determine the differential budget required for local training; The terminal device performs differential private stochastic gradient descent, generates updated parameters, calculates the local cost function value, and uploads it to the aggregation server along with the number of training data samples and the updated parameters. The aggregation server records the feedback time of the terminal device, calculates the quality of the terminal device based on the number of training data samples uploaded by the terminal device and the local cost function value, and performs terminal device filtering based on the number of terminal devices to be aggregated in each round and the quality threshold to determine the set of terminal devices. The aggregation server performs weighted aggregation, updates the list of quantized values, and distributes it to all terminal devices; Once the global training rounds are reached, the aggregation server completes training and outputs the trained model.

2. The method according to claim 1, characterized in that, The determination of the differential budget required for local training includes: Based on the list of quantized values ​​updated by the aggregation server in the previous round, the quantized value of the target terminal device is determined; based on the local learning rate updated by the aggregator in this round and the quantized value of the terminal device, the learning rate of the target terminal is determined. The larger the quantization value of a terminal device after its update, the smaller the sample size of the local training data, the greater the difference between the terminal device and the globally optimal model, the more severe the client drift and non-independent identical distribution phenomenon, the larger the required target terminal learning rate, the higher the required perturbation level, and the smaller the required difference budget.

3. The method according to claim 1, characterized in that, The aggregation server performs terminal device filtering, including: The aggregation server presets the storage module size and terminal device quality threshold for the number of terminal devices to be aggregated in the current round. Determine whether the number of feedback received in the current round is greater than the upper limit of the storage module; When it is determined that the number of feedback received in the current round is less than the upper limit of the storage module, the aggregation server sequentially stores the terminal device index and its quality function value into the storage module in the form of pairs; The aggregation server sorts the pairs from high to low according to the quality function value; When it is determined that the number of feedback received in the current round is greater than the upper limit of the storage module, the quality function value is calculated based on the new terminal device feedback received, and compared with the quality function value of the last group in the storage module. Determine whether the quality function value is greater than the quality function value of the last sorted pair in the storage module; When it is determined that the quality function value is less than the quality function value of the last sorted pair in the storage module, the current feedback from the terminal device is discarded. When the quality function value is greater than the quality function value of the last sorted pair in the storage module, a new pair is inserted into the storage module and re-sorted. The system determines whether the quality function value at the end of the sorted sequence in the storage module is greater than the terminal device quality threshold. When the quality function value is less than the terminal device quality threshold, the aggregation server continues to accept new feedback from the terminal devices. When the quality function value is greater than the terminal device quality threshold, the filtering of the terminal device set is completed.

4. The method according to claim 3, characterized in that, The quality function values ​​fed back by the terminal device include: The mass function value is obtained through the mass function. Assessment, in which This represents the quantization update of the current round number. This indicates the ranking that the terminal device reports to the aggregation server in the current round. This represents the preset scaling factor. To ensure that the denominator is not zero To ensure that the quality function is mapped to the specified interval.

5. The method according to claim 3 or 4, characterized in that, The feedback time includes: Determine the theoretical latency from the aggregation server to each terminal device; determine the average computation time for the number of training samples under the number of local training rounds; based on the theoretical latency from the aggregation server to each terminal device and the average computation time for the number of training samples under the number of local training rounds, multiply them by a preset scaling factor to determine... .

6. The method according to claim 3, characterized in that, The storage module includes: In the first storage stage, the storage module will determine the number of aggregated terminal devices according to the preset number. Store in sequence The terminal device index and its quality function value The pair, and based on the mass function value Sort from highest to lowest; In the second storage stage, a preset quality threshold for the terminal device is established. When a new terminal device feedback is received, the quality function value calculated from the feedback of that terminal device is compared with the quality function value of the last sorted pair in the storage module. If the quality function value calculated by the new terminal device is greater than the quality function value of the last sorted pair in the storage module, then... The quality function value and its terminal device index are paired and inserted into the storage module and reordered; otherwise, the terminal device feedback is discarded. When the quality function value of the last sorted pair in the storage module Not less than the preset required terminal device quality threshold At this point, the aggregation server stops receiving new feedback from terminal devices and completes the filtering of the terminal device set.

7. The method according to claim 1, characterized in that, The adaptive acquisition of the training data includes: Determine the initial sample acquisition window size for each terminal device; Determine if the current aggregation round has reached the preset delay round; When the number of aggregation rounds has not reached the preset delay rounds, the sample acquisition window size remains unchanged; When the number of aggregation rounds has reached the preset delay round number, the terminal device will determine the size of the generated sample acquisition window based on the index of the start pointer and the end pointer, and complete the sample acquisition of this round by the terminal device.

8. The method according to claim 7, characterized in that, The preset delay rounds include: determining the training data sample size from each terminal device; determining the expected number of samplings for each training sample; determining the sampling rate of the training samples; and determining the preset delay rounds based on the training data sample size, the expected number of samplings for each training sample, and the sampling rate of the training samples.

9. An electronic device, the device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the differential private federated learning method with adaptive gradients as described above.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the differential private federated learning method for adaptive gradients as described in any one of the preceding claims.

11. A computer program product containing instructions, which, when run on a computer, causes the computer to perform the adaptive gradient differential private federated learning method described in any of the preceding claims.

Images