Dynamic hyper-parameter adjustment method and device for asynchronous federated learning and medium

Abstract

The invention relates to a dynamic hyper-parameter adjustment method and device for asynchronous federated learning and a medium, the method is characterized in that when a local model of a client is updated, hyper-parameter adjustment is carried out based on a reward function of reinforcement learning, the reward function is divided into two parts, the first part quantifies reduction of relative loss of the global model on the client data set, and the second part quantifies characterization inconsistency of the global model and the local model. Compared with the prior art, the method introduces the reinforcement learning algorithm to dynamically optimize the hyper-parameters, and enhances the generalization ability of the model by designing the special reward function, thereby improving the accuracy and training efficiency of the global model.

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular to a dynamic hyperparameter adjustment method, device, and medium for asynchronous federated learning. Background Art

[0002] With the continued expansion of medical data, especially in the context of joint data analysis across medical institutions, privacy protection and data security issues are becoming increasingly prominent. Federated learning, as a distributed approach, can effectively address this issue. This approach uses a central server to coordinate the collaborative training of multiple clients to share a global model, while ensuring that each client's sensitive data remains locally. This effectively integrates fragmented medical data resources while ensuring privacy and security. This distributed learning mechanism provides important technical support for cross-institutional medical collaboration. Its flexible architecture not only adapts to the data characteristics of different medical institutions, but also significantly improves the accuracy and generalization capabilities of medical image analysis, ultimately promoting the clinical application of medical artificial intelligence technology.

[0003] Traditional synchronous federated learning methods typically require all participating devices to synchronously update the global model after each iteration. However, due to differences in network bandwidth, computing power, and energy consumption among different devices, the system often exhibits heterogeneity. This synchronous update mechanism causes the global model update speed to be slowed down by the slowest device, creating the so-called "slow node problem." In this case, faster devices must wait for slower devices to complete local training, resulting in idle computing resources and reduced overall system efficiency.

[0004] While asynchronous federated learning (AFL) can address the "slow node problem" to some extent by allowing each client to upload updates promptly after local training completes, reducing latency, this asynchronous mechanism also introduces new challenges, such as inconsistent model updates and performance loss. Asynchronous client training times can lead to unstable global model aggregation, impacting overall training performance. For example, in the Fedasyn method, local model weights can be sent to the server as soon as a client completes its local training task. This method proposes three methods for calculating aggregate weights based on staleness. The FedBuff method introduces a buffering mechanism, caching each client's updated model weights in a buffer rather than aggregating updates from all clients. However, these methods fail to account for data heterogeneity, resulting in reduced model accuracy. Furthermore, without proper staleness control, AFL's convergence can be hampered, making it crucial to determine the appropriate aggregation weight between local and global models.

[0005] Furthermore, data heterogeneity in the medical imaging field exacerbates the challenges of federated learning. Due to differences in medical equipment, data collection standards, and patient populations across hospitals, heterogeneity between data sources is significant. Traditional federated learning methods fail to effectively account for these differences, resulting in insufficient generalization across diverse datasets.

[0006] To address the data heterogeneity problem, FedProx introduces a proximal regularization term on the client side to narrow the gap between the local model and the global model. However, the fixed aggregation weights used in this method may ignore the contributions of certain specific clients. In contrast, FedCE dynamically adjusts the aggregation weights based on client contribution estimates. In addition, Auto-FedRL not only dynamically adjusts the aggregation weights, but also uses reinforcement learning to optimize multiple hyperparameters, such as local training rounds and learning rates. CN119129692A discloses an adaptive constrained federated learning method based on reinforcement learning, comprising: a local reinforcement learning agent unit, which determines the hyperparameters of the action in the current round based on the global model weights and local model weights of the previous round; during the training process, the local model is adaptively constrained based on the hyperparameters, and the local model weights of this round are sent to the server in the federated learning system for aggregation; based on the reinforcement learning agent unit, the reward for the current round is determined based on the global model weights and local model weights corresponding to the previous round and the current round; and a tuple sample of the current round is constructed and stored. Although the above methods can alleviate the impact of data heterogeneity, they all focus on synchronous federated learning.

[0007] FedASMU, based on an asynchronous federated learning architecture, dynamically calculates the time to aggregate the latest global model using a reinforcement learning algorithm during local model training. It also determines the aggregation weight based on the degree of staleness, thereby improving the system's accuracy and efficiency. However, this method requires two additional client-server communications during local training, increasing communication costs.

[0008] In summary, synchronous federated learning suffers from the "slow node problem," reducing system efficiency. Furthermore, differences in device performance and data make traditional methods often underperform when dealing with heterogeneous data. This is especially true for some unique clients, who, despite having less local training data or suboptimal performance during local training, may still contribute significantly to the global model. In asynchronous federated learning, due to differences in training speed across devices, updated local models may become outdated. Current methods do not consider how to simultaneously address both outdatedness and data heterogeneity.

[0009] Therefore, how to improve the performance of asynchronous federated learning against data heterogeneity while ensuring improved efficiency has become a technical challenge that needs to be urgently solved in asynchronous federated learning. Summary of the Invention

[0010] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a dynamic hyperparameter adjustment method, device and medium for asynchronous federated learning. Under the framework of asynchronous federated learning, based on the reinforcement learning (RL) algorithm, the hyperparameters in training (such as aggregation weights, local learning rate, and number of local training rounds) are dynamically adjusted to improve the efficiency of the entire system and shorten the model convergence time while ensuring model performance.

[0011] The purpose of the present invention can be achieved by the following technical solutions:

[0012] According to a first aspect of the present invention, a dynamic hyperparameter adjustment method for asynchronous federated learning is provided. When updating a local model of a client, the method performs hyperparameter adjustment based on a reward function of reinforcement learning. The reward function is divided into two parts, wherein the first part quantifies the reduction in the relative loss of the global model on the client dataset, and the second part quantifies the representation inconsistency between the global model and the local model.

[0013] As a preferred technical solution, the hyperparameter adjustment process of the local model includes the following steps:

[0014] In the client's current local round, a set of hyperparameters is sampled from the current hyperparameter distribution;

[0015] Based on the sampled hyperparameters, calculate the expected probability that the local model will predict the input sample as a certain category;

[0016] Calculate the prediction uncertainty of the input sample based on the Shannon entropy of the expected probability;

[0017] Calculate the reward function value of reinforcement learning in the current local round based on the prediction uncertainty;

[0018] Update the hyperparameter distribution based on the reward function value and enter the next local round of hyperparameter adjustment.

[0019] As a preferred technical solution, for an input sample x, the expected probability that it is predicted by the local model to be of category c is:

[0020]

[0021] Among them, θ is the model parameter, α is the Dirichlet parameter, y is the label value corresponding to the input sample, ρ is the classification prediction value, which is regarded as a random variable following the Dirichlet distribution, α c is the Dirichlet parameter value of category c. In the present invention, α c It can be expressed as α c =ef(x,θ) +1, f(x,θ) is the unnormalized score output by the model, C is the total number of categories, is the expected probability of predicting category c.

[0022] As a preferred technical solution, the Shannon entropy based on expected probability is used to calculate the prediction uncertainty of the input sample as follows:

[0023] The Shannon entropy of the expected probability is used to calculate the prediction uncertainty of the input sample x, which consists of epistemic uncertainty and temporal uncertainty:

[0024] U total (x,θ)=H[P(y|x,θ)]=U epi (x,θ)+U ale (x,θ)

[0025] Among them, U total (x,θ) represents the prediction uncertainty, U epi (x,θ) represents the uncertainty of knowledge, U ale (x,θ) represents the temporal uncertainty, and H(·) represents the Shannon entropy;

[0026] The epistemic uncertainty is expressed as:

[0027]

[0028] The time uncertainty is expressed as:

[0029]

[0030] where ψ(·) is the digamma function, Express expectations.

[0031] As a preferred technical solution, the calculation method of the reward function value is:

[0032]

[0033] in, represents the qth k The reward function value of local rounds, represents the qth k The time uncertainty of a local round, represents the qth k The uncertainty of the local round of knowledge, represents the qth k The loss value of each local round.

[0034] As a preferred technical solution, the update of the hyperparameter distribution based on the reward function value is specifically as follows:

[0035] Use the moving average window to smooth the reward function values of the most recent rounds and update the hyperparameter distribution:

[0036]

[0037] in, is the qth k The hyperparameter distribution of local rounds, Z is the window size, γ h is the learning rate, represents the qth k The reward function value of local rounds, represents the derivative symbol, represents the hyperparameter combination of the τth local round of client k, It means that the hyperparameter combination of client k in the τth local round is The probability of Indicates that client k is in the qth k The average of the latest Z reward functions in the local round.

[0038] As a preferred technical solution, the training process of asynchronous federated learning includes the following steps:

[0039] The central server distributes the current global model to all participating clients;

[0040] After receiving the current global model, each client adjusts the hyperparameters using the dynamic hyperparameter adjustment method and uses the adjusted hyperparameters to perform local model training. The hyperparameters include the number of local training rounds, learning rate, and aggregation weight.

[0041] When any client completes local model training, the client asynchronously uploads the local model and aggregated weights to the central server;

[0042] The central server aggregates the model based on the local model and aggregation weight sent by the client and updates the global model;

[0043] After the aggregation is completed, the central server sends the updated global model to the corresponding clients for the next round of local model training.

[0044] As a preferred technical solution, the model aggregation method is:

[0045] Θ t+1 =(1-β k )·Θ t +β k ·Θ k

[0046] Among them, Θ t is the global model of the tth global round, Θ k is the local model uploaded by client k, β k is the aggregate weight of client k.

[0047] According to a second aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the method when executing the program.

[0048] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method described above is implemented.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] (1) The present invention proposes a dynamic hyperparameter adjustment method under the asynchronous joint learning framework. By utilizing the REINFORCE reinforcement learning algorithm, the method can adjust the hyperparameters according to the current training results without the need to understand the prior knowledge related to the system, thereby improving the efficiency and flexibility of the system in heterogeneous environments.

[0051] (2) In response to the data heterogeneity problem existing in the existing technology, the present invention introduces a new reward function to optimize the hyperparameters, which not only takes into account the reduction value of the relative loss, but also takes into account the cognitive uncertainty of the global model and the estimation uncertainty of the local model, thereby improving the generalization ability of the model in the case of data heterogeneity.

[0052] (3) In response to the problem of increased communication costs in the existing technology, the present invention fixes the model aggregation time. The client only communicates with the server after local training is completed, and synchronously optimizes the global model aggregation weight and the number of local training rounds based on the reinforcement learning algorithm, thereby effectively alleviating the model obsolescence problem caused by the fixed aggregation time. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of the method of the present invention;

[0054] Figure 2 Schematic diagram of asynchronous federated learning of the present invention, where (a) is a schematic diagram of the training process of the tth global round, and (b) is a schematic diagram of the asynchronous federated learning framework using three clients as an example;

[0055] Figure 3 A comparison chart of training results of the present invention and the prior art in one embodiment;

[0056] Figure 4Schematic diagram of the changes in the aggregate weights of different clients during the training process of the present invention in one embodiment. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0058] Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by a person of ordinary skill in the technical field to which this application belongs. The words "one", "a", "the" and the like used in this application do not indicate a limit on quantity and may indicate the singular or plural. The terms "include", "comprise", "have" and any variations thereof used in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units that are inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The word "multiple" used in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0059] This paper proposes a dynamic hyperparameter tuning method for an asynchronous federated learning architecture. It uses a policy-based reinforcement learning algorithm to dynamically optimize hyperparameters, including aggregation weights, learning rate, and number of local training rounds. Each client has its own RL agent and adjusts hyperparameters based on a reward function. The reward function is divided into two parts: the first part quantifies the reduction in relative loss of the global model on the client dataset, and the second part quantifies the representation inconsistency between the global model and the local model. The main contributions of this paper are summarized as follows:

[0060] (1) This paper proposes an asynchronous federated learning framework AFedRL, which can dynamically adjust hyperparameters to improve the efficiency and flexibility of the system in heterogeneous environments.

[0061] (2) This paper introduces a new reward function to optimize hyperparameters, which not only takes into account the reduction of relative loss, but also takes into account the cognitive uncertainty of the global model and the estimation uncertainty of the local model, thereby improving the generalization ability of the model in the case of heterogeneous data.

[0062] In one embodiment, Figure 1 As shown in Figure 2, the hyperparameter tuning process for the local model includes the following steps:

[0063] S1, in the current local round q of client k k From the current hyperparameter distribution Sample a set of hyperparameters from .

[0064] The goal of the RL agent in each round is to maximize the cumulative reward, which is:

[0065]

[0066] Therefore, the inclusion of a reward function is crucial to the training process. In Auto-FedRL, the reward function is based solely on the reduction of relative loss. To address the "slow node problem" and enhance generalization, this example incorporates both the epistemic uncertainty of the global model and the estimated uncertainty of the local model into the reward function. These uncertainties are quantified using a Dirichlet-based evidence model, specifically implemented through the following steps.

[0067] S2, based on the sampled hyperparameters, calculates the expected probability that the local model predicts the input sample as a certain category.

[0068] For an input sample x, the expected probability that it is predicted by the local model to be category c is:

[0069]

[0070] Among them, θ is the model parameter, α is the Dirichlet parameter, y is the label value corresponding to the input sample, ρ is the classification prediction value, which is regarded as a random variable following the Dirichlet distribution, α c is the Dirichlet parameter value of category c. In the present invention, α c It can be expressed as α c =e f(x,θ) +1, f(x,θ) is the unnormalized score output by the model, C is the total number of categories, is the expected probability of predicting category c.

[0071] S3, calculates the prediction uncertainty of the input sample based on the Shannon entropy of the expected probability.

[0072] The Shannon entropy of the expected probability is used to calculate the prediction uncertainty of the input sample x. The prediction uncertainty consists of epistemic uncertainty and temporal uncertainty:

[0073] U total (x,θ)=H[P(y|x,θ)]=U epi (x,θ)+U ale (x,θ)

[0074] Among them, U total (x,θ) represents the prediction uncertainty, U epi (x,θ) represents the uncertainty of knowledge, U ale (x, θ) represents temporal uncertainty, and H(·) represents Shannon entropy. Temporal uncertainty captures the inherent complexity of local data and quantifies the reduction in relative loss of the global model on the client dataset. Epistemic uncertainty, on the other hand, reflects the uncertainty associated with the domain shift between the global model and local data and quantifies the representational inconsistency between the global and local models. Therefore, lower uncertainty in the local model indicates higher feasibility, while lower uncertainty in the global model indicates better generalization across all clients.

[0075] Among them, epistemic uncertainty is expressed as:

[0076]

[0077] The time uncertainty is expressed as:

[0078]

[0079] where ψ(·) is the digamma function, Express expectations.

[0080] S4, calculates the reward function value of reinforcement learning in the current local round based on the prediction uncertainty:

[0081]

[0082] in, represents the qth k The reward function value of local rounds, represents the qth k The time uncertainty of a local round, represents the qth k The uncertainty of the local round of knowledge, represents the qth k The loss value of each local round.

[0083] S5: Update the hyperparameter distribution based on the reward function value and enter the next local round of hyperparameter adjustment.

[0084] In each round q k Update hyperparameter distribution In order to dynamically adjust parameters in each round, this embodiment uses a moving average window to smooth the rewards of the most recent rounds to update the hyperparameter distribution, rather than relying on the cumulative rewards of all rounds, that is:

[0085]

[0086] in, is the qth k The hyperparameter distribution of local rounds, Z is the window size, γ h is the learning rate, represents the qth k The reward function value of local rounds, represents the derivative symbol, represents the hyperparameter combination of the τth local round of client k, It means that the hyperparameter combination of client k in the τth local round is The probability of Indicates that client k is in the qth k The average of the latest Z reward functions in the local round.

[0087] Based on the hyperparameter adjustment method of the above local model, such as Figure 2 As shown in Figure 2, the training process of asynchronous federated learning includes the following steps:

[0088] A1, in the tth global round, the central server sends the current global model Θ t Distribute to all participating clients;

[0089] A2, each client receives the current global model Θ t Then, the hyperparameters are adjusted using the dynamic hyperparameter adjustment method of steps S1 to S5 above, and the adjusted hyperparameters are used for local model training. The hyperparameters include the number of local training rounds E k , learning rate γ k and the aggregation weight β k ;

[0090] A3: When any client k completes the local model training, the client k asynchronously transfers the local model Θ to the k and the aggregation weight β k Upload to the central server; in this step, each client can upload its results immediately after completing local training;

[0091] A4, the central server is based on the local model Θ sent by the client k and the aggregation weight β k Perform model aggregation and update the global model:

[0092] Θ t+1 =(1-β k )·Θ t +β k ·Θ k

[0093] Among them, Θ t is the global model of the tth global round, Θ k is the local model uploaded by client k, β k is the aggregate weight of client k.

[0094] A5, after the aggregation is completed, the central server sends the updated global model to the corresponding client k for the next round of local model training.

[0095] To demonstrate the effectiveness of the present invention, this example uses two medical image segmentation datasets: a prostate MRI dataset from six institutions, and a retinal fundus image dataset from six sources. Each data source is treated as a separate client, and the dataset is divided into a 50% training set, a 25% validation set, and a 25% test set. Segmentation performance is evaluated using the Dice coefficient, and training efficiency is assessed by tracking convergence time.

[0096] Specifically, in this embodiment, all medical imaging data are uniformly preprocessed to a resolution of 256×256 pixels. The model training environment is deployed on a computing node with two NVIDIA RTX 4090 GPUs (24GB of video memory). The network architecture uses 2D U-Net as the basic segmentation model, whose optimization objective is measured by the Dice loss function. The Adam optimizer is used for parameter update, the initial learning rate is set to 1e-3, the momentum parameter combination is (0.9, 0.99), and the sample size of each batch is 32. To ensure model convergence, 200 global training rounds are set in synchronous federated learning, and each client performs one local model update per round; asynchronous federated learning is extended to 1200 global rounds. In particular, in the method of the present invention, the RL agent uses an independent Adam optimizer with a learning rate of 1e-2. To realistically simulate the operating environment of distributed medical equipment, in the prostate MRI segmentation task, this embodiment artificially introduces a delay mechanism for clients 2 and 5 to double the single-round training time, thereby constructing a system heterogeneity test scenario.

[0097] This example compares the present invention with eight federated learning methods, including synchronous federated learning methods (e.g., FedAvg, q-FedAvg, CFFL, FedCI, CGSV, and FedCE) and asynchronous federated learning methods (e.g., Fedasyn(const) and FedIBD(DW)). Furthermore, a comparison is made with independent training methods, where the model is trained and evaluated independently on each client. All methods share the same training configuration.

[0098] Table 1 lists the results of each method on the two segmentation tasks, where (#) represents the size of the training set on that client. On the retinal fundus dataset, due to the large heterogeneity of the training data on client 5, the performance of the methods that cannot address this difference degrades. However, the performance of the present invention on client 5 is 15.46% higher than the second best method and significantly reduces the standard deviation. In addition, the average performance of the present invention across all clients exceeds all baseline methods. For the prostate dataset, all AFL methods experience performance degradation when an artificial time delay is introduced for the two clients. However, the performance degradation of the present invention is the smallest, especially compared to Fedasyn (const) and FedIBD (DW). This shows that it is robust to data and device heterogeneity.

[0099] Table 1 Dice coefficients of various methods on two datasets

[0100]

[0101] In addition, if Figure 3 As shown, the present invention converges faster than FedAvg in both segmentation tasks. The improvement in convergence speed is particularly noticeable in the prostate MRI segmentation task, as exemplified by client 5 of the retinal fundus dataset. The present invention reached convergence in approximately 1.11 hours, while FedAvg required approximately twice as long. This significant reduction in training time highlights the efficiency of the present invention.

[0102] In order to intuitively show the changes in hyperparameters during training, this embodiment Figure 4 The aggregate weights for each round of training for the retinal fundus task are plotted in Figure 2. Later in training, Clients 2 and 5 achieve larger aggregate weights, indicating that the RL agent recognizes their significant contribution to the global model. This suggests that while the datasets for these two clients are smaller than those for the other clients, they contain more valuable information. This weight adjustment also significantly improves the performance of our method on Client 5.

[0103] To investigate the impact of dynamically adjusting each hyperparameter on the performance of the two segmentation tasks, this example conducted an ablation study. As shown in Table 2, the performance of the global model improves with increasing the number of dynamically adjusted hyperparameters. This demonstrates that the choice of learning rate, number of local training epochs, and aggregate weights plays a key role in model performance in AFL.

[0104] Table 2 Ablation experiments on search space

[0105]

[0106] Extensive experiments were conducted on prostate MRI and retinal fundus image datasets, with artificially added time delays to simulate client-side heterogeneity caused by network or computing power. The experimental data demonstrates that the proposed method significantly reduces model training time and improves system efficiency while maintaining model performance.

[0107] In one embodiment, the electronic device includes a computing unit that can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). Various programs and data required for device operation can also be stored in the RAM. The computing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0108] Many components in an electronic device are connected to the I / O interface, including: input units, such as a keyboard and mouse; output units, such as various types of displays and speakers; storage units, such as magnetic disks and optical disks; and communication units, such as network cards, modems, and wireless communication transceivers. The communication unit allows the device to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks.

[0109] The computing unit can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of computing units include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit performs the various methods and processes described above, such as image processing methods and / or model training methods. For example, in some embodiments, the image processing methods and / or model training methods can be implemented as computer software programs that are tangibly contained in machine-readable media, such as storage units. In some embodiments, part or all of the computer program can be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the computing unit, one or more steps of the image processing methods and / or model training methods described above can be performed. Alternatively, in other embodiments, the computing unit can be configured to perform image processing methods and / or model training methods in any other appropriate manner (e.g., by means of firmware).

[0110] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0111] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0112] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0113] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A dynamic hyperparameter adjustment method for asynchronous federated learning, characterized in that: When updating the client's local model, this method performs hyperparameter adjustment based on a reinforcement learning reward function. The reward function is divided into two parts. The first part quantifies the reduction in the relative loss of the global model on the client dataset, and the second part quantifies the representation inconsistency between the global model and the local model.

2. A dynamic hyperparameter adjustment method for asynchronous federated learning according to claim 1, characterized in that: The hyperparameter tuning process of the local model includes the following steps: In the client's current local round, a set of hyperparameters is sampled from the current hyperparameter distribution; Based on the sampled hyperparameters, calculate the expected probability that the local model will predict the input sample as a certain category; Calculate the prediction uncertainty of the input sample based on the Shannon entropy of the expected probability; Calculate the reward function value of reinforcement learning in the current local round based on the prediction uncertainty; Update the hyperparameter distribution based on the reward function value and enter the next local round of hyperparameter adjustment.

3. A dynamic hyperparameter adjustment method for asynchronous federated learning according to claim 2, characterized in that: For an input sample x, the expected probability that it is predicted by the local model to be category c is: Among them, θ is the model parameter, α is the Dirichlet parameter, y is the label value corresponding to the input sample, ρ is the classification prediction value, which is regarded as a random variable following the Dirichlet distribution, α c is the Dirichlet parameter value of category c, α c Represented as α c =e f(x,θ) +1, f(x,θ) is the unnormalized score output by the model, C is the total number of categories, is the expected probability of predicting category c.

4. A dynamic hyperparameter adjustment method for asynchronous federated learning according to claim 3, characterized in that: The Shannon entropy based on expected probability is used to calculate the prediction uncertainty of the input sample as follows: The Shannon entropy of the expected probability is used to calculate the prediction uncertainty of the input sample x, which is composed of the epistemic uncertainty and the temporal uncertainty. composition: U total (x,θ)=H[P(y|x,θ)]=U epi (x,θ)+U ale (x,θ) Among them, U total (x,θ) represents the prediction uncertainty, U epi (x,θ) represents the uncertainty of knowledge, U ale (x,θ) represents the temporal uncertainty, and H(·) represents the Shannon entropy; The epistemic uncertainty is expressed as: The time uncertainty is expressed as: where ψ(·) is the digamma function, Express expectations.

5. The method for dynamic hyperparameter adjustment for asynchronous federated learning according to claim 2, characterized in that: The calculation method of the reward function value is: in, represents the qth k The reward function value of local rounds, represents the qth k The time uncertainty of a local round, represents the qth k The uncertainty of the local round of knowledge, represents the qth k The loss value of each local round.

6. A dynamic hyperparameter adjustment method for asynchronous federated learning according to claim 2, characterized in that: The specific method of updating the hyperparameter distribution based on the reward function value is: Use the moving average window to smooth the reward function values of the most recent rounds and update the hyperparameter distribution: in, is the qth k The hyperparameter distribution of local rounds, Z is the window size, γ h is the learning rate, represents the qth k The reward function value of local rounds, represents the derivative symbol, represents the hyperparameter combination of the τth local round of client k, It means that the hyperparameter combination of client k in the τth local round is The probability of Indicates that client k is in the qth k The average of the latest Z reward functions in the local round.

7. The method for dynamic hyperparameter adjustment for asynchronous federated learning according to claim 1, characterized in that: The training process of asynchronous federated learning includes the following steps: The central server distributes the current global model to all participating clients; After receiving the current global model, each client adjusts the hyperparameters using the dynamic hyperparameter adjustment method and uses the adjusted hyperparameters to perform local model training. The hyperparameters include the number of local training rounds, learning rate, and aggregation weight. When any client completes local model training, the client asynchronously uploads the local model and aggregated weights to the central server; The central server aggregates the model based on the local model and aggregation weight sent by the client and updates the global model; After the aggregation is completed, the central server sends the updated global model to the corresponding clients for the next round of local model training.

8. A method for dynamic hyperparameter adjustment for asynchronous federated learning according to claim 7, characterized in that: The model aggregation method is: I t+1 =(1-β k )·Θ t +b k ·I k Among them, Θ t is the global model of the tth global round, Θ k is the local model uploaded by client k, β k is the aggregate weight of client k.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images