A multi-stage adaptive model training method for high-quality services in elderly care communities

Abstract

The present invention is a multi-stage adaptive model training method for high-quality services in a retirement community. A smart device is randomly selected in the retirement community as a server, and other smart devices are clients. The difference item of the local model is calculated based on the global model and the local model of the previous round, and then the local model difference item is extracted by a dynamic weighting method to correct the local model, and the corrected local model is used as the starting model of a new round of local training. The model is locally updated to obtain a new round of local model; the updated local model parameter W of the tth round is j,t and gradient information #imgabs0# are transmitted to the server; the server first uses the aggregate weights and local model aggregation to generate a new round of global model W t , and then according to the gradient information and the local model W j,t The aggregation weights are constrained and optimized, and the above process is iterated continuously until the training is completed. The training direction is adjusted in stages to ensure the high efficiency of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of edge environment of elderly care communities, and in particular to a multi-stage adaptive model training method for high-quality services in elderly care communities. Background Art

[0002] With the development of artificial intelligence and the intensification of the aging trend of the social population, smart elderly care communities have emerged as a new type of elderly care solution. However, due to the necessity of privacy protection, the edge devices in the elderly care community cannot directly transmit data to the cloud for model training, which directly leads to the difficulty of the model to provide high-quality services. In order to effectively improve the quality of intelligent services in the elderly care community, such as realizing stranger face recognition and dangerous behavior detection, the use of federated learning to train models for elderly care community services is undoubtedly a very feasible solution. However, in actual situations, there are significant differences in the user population and environment of different devices in the community, which makes the data show obvious heterogeneity. More importantly, the individuals to whom the devices belong may not always be willing to let the devices participate in training, which may cause these devices to join or exit the current training task at any time, thereby causing the heterogeneity of the training data set to continue to change.

[0003] Federated learning, as a distributed model training method, has become a popular solution for protecting data privacy and security. In a federated learning system, the model training process does not rely on centralizing data to a central server, but allows each participating node to train on local data and periodically send local model updates to the central server for aggregation. In this way, the data is always retained on the local device, effectively protecting user privacy. The training process mainly includes the following steps: First, the central server initializes a global model and sends it to all participating nodes. Each node trains the model on local data to generate updated local model parameters. Then, each node sends the local model parameters back to the central server, which aggregates these parameters, usually using a weighted average method, to obtain an updated global model. This process is repeated until the global model converges. In each round, the participating nodes may temporarily exit or enter a round of training for various reasons, causing changes in the training data. Due to the different data distributions of different nodes, the training data set participating in each round is different from the overall distribution, which affects the effect of the global model.

[0004] This has led to the following serious problems that need to be solved urgently. 1) Inaccurate aggregation weights: In a changing data environment, local models at different stages contribute differently to the global model. Inaccurate aggregation weights can easily lead to accelerated growth of aggregation deviations, resulting in inefficient global models. 2) The state of the global model with a fixed weight learning rate depends on the constant changes in the local model. The update step size of the aggregation weights determines the convergence speed of the global model. Too large a step size can easily lead to falling into a local optimal solution, while too small a step size can easily lead to slow convergence. 3) The multi-faceted impact of data heterogeneity: In a dynamic training environment, data heterogeneity not only affects the update of local models, but also leads to inefficient aggregation and slow training.

[0005] Due to the large differences in data distribution on different devices, local models are prone to deviate from the global model, resulting in the global model after aggregation often being difficult to converge effectively and having poor performance. In addition, most existing methods statically weight the data based on the size of the client data, but this easily leads to the global model being unable to represent the global data and being more biased towards the data distribution of clients with large amounts of data. For clients with small amounts of data or scarce features, the accuracy is low, and the impact of the aggregation weight is ignored, resulting in less than ideal results in its application in smart elderly care communities. Summary of the invention

[0006] In view of the various types of smart devices in retirement communities, the large differences in usage methods lead to uneven distribution of device data, large differences in local model training directions of different devices, difficulty in measuring model contribution, and the loss of connection or addition of devices will lead to heterogeneous changes in the training data sets, resulting in the inability to guarantee the performance of the aggregated global model. The present invention provides a multi-stage adaptive model training method for high-quality services in retirement communities. By adjusting the training direction in stages, not only the training speed of local models and global models of smart devices in retirement communities is improved, but also the efficiency of the model is guaranteed.

[0007] To achieve the above object, the technical solution adopted by the present invention is:

[0008] In a first aspect, the present invention provides a multi-stage adaptive model training method for high-quality services in a retirement community, wherein a randomly selected smart device in the retirement community is used as a server, and other smart devices are clients, all smart devices have computing and storage capabilities, and data transmission and communication can be performed between devices;

[0009] Step 1: Local training:

[0010] Based on the global model and the local model of the previous round, the difference item of the local model is calculated, and then the local model difference item is extracted using a dynamic weighting method to correct the local model. The corrected local model is used as the starting model for a new round of local training. After the model is locally updated, a new round of local model is obtained. Before executing a new round of local training, the smart device of each client role will reinitialize its local model according to formula (3):

[0011]

[0012] Among them, the former | W j |-All elements of the 1st layer are 1, and the top layer uses β j,t express; is the starting model of the tth round of local model training; W j,t-1 is the local model of client j after the (t-1) round of local training is completed; W t-1 is the global model after the (t-1)th round of aggregation; W t-1 -W j,t-1 is the local model difference term; |W j | is the number of layers of the local model;

[0013] Only β j,t Before initializing a new round of local models, use the previous round of global model W t-1 The gradient information of local data is updated according to formula (4):

[0014]

[0015] Among them, η β is a constant learning rate, generally set to 0.001; W t-1 is the global model after the t-1th round of aggregation; D j is the data sampled from the jth smart device in the elderly care community, j = 1, 2, ..., M, M is the number of smart devices acting as clients in the elderly care community; F j represents the loss function on client j; β j,t-1 is the top layer of round t-1;

[0016] Step 2: Dynamic aggregation:

[0017] Updated local model parameters W for round t j,t and gradient information is sent to the server;

[0018] Gradient descent is used to synchronously calculate the weight learning paths of different clients in the current round and update their respective aggregation weights, ensuring that the updated aggregation weight is the optimal aggregation weight that can reduce the current global model loss the fastest. The update formula of the aggregation weight is formula (7):

[0019]

[0020] Among them, α j,t is the aggregate weight corresponding to client j in round t; α j,t+1 is the aggregate weight corresponding to client j in round t+1; η j，t is the dynamic learning rate of the weight update of client j in round t, which is a tensor with the same structure as the local model parameters and is obtained according to formula (8), where γ is a constant;

[0021] η j,t ☉W j，t =γ (8)

[0022] So far, a round of adaptive model training has been completed.

[0023] The server first uses the aggregation weights and local model aggregation to generate a new round of global model W t , and then according to the gradient information and the local model W j ,t performs constrained optimization on the aggregation weights by continuously iterating the local training and dynamic aggregation process until the training is completed;

[0024] At this time, the global model on the server side provides intelligent services for the high-quality services of the retirement community.

[0025] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0026] There are M+1 smart devices, one of which is a server and the others are clients. All smart devices have computing and storage capabilities, and data transmission and communication can be performed between devices. Based on federated learning, local training and dynamic aggregation training are performed. The server loads the global model and the client loads the local model.

[0027] Step 1: Local training:

[0028] Based on the global model and the local model of the previous round, the difference item of the local model is calculated, and then the local model difference item is extracted using a dynamic weighting method to correct the local model. The corrected local model is used as the starting model for a new round of local training. After the model is locally updated, a new round of local model is obtained. Before executing a new round of local training, the smart device of each client role will reinitialize its local model according to formula (3):

[0029]

[0030] Among them, the former | W j |-All elements of the 1st layer are 1, and the top layer uses βj,t express; is the starting model of the tth round of local model training; W j,t-1 is the local model of client j after the (t-1) round of local training is completed; W t-1 is the global model after the (t-1)th round of aggregation; W t-1 -W j,t-1 is the local model difference term; |W j | is the number of layers of the local model;

[0031] Only β j,t Before initializing a new round of local models, use the previous round of global model W t-1 The gradient information of local data is updated according to formula (4):

[0032]

[0033] Among them, η β is a constant learning rate, generally set to 0.001; W t-1 is the global model after the t-1th round of aggregation; D j is the data sampled from the jth smart device in the elderly care community, j = 1, 2, ..., M, M is the number of smart devices acting as clients in the elderly care community; F j represents the loss function on client j; β j,t-1 is the top layer of round t-1;

[0034] Step 2: Dynamic aggregation:

[0035] Updated local model parameters W for round t j,t and gradient information is sent to the server;

[0036] Gradient descent is used to synchronously calculate the weight learning paths of different clients in the current round and update their respective aggregate weights. The update formula of the aggregate weight is formula (7):

[0037]

[0038] Among them, α j,t is the aggregate weight corresponding to client j in round t; α j,t+1 is the aggregate weight corresponding to client j in round t+1; η j，t is the dynamic learning rate of the weight update of client j in round t, which is a tensor with the same structure as the local model parameters and is obtained according to formula (8);

[0039] η j,t ☉W j，t =γ (8)

[0040] The server first uses the aggregation weights and local model aggregation to generate a new round of global model W t , and then according to the gradient information and the local model W j,t Constrained optimization of the aggregation weights is performed by continuously iterating the local training and dynamic aggregation process until the training is completed;

[0041] When generating a new global model, the average gradient of each round of training is used to correct the aggregation direction of each local model. Then, it is used as a regularization term ΔW to constrain the aggregation behavior, as shown in formula (9):

[0042]

[0043] Among them, S is the number of clients that can participate in training in each round; ΔW represents the regularization term, is the loss function F on client i i Find the partial derivative of the global model for client i in round t; γ is a constant;

[0044] This completes the adaptive model training.

[0045] The above storage medium can obtain a more efficient model, provide higher quality services, have a faster speed under the same model training accuracy, and improve the hardware training efficiency.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The present invention takes into account the impact of data heterogeneity of smart devices in the elderly care community at different stages, redefines the expression of the learning rate of the aggregate weight, reflects the specific contribution of the local models of different devices in the aggregation in more detail, and uses local models to constrain the learning rate of the weight to keep all local models updated at the same rate to accelerate model convergence. The learning direction and update step size of the weight are recalculated on each round of training data. By keeping the weight update target consistent with the model training target, the inconsistency of data heterogeneity is overcome, ensuring that the learning direction of the aggregate weight is always along the optimal direction of the training target at different stages. By updating the aggregate weights with a constrained dynamic learning rate, it is possible to accurately extract general information from the local models involved in the training, successfully reduce the growth rate of the aggregation error during the training process, and generate an efficient global model for high-quality services in elderly care communities. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 :An example diagram of the equipment environment of a retirement community.

[0049] Figure 2 : Workflow diagram of the multi-stage adaptive model training method for high-quality services in retirement communities.

[0050] Figure 3:The aggregation error trend of the classic algorithm FedAvg under different client participation rates (ratio).

[0051] Figure 4 : Diagram of the learning process of the dynamic aggregation stage of the present invention.

[0052] Figure 5 : Performance evaluation diagram of the present invention on different data sets.

[0053] Figure 6 : Evaluation diagram of the growth rate of aggregation error under the present invention.

[0054] Figure 7 : Loss convergence trend diagram of the present invention. DETAILED DESCRIPTION

[0055] The present invention is further explained below in conjunction with the embodiments and drawings, but this is not intended to limit the scope of protection of the present application.

[0056] like Figure 1 As shown, smart elderly care communities need to provide a variety of intelligent services such as face recognition, behavior detection, etc. The quality of these services depends on the efficiency of the models they rely on. The present invention aims to achieve high efficiency of its service model through multi-stage adaptive model training.

[0057] When training models in a retirement community environment, it is necessary not only to protect the privacy of residents' data, but also to consider the efficiency of the training model. Based on the idea of ​​a federated learning training model, the present invention trains local models on data on different smart devices in the retirement community and performs weighted aggregation. In view of the large data differences and low aggregation efficiency caused by the usage methods or environment of these devices, the present invention uses a multi-stage adaptive optimization method to extract common information to the maximum extent to generate a more efficient global model. Different from a single local optimization method, this multi-stage optimization can solve the impact of local adaptation on other training stages, and the distribution overcomes the impact of data heterogeneity. While avoiding the risk of resident data leakage, high-quality models are obtained to empower equipment and improve the quality of intelligent services in retirement communities.

[0058] The present invention randomly selects a smart device in the elderly care community as a server, and other smart devices are clients. These devices can be smart devices with computing and storage capabilities such as gateways and cameras. Data transmission and communication can be carried out between devices. Figure 2 As shown in the figure, S devices are selected as the clients currently participating in the training, and one device is selected as the server. The global model is downloaded on the server, and the local model training is performed on the client. The maximum number of training rounds is T.

[0059] Adaptive model training consists of two stages: local model training of smart devices and global dynamic aggregation of all device models. Local models are trained on all smart devices, the global model is aggregated on one of the devices (server), and the gap between the local model and the global model is calculated as the difference term of the local model; at the beginning of local model training, each smart device uses the difference term as a regular term to constrain the new round of local models, and uses local data to adjust the weight coefficient of the regular term.

[0060] The dynamic aggregation process is a fine-grained adaptive weight process based on a constrained dynamic learning rate. Before each round of aggregation, the smart devices selected as servers in the elderly care community can update the aggregated weights in real time based on the data of other smart devices, and realize the adaptive update of the weights by calculating the optimal learning path of the weights. More importantly, the dynamic learning rate is constrained by the current state of the local model on the server role device that performs the aggregation operation, so that the aggregated weights can capture more fine-grained path information while reducing the computational overhead, and can balance the speed of parameter updates of different clients during the aggregation process. This dynamic setting of aggregated weights is designed to optimize the model aggregation process in federated learning FL, thereby reducing the growth rate of aggregation errors and improving model convergence and performance.

[0061] In this embodiment, it is assumed that there are M+1 smart devices in the retirement community, and these devices are used by different users. M of the smart devices are selected as the client role for training the local model, and the remaining device acts as the server role. Obviously, the training data set composed of these devices is not independent and identically distributed. The training goal of the present invention is to minimize the loss F(W t ), as shown in formula (1), to obtain an efficient model to provide high-quality services for devices in the elderly care community. Where t represents the current communication round, D j is the sample set of client j, W t represents the global model obtained from t rounds of aggregation.

[0062]

[0063] Here, {D1, ...D j , ..., D M} is the data sampled by M smart devices in the elderly care community, f p represents the loss value of sample p, and D is the total data size on all devices. j (W t ) is the loss function on client j. Due to different reasons (such as users changing devices or adding new devices), the heterogeneous changes in the training data lead to inconsistency with the population. Figure 3The aggregation error trend of the classic algorithm FedAvg under different client participation rates (ratio) proves that this inconsistency leads to an accelerating growth trend of the aggregation error. This brings challenges to the realization of efficient model training. The reasonable allocation of weights in the dynamic aggregation stage in the present invention is an effective means to solve this problem. The specific process of the multi-stage adaptive model training method is:

[0064] 1. Local Training

[0065] Before executing a new round of local training, the smart device of each client role will reinitialize its local model according to formula (3). The initialization process not only uses the global model, but also considers the gradient information of the global model to the local data, overcoming the deficiency of the existing technology that only uses the global model to cover the local model.

[0066]

[0067] Among them, the former | W j |-All elements of the 1st layer are 1, and the top layer uses β j,t express; is the starting model of the tth round of local model training; W j,t-1 is the local model of client j after the (t-1) round of local training is completed; W t-1 is the global model after the (t-1)th round of aggregation; W t-1 -W j,t-1 is the local model difference term; |W j | is the number of layers of the local model;

[0068] Only β j,t Before initializing a new round of local models, use the previous round of global model W t-1 The gradient information of local data is updated according to formula (4):

[0069]

[0070] Among them, η β is a constant learning rate, generally set to 0.001; W t-1 is the global model after round t-1 aggregation; D j is the data sampled from the jth smart device in the elderly care community, j = 1, 2, ..., M, M is the number of smart devices acting as clients in the elderly care community; F j represents the loss function on client j; β j,t-1 is the top layer of round t-1;

[0071] 2. Dynamic Aggregation

[0072] In order to obtain as much global information as possible from other smart devices, it is also necessary to adapt to the real-time changes of smart devices and continuously adjust the aggregation direction of local models. The present invention adopts constrained dynamic learning rate weight adaptation to achieve more accurate weight update, thereby improving aggregation efficiency.

[0073] Considering that some smart devices may not actually participate in the entire training process, in this example, S devices participate in the tth round of training, and the participating client device set is {D1,D2,......,D s}, α j,t is the aggregate weight corresponding to client j in round t, T is the total number of training rounds, S is the number of clients that can participate in training in each round, and D is the total dataset; therefore, the global model can be expressed by formula (5):

[0074]

[0075] Obviously, the aggregation weights and local models of different smart devices in each round determine the aggregation efficiency of the round. The present invention adjusts the update direction of the local model (i.e., formula (4)) during the local training phase of the smart device, and uses a dynamic learning rate to dynamically update the aggregation weight. The aggregation weight in the present invention is neither a constant nor a single variable, but a high-dimensional tensor consistent with the local model structure, which can capture the differences at different levels of the local model, even if the differences are very small.

[0076] The aggregation step can actually be regarded as part of the global optimization. The aggregation weight optimization objective is aligned with the model optimization objective through formula (6), and the dynamic learning rate is used to update the aggregation weight more accurately.

[0077]

[0078] The present invention points out that the direction in which the global loss decreases fastest is the direction of the optimal aggregation weight. αj,t represents the aggregation weight used by the local model of client j after the tth communication. Figure 4 This is the learning process of the dynamic aggregation stage. During training, gradient descent is used to synchronously calculate the weight learning paths of different clients in this round and update their respective aggregation weights. This method ensures that the updated aggregation weight is the optimal aggregation weight that can reduce the current global model loss the fastest. Specifically, at the initial stage of training, the element value in the weight tensor is initialized to the data volume ratio on the corresponding client. Then, before each round of aggregation operation is executed, the weight learning path is calculated based on all the gradient information of the participating clients in this round. The dynamic learning rate is a tensor with the same structure as the local model parameters, which contains higher information entropy and allows the weights to obtain a more accurate update step. It allows the weights to be learned in more detail in different dimensions. In addition, the present invention uses the local model to constrain the dynamic learning rate so that the aggregate weight update will not cause the global model to be unstable due to excessive changes.

[0079] Formula (7) is the weight update formula.

[0080]

[0081] Among them, α j,t is the aggregate weight corresponding to client j in round t; α j,t+1 is the aggregate weight corresponding to client j in round t+1; η j，t is the dynamic learning rate of the weight update of client j in round t, which is a tensor with the same structure as the local model parameters and is obtained according to formula (8);

[0082] η j,t ☉W j，t =γ (8)

[0083] In fact, the average gradient of each round of training is used to correct the aggregation direction of each local model when generating a new global model, and then it is used as a regularization term ΔW to constrain the aggregation behavior as shown in formula (9).

[0084]

[0085] This ensures that changes in aggregate weights are constrained by the local model and remain consistent. The reason for this design is that the data of different devices in the elderly care community are very different, resulting in the greater contribution of its local model to the global model. The smaller the learning rate required to adjust it during the update should be to avoid excessive impact on the overall model. Conversely, the smaller its impact on the entire global model, the larger the updated learning rate, which can be used to adapt to changes in the global model faster. In addition, by ensuring that the dynamic learning rate and local model parameters W j,t The product of the local model is a constant, which keeps the update rate of the aggregate weights consistent, and all client devices are constrained to the same update rate. Achieving a balance in update rates helps speed up convergence and improve global model adaptability.

[0086] In the elderly care community, one device is used to perform aggregation operations, and other devices are used to train local models. Appropriate constraints are used to control the adaptive adjustment of hyperparameters during local model training and global dynamic aggregation to improve model efficiency, avoid excessive computational overhead, and ensure computational efficiency. The specific execution steps of the program are as follows:

[0087] Input: M (the number of smart devices in the retirement community that can be used for local training), S (the number of devices that can participate in training per round), F (model loss function)

[0088] Output: W (global model)

[0089] Smart devices that perform aggregation operations: Initialize the global model W = W0

[0090] The loop iterates the following steps until it reaches the specified round:

[0091] Smart devices that perform aggregation operations: Select S available devices and send the global model of this round to these devices for local training:

[0092] Step 1: Initialize β j , and then update it using local data.

[0093] Step 2: Based on β j Initialize the local model (Formula (3)).

[0094] Step 3: Train the local model in the device based on local data and gradient algorithm.

[0095] Step 4: Use the trained local model W j and local data gradient Delivered to the smart device that performs the aggregation operation.

[0096] Smart devices that perform aggregate operations:

[0097] 1) Generate a new global model using the aggregated weights and local models.

[0098] 2) Receive the data information of the devices participating in the training, update the step size based on all the gradient information and the local model state constraints according to formula (7), and calculate the local model aggregation weight to be used in the next round of aggregation.

[0099] Even if only a few smart devices participate, the present invention can still control the global model so that it does not deviate too much from these customers; at the same time, the global information of these devices is maximized. The main reason is that the aggregate weight of each local model continuously adjusts the update direction to make it consistent with the global optimization goal. In extreme cases, such as when all clients can participate in model training or the data is independently and identically distributed (which does not happen in reality), it may be downgraded, resulting in results close to FedALA. The present invention is more robust and efficient than other training methods. Dynamic weighting effectively overcomes the problems caused by extracting local model deviation information during the aggregation process. The constrained dynamic learning rate can also ensure that the updated step size is consistent with the heterogeneous change trend of the data, achieving more accurate updates without the problem of excessive step size. Therefore, the model will not be biased towards local models that have participated in more training times. The present invention uses global loss as the objective function and updates the aggregate weights in the negative gradient direction to minimize the loss. More importantly, when updating the weights, the dynamic learning rate avoids model dispersion caused by significant changes in model parameters during the update process.

[0100] This example uses Dirichlet distribution to sample data to represent a real-world heterogeneous environment. Each client receives a unique dataset according to the distribution parameter configuration. When the parameter value decreases, the data differences between clients increase and the distribution of the dataset becomes more heterogeneous. Throughout the investigation, the test accuracy and the number of communication rounds to reach the specified accuracy are used as evaluation metrics. The mean and standard deviation are calculated and reported by executing each task four times. All experiments were performed using the PyTorch implementation, and the training was terminated after 100 communications on a server equipped with CentOS Linux 7, 2 Intel Xeon CPU E5-2603 v3@1.60GHz CPU (12 cores) and 128GB RAM.

[0101] In order to demonstrate the efficiency of the present invention, six state-of-the-art algorithms were used as comparison algorithms. Specifically, four Dirichlet parameters (0.5, 0.1, 0.05, and 0.01) were used to set data sets with different heterogeneity, and 0.05 was the default heterogeneous configuration. Similar to the method used in previous studies, for each data set, the heterogeneity was distributed on 20 clients using the default settings to create a heterogeneous environment, the models of each algorithm were trained separately, and the performance of the learning models was evaluated. The TA value of each algorithm was recorded to determine its effectiveness, and the CRA value was used to evaluate its convergence speed. Then, the robustness was evaluated based on Cifar100 for different degrees of heterogeneity and client availability. In addition, the number of clients was expanded from 50 to 500 to explore the scalability of the algorithm. In all experiments, the number of local epochs was set to 1, the batch was set to 10, and the model used was DNN.

[0102] FedAvg, FedProx, perFedAvg, SCAFFOLD, FedALA, and MOONs are currently the best performing algorithms. MNIST, Cifar100 / 10, and FMNIST are commonly used public datasets. The training method of this application (denoted as FedAAW) is compared with the performance of the other six most advanced algorithms on Cifar100, MNIST, FMNIST, and Cifar10. The performance evaluation results of the multi-stage adaptive model training method for high-quality services in elderly care communities disclosed in the present invention are shown in Figure 1. Figure 5 The results show that the global model accuracy trained by the proposed FedAAW is always higher than that of other algorithms. In all results, FedAAW improves the accuracy to nearly 50%. Compared with the best results in the baseline, MNIST improves by 6%, FMNIST improves by 10%, and Cifar100 improves by 14%.

[0103] Table 1 Model accuracy table of the present invention under heterogeneous settings

[0104]

[0105] The data in Table 1 prove that the overall performance of the FedAAW of the present invention is significantly better than other algorithms. This is because AAW (dynamic adaptive aggregation) updates the aggregation weights in real time, effectively reducing the aggregation error at each aggregation. FedALA extracts more general information by learning from the global model during local training, and is the best performer among all algorithms. But in most cases, FedAAW and AAW are much more accurate than all other algorithms. Although FedALA has similar efficiency to AAW on Cifar10, it also converges faster than FedALA, which means that AAW can be used directly to achieve high performance in many scenarios while reducing the computational loss of participating clients. The worst is SCAFFOLD, probably because it aggregates information on the server with each local update. Therefore, correcting the information bias that already exists in the model may be a better approach. MOON adds a contrast term during local training to prevent local optimization from deviating from the ideal path. However, it still biases the local model more towards the local direction rather than the global direction, resulting in poor performance. FedProx adds a prox term to correct the local training direction, but the effect is not significant either. The FedAAW of the present invention not only ensures that the direction of aggregation of each local model is always the same as the direction of global model optimization, but also accurately extracts the general information in each local model and contributes it to the global model, thereby reducing the growth rate of aggregation error.

[0106] Table 2 Convergence speed evaluation table of the present invention reaching the specified accuracy

[0107]

[0108] In addition, by observing the number of communication rounds to achieve a specified accuracy, Table 2 demonstrates that the convergence rate of the present invention is much higher than that of multiple other algorithms. The number of communications required to achieve a test accuracy of 0.7 and 0.8 on MNIST. The number of communication rounds used by FedAvg is used as a benchmark. Compared with other state-of-the-art algorithms, SCAFFOLD is the most efficient in achieving the required 0.7 accuracy because the control variables can correct the direction of system convergence. However, when higher accuracy is required, the error accumulation on the server side requires more rounds of communication. Regardless of the accuracy requirement, the present invention FedAAW requires fewer communication rounds than all algorithms, which is an exciting discovery. This result shows that FedAAW converges faster than other algorithms. The results show that at accuracy = 0.7, the convergence speed is 2.75 times faster than FedAvg, and at accuracy = 0.8, the convergence speed is 3.9 times faster, almost three times that of FedALA. This advantage will play an important role in resource-constrained environments.

[0109] Figure 6 It is demonstrated that the aggregation error growth rate of FedAAW is much lower than that of other algorithms. The results show that the aggregation error of all algorithms has a clear increasing trend, which is mainly affected by the heterogeneity of data distribution in the learning environment. Although the dynamic increase of aggregation error affects all algorithms, FedAAW still shows a relatively low level of aggregation error overall. In addition, the aggregation error curve of FedAAW shows a smaller slope, which means that the aggregation error grows slower. This is because FedAAW uses more fine-grained weight adjustment during the aggregation process to ensure that the contribution of each client model is balanced and merged into the global model more smoothly. This fine weighting can reduce the inconsistency between models and reduce the volatility of the global model in each round of aggregation, resulting in a slower growth rate of the aggregation error. The manifestation of this performance advantage shows that FedAAW maintains its efficiency when handling learning tasks in heterogeneous data environments. These observations highlight the effectiveness of FedAAW in handling heterogeneous data environments. Therefore, in dynamic learning scenarios, FedAAW shows relatively low levels of aggregation error and a more stable growth trend, making it a compelling choice for handling learning tasks involving diverse data. In the FL environment where heterogeneous data is constantly changing, the aggregation operation plays a vital role in the global model performance, which makes the setting of aggregation weights a key challenge. AAW in FedAAW, a multi-stage adaptive model training method for high-quality services in elderly care communities, can be applied alone to solve this problem in different scenarios.

[0110] This example uses the Dirichlet distribution with a parameter of 0.1 to expand the number of clients to 100 to create a heterogeneous environment for different data sets. The client availability is set to 10% to evaluate the performance of the algorithm under low client availability. Secondly, this example generates heterogeneous environments with different degrees of heterogeneity based on Cifar100, and uses the parameter set (0.05, 0.1, 0.5, 1) to measure the model performance under different degrees of heterogeneity. The results in Table 3 prove that the FedAAW of the present invention adapts well to different degrees of heterogeneity and maintains its performance advantage under low client availability. It shows higher accuracy and higher robustness in all baseline results of all experiments. With the increase of heterogeneity, the effectiveness of FedAAW is significantly improved. High data heterogeneity means that there is more significant variance in local data. And FedAAW retains more information provided by the variance during training to reduce the growth rate of aggregation errors. This phenomenon illustrates the complex relationship between data heterogeneity and model performance. Therefore, it is not enough to simply improve the heterogeneous distribution state of the data, but it is also necessary to utilize the heterogeneous characteristics of the data.

[0111] Table 3 Robustness evaluation table of the present invention

[0112]

[0113] Table 4 Scalability evaluation table of the present invention

[0114]

[0115] Table 4 demonstrates that FedAAW’s performance always maintains a nearly 10% advantage over other algorithms regardless of the scale. As the number of clients increases, the accuracy of the model does not increase or decrease consistently but increases first and then decreases. This phenomenon is due to performance changes caused by differences in data distribution between clients of different sizes. When the client scale is large, the data differences between clients become more obvious, and this increase in heterogeneity limits the performance of the model. If the client scale is small and the number of clients increases moderately, the increase in heterogeneity is not obvious. At this time, increasing the number of clients can better capture specific alternative features.

[0116] Figure 7Each arc in represents a loss value. The arc corresponding to the circle is centered at (0, 0) and the loss value is the radius. The position of the arc from right to left indicates the convergence order of the model loss. The red arc represents the trend of the loss function change of FedAAW training, and the black arc represents the trend of FedALA change. The results show that there is less overlapping area between the two algorithms, and the results show that AAW allows training to approach the optimal point of the loss function faster. The convergence position of the red curve is also much lower than that of the black curve, indicating that AAW expands the range of loss optimization paths and improves the performance of the model. AAW corrects the aggregation direction by changing the weights to extract general information to move the global model in a better direction, ensuring that the model can find a better area in the parameter space.

[0117] FedAAW has achieved remarkable results in FL. Its AAW dynamically adjusts weights in the process of aggregating model parameters, successfully reducing the growth rate of aggregation error and improving the stability, convergence speed and generalization ability of the model. It is not only efficient, but also robust and scalable in various data sets and environments, and can provide effective solutions for high-precision tasks in various application scenarios.

[0118] The present invention not only dynamically adjusts the initialization of the local model in the local model training stage. It also dynamically adjusts the update direction and update step of the aggregation weight in a more fine-grained manner in the dynamic aggregation stage, eliminating the influence of data heterogeneity from multiple stages. The present invention shows unique advantages in the dynamic aggregation stage. When facing complex application scenarios such as retirement communities, its advantages are particularly obvious. There are many devices in retirement communities and the data differences are large. At the same time, the devices are prone to exit or enter for various reasons, which leads to changes in data heterogeneity during training, resulting in poor model training effects. However, the present invention can cope with the complexity and uncertainty of such data well by dynamically adjusting the update direction and update step of the aggregation weight based on the constrained dynamic learning rate and global gradient information in the dynamic aggregation stage. Specifically, it can adjust the weights in time according to the changes in the data of the participating devices in each round of changes, so that the model can still maintain stable and efficient training when facing frequent changes in equipment and changes in data heterogeneity. For example, when some devices exit, resulting in the loss or change of some data, the present invention can automatically adjust the weight distribution, thereby effectively solving the problem of low model training effect caused by problems such as device loss of connection, ensuring the effective integration of various types of equipment in the retirement community and the accurate use of data, and providing strong support for improving the intelligent management level of the retirement community.

[0119] The optimization direction is gradually changed throughout the training, including local training and global aggregation, to reduce the inefficiency of the global model caused by data heterogeneity. Considering that the global and local model information is dynamic in each round, this patent regenerates the initial model based on the current data at the beginning of each local training. In the dynamic aggregation stage, by sharing the optimization target with the model training, the aggregation weights of each round are optimized based on the gradient information, and the update direction of the aggregation weights is calculated using the gradient information from other clients, maintaining the consistency between the weight update and the global optimization. This method not only supports the federated learning distributed training model to protect data privacy and security, but also guarantees the performance of the final global model. In addition, its training process also allows the client to enter or exit the training task at will, which improves the freedom of model training.

[0120] The present invention first calculates the difference terms of the local model based on the global model and the local model of the previous round, then uses a dynamic weighted method to extract the local model difference terms to correct the local model, and uses the corrected local model as the starting model of a new round for local update. The updated parameters and gradient information are transmitted to the server. The server first uses the aggregation weights and local model aggregation to generate a new round of global models, and then constrains and optimizes the aggregation weights according to the gradient information and the local model, and continuously iterates the above local training and dynamic aggregation process until the training is completed. The global model on the server side can provide intelligent services for different applications.

[0121] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A multi-stage adaptive model training method for high-quality services in elderly care communities, characterized in that: A smart device randomly selected in the retirement community is used as a server, and other smart devices are clients. All smart devices have computing and storage capabilities, and data transmission and communication can be performed between devices. Step 1: Local training: Based on the global model and the local model of the previous round, the difference item of the local model is calculated, and then the local model difference item is extracted using a dynamic weighting method to correct the local model. The corrected local model is used as the starting model for a new round of local training. After the model is locally updated, a new round of local model is obtained. Before executing a new round of local training, the smart device of each client role will reinitialize its local model according to formula (3): Among them, all elements in the first |Wj|-1 layers are 1, and the top layer uses β j,t express; is the starting model of the tth round of local model training; W j,t-1 is the local model of client j after the (t-1) round of local training is completed; W t-1 is the global model after the (t-1)th round of aggregation; W t-1 -W j,t-1 is the local model difference term; |W j | is the number of layers of the local model; Only β j,t Before initializing a new round of local models, use the previous round of global model W t-1 The gradient information of local data is updated according to formula (4): Among them, η β is a constant learning rate, generally set to 0.001; W t-1 is the global model after round t-1 aggregation; D j is the data sampled from the jth smart device in the elderly care community, j = 1, 2, ..., M, M is the number of smart devices acting as clients in the elderly care community; F j represents the loss function on client j; β j,t-1 is the top layer of round t-1; Step 2: Dynamic aggregation: Updated local model parameters W for round t j,t and gradient information is sent to the server; Gradient descent is used to synchronously calculate the weight learning paths of different clients in the current round and update their respective aggregation weights, ensuring that the updated aggregation weight is the optimal aggregation weight that can reduce the current global model loss the fastest. The update formula of the aggregation weight is formula (7): Among them, αj,t is the aggregate weight corresponding to client j in round t; αj,t +1 is the aggregate weight corresponding to client j in round t+1; η j，t is the dynamic learning rate of the weight update of client j in round t, which is a tensor with the same structure as the local model parameters and is obtained according to formula (8), where γ is a constant; or j,t ⊙W j，t =γ (8) So far, a round of adaptive model training has been completed. The server first uses the aggregation weights and local model aggregation to generate a new round of global model W t , and then according to the gradient information and the local model W j,t Constrained optimization of the aggregation weights is performed by continuously iterating the local training and dynamic aggregation process until the training is completed; At this time, the global model on the server side provides intelligent services for the high-quality services of the retirement community.

2. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, the following steps are implemented: There are M+1 smart devices, one of which is a server and the others are clients. All smart devices have computing and storage capabilities, and data transmission and communication can be performed between devices. Based on federated learning, local training and dynamic aggregation training are carried out; The server loads the global model, and the client loads the local model; Step 1: Local training: Based on the global model and the local model of the previous round, the difference item of the local model is calculated, and then the local model difference item is extracted using a dynamic weighting method to correct the local model. The corrected local model is used as the starting model for a new round of local training. After the model is locally updated, a new round of local model is obtained. Before executing a new round of local training, the smart device of each client role will reinitialize its local model according to formula (3): Among them, the former | W j |-All elements of the 1st layer are 1, and the top layer uses β j,t express; is the starting model of the tth round of local model training; W j,t-1 is the local model of client j after the (t-1) round of local training is completed; W t-1 is the global model after the (t-1)th round of aggregation; W t-1 -W j,t-1 is the local model difference term; |W j | is the number of layers of the local model; Only β j,t Before initializing a new round of local models, use the previous round of global model W t-1 The gradient information of local data is updated according to formula (4): Among them, η β is a constant learning rate, generally set to 0.001; W t-1 is the global model after round t-1 aggregation; D j is the data sampled from the jth smart device in the elderly care community, j = 1, 2, ..., M, M is the number of smart devices acting as clients in the elderly care community; F j represents the loss function on client j; β j,t-1 is the top layer of round t-1; Step 2: Dynamic aggregation: Updated local model parameters W for round t j,t and gradient information is sent to the server; Gradient descent is used to synchronously calculate the weight learning paths of different clients in the current round and update their respective aggregate weights. The update formula of the aggregate weight is formula (7): Among them, α j,t is the aggregate weight corresponding to client j in round t; α j,t+1 is the aggregate weight corresponding to client j in round t+1; η j，t is the dynamic learning rate of the weight update of client j in round t, which is a tensor with the same structure as the local model parameters and is obtained according to formula (8); or j,t ☉W j，t =γ (8) The server first uses the aggregation weights and local model aggregation to generate a new round of global model W t , and then according to the gradient information and the local model W j,t Constrained optimization of the aggregation weights is performed by continuously iterating the local training and dynamic aggregation process until the training is completed; When generating a new global model, the average gradient of each round of training is used to correct the aggregation direction of each local model. Then, it is used as a regularization term ΔW to constrain the aggregation behavior, as shown in formula (9): Among them, S is the number of clients that can participate in training in each round; ΔW represents the regularization term, is the loss function F on client i i Find the partial derivative of the global model for client i in round t; γ is a constant; This completes the adaptive model training.

Images