Personalized federal learning method and system oriented to meta universe

Abstract

The invention discloses a personalized federal learning method and system for element universe, and the method carries out the strict classification and finer processing of element users in the gradient iteration calculation of gradient rarefaction. Therefore, the users can be classified and selected, some potential low-quality users can be excluded, and all users can be ensured to participate in federal learning as much as possible; according to the method, a personalized federated learning algorithm is adopted, knowledge transfer learned to related meta-space users is considered, a federated learning difference function is used for calculating a difference value between model parameters, and therefore cooperation between meta-space users with small data heterogeneity is enhanced in a self-adaptive mode; according to the method, updating strategies of private data and public data are designed, the latest updated data participate in the next iteration, and the training data of the model are optimized step by step by updating the data, so that the model precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning, and in particular, relates to a personalized federated learning method and system for a metaverse. Background Art

[0002] As the next generation of digital frontier technology, virtual worlds provide immersive experiences for virtual world users through virtual avatars. As a paradigm of distributed learning, federated learning (FL) promotes the collaborative training of global model parameters among metaspace users, thereby improving the metaspace experience. The current federated learning algorithms for the metaverse are flawed and cannot incorporate the characteristics of the metaverse into federated learning. For example, when an avatar gains experience in the metaverse, the corresponding user will obtain experience data, and the metaverse will also update the experience data of some metaverse users as public data. However, existing federated learning methods do not reflect the characteristics of dynamic updates of data in metadata well. Summary of the invention

[0003] In order to solve the problems existing in the prior art, the present invention proposes a personalized federated learning method and system for the metaverse.

[0004] The technical solution of the present invention is as follows:

[0005] A personalized federated learning method for a metaverse, wherein the metaverse has a plurality of meta-users, including:

[0006] Constructing a metaverse network consisting of a meta-server and a plurality of clients, wherein the meta-server is deployed with a global model, and the clients are connected to the meta-server and deploy a local model and correspond to the meta-users;

[0007] Based on the initial private data of each client, the local model is trained individually to obtain the local model parameters of each local model;

[0008] Constructing a knowledge coefficient matrix reflecting the similarity between different clients, and then obtaining the collaborative knowledge of each client based on the initial public data and public loss function of the meta server, thereby obtaining the personalized loss function of each client and the local gradient based on the knowledge coefficient matrix and the collaborative knowledge, and updating the local model parameters of the local model using a sparse distillation method for the local gradient and the collaborative knowledge;

[0009] The meta-users are screened in consideration of their online communication capabilities, and the clients corresponding to the screened meta-users upload the acquired local gradients to the meta-server. The meta-server aggregates all uploaded local gradients to form a global gradient of the global model and the knowledge coefficient matrix, and then distributes the formed global gradient and knowledge coefficient matrix to each client;

[0010] Perform equal-dimensional updating of the private data of each client and the public data of the meta-server in consideration of the dynamic characteristics of the meta-universe data, and obtain new private data of each client and new public data of the meta-server;

[0011] Iterate from new private data and new public data to form a new global gradient and knowledge coefficient matrix and distribute it to each client until the iteration termination condition is reached.

[0012] Furthermore, the specific method of performing personalized training on the local model based on the initial private data of each client to obtain the local model parameters of each local model includes:

[0013] The local model is trained individually using the stochastic gradient descent method. The individualized training is performed using the following formula:

[0014]

[0015] In the formula, is the local model parameter of meta-user u in the tth iteration; is the loss function of meta-user u in the tth iteration; is a real number space of dimension d; is the private data of meta-user u in the tth iteration; is the i-th data in the private data of meta-user u in the t-th iteration; is the label of the i-th data in the private data of meta-user u in the t-th iteration; i is the index of the data.

[0016] Furthermore, the knowledge coefficient matrix reflecting the similarity between different clients The expression is:

[0017]

[0018] In the formula, c u′u is the similarity between any two clients u′ and client u;

[0019] The expression of the client's collaborative knowledge is:

[0020]

[0021] In the formula, is the collaborative knowledge of meta-user u; l pub is the public loss function;

[0022] The expression of the personalized loss function of the client is:

[0023]

[0024] In the formula, is the personalized loss function of meta-user u at the tth iteration;

[0025] λ is a hyperparameter;

[0026] KL(·,·) is the KL divergence;

[0027] is the set of meta-users that are screened in the tth iteration; is the collaborative knowledge of meta-user u'.

[0028] Furthermore, the specific steps of updating the local model parameters of the local model by using the sparse distillation method for the local gradient and the collaborative knowledge include:

[0029]

[0030] In the formula, is the local model parameter of meta-user u in the t+1th iteration; η 2 is the second learning rate; is the gradient sparsification; is the set of online meta-users in the tth iteration; is the local gradient of meta-user u on the 1st dimension, 2nd dimension, ..., dth dimension in the tth iteration.

[0031] Furthermore, the specific method of screening meta-users considering the online communication capabilities of users includes:

[0032] Obtain the binary indicator related to whether the meta-user is online and the potential value of the meta-user's participation in communication. The calculation formulas are as follows:

[0033]

[0034] In the formula, is a binary indicator to determine whether the meta-user u is online in the tth iteration; is the potential value of the meta-user u’s participation in communication in the tth iteration; h us p us is the communication capability value of meta-user u; is the number of times meta-user u has participated in federated learning in the tth iteration;

[0035] Based on the binary indicator, a primary screening is performed to determine whether the meta-user is online, and then a secondary screening is performed on the online meta-users screened out initially based on their potential values ​​for participating in communication. The secondary screening is performed by screening out meta-users whose potential values ​​for participating in communication are greater than or equal to a first set threshold, or by screening out meta-users whose potential values ​​for participating in communication are the largest before a second set threshold.

[0036] Furthermore, the client corresponding to the selected meta-user uploads a local gradient to the meta-server, the meta-server aggregates all uploaded local gradients and updates the global gradient of the global model and the knowledge coefficient matrix, and then distributes the global gradient and the knowledge coefficient matrix to each client. The specific steps include:

[0037] The global gradient update formula of the global model is as follows:

[0038]

[0039] in,

[0040] In the formula, is the local gradient of meta-user u in the tth iteration; g t is the global gradient of the tth iteration;

[0041] The update formula of the knowledge coefficient matrix is ​​as follows:

[0042]

[0043] In the formula, and is the knowledge coefficient matrix that only contains the meta-users participating in federated learning in the tth iteration; is the set of users participating in federated learning; η 3 is the third learning rate.

[0044] Furthermore, the specific steps of performing equal-dimensional complementary updating on the private data of each client and the public data of the meta-server in consideration of the dynamic characteristics of the meta-universe data to obtain new private data of each client and new public data of the meta-server include:

[0045] The current iteration number t is judged. If t<τ, the private data of each client and the public data of the meta server are updated using the following formula:

[0046]

[0047] In the formula, is the private data of meta-user u in the t+1th iteration; is the private data of meta-user u in the tth iteration; The newly added private data of meta-user u in the t+1th iteration; is the public data of the t+1th iteration; is the public data of the tth iteration; Newly added public data for the t+1th iteration;

[0048] If t ≥ τ, calculate the private data of meta-user u in the tth iteration The personalized loss of each data in the filter is used to select the largest forward data, and recorded as the private data deleted by meta-user u in the tth iteration Calculate the deleted private data of meta-user u in the tth iteration The personalized loss of each data in the filter is used to select the largest forward data, and recorded as the deleted public data in the tth iteration

[0049]

[0050] A personalized federated learning system for a metaverse, wherein the metaverse has a plurality of meta-users, including a grid construction module, a personalized training module, a local update module, a global update module, a data update module, and an iteration module;

[0051] The grid construction module is used to construct a metaverse network consisting of a meta server and a plurality of clients, wherein the meta server is deployed with a global model, and the clients are connected to the meta server and deployed with a local model and correspond to the meta users;

[0052] The personalized training module is used to perform personalized training on the local model based on the initial private data of each client to obtain local model parameters of each local model;

[0053] The local update module is used to construct a knowledge coefficient matrix reflecting the similarity between different clients, and then obtain the collaborative knowledge of each client based on the initial public data and public loss function of the meta server, thereby obtaining the personalized loss function of each client and the local gradient based on the knowledge coefficient matrix and the collaborative knowledge, and updating the local model parameters of the local model using a sparse distillation method for the local gradient and the collaborative knowledge;

[0054] The global update module is used to screen meta-users in consideration of their online communication capabilities. The clients corresponding to the screened meta-users upload the acquired local gradients to the meta-server. The meta-server aggregates all uploaded local gradients to form the global gradient of the global model and the knowledge coefficient matrix, and then distributes the formed global gradient and knowledge coefficient matrix to each client.

[0055] The data update module is used to perform equal-dimensional complementary updates on the private data of each client and the public data of the meta-server in consideration of the dynamic characteristics of the meta-universe data, so as to obtain new private data of each client and new public data of the meta-server;

[0056] The iteration module is used to perform iterations from new private data and new public data to form a new global gradient and knowledge coefficient matrix and distribute it to each client until an iteration termination condition is reached.

[0057] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is used to call and run the computer program stored in the memory to execute any of the methods described above.

[0058] A computer-readable storage medium stores a computer program, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] The present invention proposes a personalized federated learning method and system for the metaverse, which strictly classifies and more carefully processes meta-users in the gradient iterative calculation of gradient sparsification, thereby classifying and selecting users, excluding some potential low-quality users, and ensuring that all users can participate in federated learning as much as possible;

[0061] The method of the present invention adopts a personalized federated learning algorithm, considers the knowledge transfer learned from related metaspace users, and uses a federated learning difference function to calculate the difference between model parameters, thereby adaptively strengthening the collaboration between metaspace users with less data heterogeneity;

[0062] The present invention designs an update strategy for private data and public data so that the latest updated data participates in the next iteration. By updating the data, the training data of the model is optimized step by step, thereby improving the model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of a personalized federated learning method for the metaverse in an embodiment;

[0064] Figure 2 It is a framework diagram of the personalized federated learning method for the metaverse in the embodiment;

[0065] FIG3( a ) is a schematic diagram showing the accuracy of different models on the MNIST dataset in the application example;

[0066] FIG3( b ) is a schematic diagram of the accuracy of different models on the CIFAR-10 dataset in the application example;

[0067] FIG4( a ) is a schematic diagram showing the accuracy of different models in an application embodiment on the MNIST dataset considering an offline scenario;

[0068] FIG4( b ) is a schematic diagram showing the accuracy of different models in the application example on the CIFAR-10 dataset considering an offline scenario. DETAILED DESCRIPTION

[0069] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent modifications of the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0070] Embodiment 1:

[0071] The present invention provides a personalized federated learning method for a metaverse, wherein the metaverse has a plurality of meta-users, such as Figure 1 and Figure 2 As shown, the following steps are included:

[0072] S1. Construct a metaverse network consisting of a meta-server (also called a "central server") and several clients. The meta-server is deployed with a global model. The client is connected to the meta-server and deploys a local model corresponding to the meta-user.

[0073] S2. Based on the initial private data of each client, the local model is locally trained to obtain local model parameters of each local model;

[0074] S3, construct a knowledge coefficient matrix reflecting the similarity between different clients, and then obtain the collaborative knowledge of each client based on the initial public data and public loss function of the meta server, thereby obtaining the personalized loss function of each client and the local gradient based on the knowledge coefficient matrix and the collaborative knowledge, and using the sparse distillation method to update the local model parameters of the local model for the local gradient and the collaborative knowledge;

[0075] S4, screening meta-users in consideration of their online communication capabilities, the clients corresponding to the screened meta-users upload the acquired local gradients to the meta-server, the meta-server aggregates all uploaded local gradients to form a global gradient and knowledge coefficient matrix of the global model, and then distributes the formed global gradient and knowledge coefficient matrix to each client;

[0076] S5. Perform equal-dimensional updating on the private data of each client and the public data of the meta-server in consideration of the dynamic characteristics of the meta-universe data, and obtain new private data of each client and new public data of the meta-server;

[0077] S6, iterate from the new private data and the new public data to form a new global gradient and knowledge coefficient matrix and distribute it to each client, that is, return to S2 with the new private data and the new public data and repeat the iteration of S2 to S5 until the iteration termination condition is reached.

[0078] Embodiment 2:

[0079] This embodiment is further designed on the basis of the first embodiment in that, in this embodiment, based on the initial private data of each client, the local model is trained locally in a personalized manner, and the specific method for obtaining the local model parameters of each local model includes:

[0080] The stochastic gradient descent method is used to perform personalized training on the local model. The personalized training is performed using the following formula:

[0081]

[0082] In the formula, is the local model parameter of meta-user u (i.e. local model u, i.e. client u) in the tth iteration; is the loss function of meta-user u in the tth iteration; is a real number space of dimension d; is the private data of meta-user u in the tth iteration; is the i-th data in the private data of meta-user u in the t-th iteration; is the label of the i-th data in the private data of meta-user u in the t-th iteration; i is the index of the data.

[0083] Embodiment three:

[0084] This embodiment is further designed on the basis of the second embodiment in that the knowledge coefficient matrix reflecting the similarity between different clients in this embodiment is The expression is:

[0085]

[0086] In the formula, c u′u is the similarity between any two clients u′ and client u;

[0087] The expression of the client's collaborative knowledge is:

[0088]

[0089] In the formula, is the collaborative knowledge of meta-user u; l pub is the public loss function;

[0090] The expression of the client's personalized loss function is:

[0091]

[0092] In the formula, is the personalized loss function of meta-user u at the tth iteration;

[0093] λ is a hyperparameter;

[0094] KL(·,·) is the KL divergence;

[0095] is the set of meta-users that are screened in the tth iteration; is the collaborative knowledge of meta-user u'.

[0096] KL (Kullback-Leibler) divergence function, the purpose is to transfer personalized knowledge from Transfer to U of C u (·,·) is the collaborative knowledge of user u in the metaverse, involving model parameters, It is public data that conforms to the distribution of users u in the Metaverse.

[0097] Let's take the second variable of the KL divergence function as an example. It can be expressed as:

[0098]

[0099] in, It is stored in the meta server and learned through public data in each iteration. The specific update formula is as follows:

[0100]

[0101] For the calculation of KL divergence function, without loss of generality, let and satisfy and Then, the KL divergence function can be expressed as:

[0102]

[0103] In round t, the Metaverse server knows exactly which selected Metadomain users successfully submitted their gradients, so according to The partial knowledge coefficient matrix is ​​given by Therefore, global optimization becomes minimizing the objective function in round t:

[0104]

[0105] in, can be viewed as all the joint local model parameters, is the balance parameter, 1 is The identity matrix with all elements equal to 1, ||·|| 2 is the standard 2-norm. It is called the regularization term, which aims to ensure the generalization ability of the overall learning framework. No In the case of, then try to minimize the value of the KL function, so it is inferred that That is to say, c uu =1, c u′u = 0 if u′≠u. In this case, no collaboration will take place between metadatabase users.

[0106] Embodiment 4:

[0107] This embodiment is further designed on the basis of the third embodiment in that the specific steps of updating the local model parameters of the local model by using the sparse distillation method for the local gradient and the collaborative knowledge in this embodiment include:

[0108]

[0109] In the formula, is the local model parameter of meta-user u in the t+1th iteration; η 2 is the second learning rate; is the gradient sparsification; is the set of online meta-users in the tth iteration; is the local gradient of meta-user u on the 1st dimension, 2nd dimension, ..., dth dimension in the tth iteration.

[0110] Embodiment five:

[0111] This embodiment is further designed on the basis of the fourth embodiment in that the specific method of screening the meta-users considering the online communication ability of the users in this embodiment includes:

[0112] In the meta-domain setting, meta-domain users may be unstable. When U is large and exceeds the server processing limit, meta-domain users should be further selected to prevent the meta-domain server from being overloaded. In addition, due to the error rate of gradient transmission, the set of meta-users performing aggregation in FL may be smaller than expected.

[0113] Obtain the binary indicator related to whether the meta-user is online and the potential value of the meta-user's participation in communication. The calculation formulas are as follows:

[0114]

[0115] In the formula, is a binary indicator to determine whether the meta-user u is online in the tth iteration; is the potential value of the meta-user u’s participation in communication in the tth iteration; h us p us is the communication capability value of meta-user u; is the number of times meta-user u has participated in federated learning in the tth iteration;

[0116] Based on the binary indicator, a primary screening is performed to determine whether the meta-user is online, and then a secondary screening is performed on the online meta-users screened out initially based on their potential values ​​for participating in communication. The secondary screening is performed by screening out meta-users whose potential values ​​for participating in communication are greater than or equal to a first set threshold, or by screening out meta-users whose potential values ​​for participating in communication are the largest before a second set threshold.

[0117] Embodiment six:

[0118] This embodiment is further designed on the basis of the fifth embodiment in that the client corresponding to the selected meta-user in this embodiment uploads the local gradient to the meta-server, the meta-server aggregates all uploaded local gradients and updates the global gradient and knowledge coefficient matrix of the global model, and then distributes the global gradient and knowledge coefficient matrix to each client. The specific steps include:

[0119] The global gradient update formula of the global model is as follows:

[0120]

[0121] in,

[0122] In the formula, is the local gradient of meta-user u in the tth iteration; g t is the global gradient of the tth iteration;

[0123] The update formula of the knowledge coefficient matrix is ​​as follows:

[0124]

[0125] In the formula, and is the knowledge coefficient matrix that only contains the meta-users participating in federated learning in the tth iteration; is the set of users participating in federated learning; η 3 is the third learning rate.

[0126] Assume w t is the global model parameter of the meta-inverse in event t, L t (w t ) is the global loss of the t-th global communication, which can be expressed as:

[0127]

[0128] Therefore, the FL of the metaverse in the tth global communication can be obtained by obtaining the optimal global model parameter w t，* , to formalize, to solve the following optimization problem

[0129]

[0130] By solving the above equation, Metaverse users can submit local gradients instead of local model parameters. In this way, the Metaverse server aggregates local gradients. Get the global gradient Then the global gradient Assigned to each metaverse user. Specifically, the local gradient updated by user u in the tth global communication can be expressed as

[0131]

[0132] Global gradient after aggregation Expressed as

[0133]

[0134] The update formula of the knowledge matrix can be described as:

[0135]

[0136] where η 3 is the third learning rate, and note that gradient sparsification is only performed in local model parameter updates and not for knowledge coefficient matrix updates.

[0137] Embodiment seven:

[0138] This embodiment is further designed on the basis of the first embodiment in that in this embodiment, the private data of each client and the public data of the meta-server are updated in equal dimensions taking into account the dynamic characteristics of the meta-universe data, and the specific steps of obtaining the new private data of each client and the new public data of the meta-server include:

[0139] The current number of iterations t is judged. If t < τ, the following formula is used to update the private data of each client and the public data of the meta server:

[0140]

[0141] In the formula, is the private data of meta-user u in the t+1th iteration; is the private data of meta-user u in the tth iteration; The newly added private data of meta-user u in the t+1th iteration, which can be obtained through public data; is the public data of the t+1th iteration; is the public data of the tth iteration; The newly added public data of the t+1th iteration may be generated based on the information uploaded by the client;

[0142] If t ≥ τ, calculate the private data of meta-user u in the tth iteration The personalized loss of each data in the filter is used to select the largest forward data, and recorded as the private data deleted by meta-user u in the tth iteration Calculate the deleted private data of meta-user u in the tth iteration The personalized loss of each data in the filter is used to select the largest forward data, and recorded as the deleted public data in the tth iteration

[0143]

[0144] The following table gives the pseudo code for equal-dimensional recursive update:

[0145]

[0146]

[0147] Embodiment eight:

[0148] The invention provides a personalized federated learning system for a metaverse, wherein the metaverse has a plurality of meta-users, including a grid construction module, a personalized training module, a local update module, a global update module, a data update module and an iteration module;

[0149] A grid construction module is used to construct a metaverse network consisting of a meta-server and several clients. The meta-server is deployed with a global model. The clients are connected to the meta-server and deploy local models corresponding to meta-users.

[0150] A personalized training module, used to perform personalized training on the local model based on the initial private data of each client, and obtain local model parameters of each local model;

[0151] A local update module is used to construct a knowledge coefficient matrix reflecting the similarity between different clients, and then obtain the collaborative knowledge of each client based on the initial public data and public loss function of the meta server, thereby obtaining the personalized loss function of each client and the local gradient based on the knowledge coefficient matrix and the collaborative knowledge, and using the sparse distillation method to update the local model parameters of the local model for the local gradient and the collaborative knowledge;

[0152] The global update module is used to screen meta-users considering their online communication capabilities. The clients corresponding to the screened meta-users upload the local gradients obtained to the meta-server. The meta-server aggregates all uploaded local gradients to form the global gradient and knowledge coefficient matrix of the global model, and then distributes the formed global gradient and knowledge coefficient matrix to each client.

[0153] A data update module is used to perform equal-dimensional updating of the private data of each client and the public data of the meta-server in consideration of the dynamic characteristics of the meta-universe data, so as to obtain new private data of each client and new public data of the meta-server;

[0154] The iteration module is used to perform iterations from new private data and new public data to form a new global gradient and knowledge coefficient matrix and distribute them to each client until the iteration termination condition is reached.

[0155] Embodiment nine:

[0156] An electronic device of the present invention includes a memory and a processor. The memory stores a computer program. The processor is used to call and run the computer program stored in the memory to execute the method of any of the above embodiments.

[0157] A computer-readable storage medium of the present invention stores a computer program, and when the computer program is executed by a processor, the steps of any of the above embodiments are implemented.

[0158] Application examples:

[0159] This example implements the personalized federated learning method of the present invention on two image datasets to verify the effectiveness of the method of the present invention. The two image datasets are the MNIST dataset and the CIFAR-10 dataset. The two datasets are divided into 10 categories. The MNIST dataset consists of 70,000 images, of which 60,000 are used for training and 10,000 are used for testing, and the resolution of each image is 28×28 pixels. The CIFAR-10 dataset also contains 70,000 images, of which 50,000 are used for training and 10,000 are used for testing, and the resolution of each image is 32×32 pixels. The effectiveness of the method of the present invention depends on the average accuracy of all clients in the framework.

[0160] In the meta environment of this example, all clients and meta servers share the same model. In order to simulate the lightweight federated learning process, the well-known lightweight model ShuffleNetV2 is used as the training model. To verify the effectiveness of the proposed algorithm, it is compared with classic algorithms and state-of-the-art algorithms in federated learning, including Fedavg, KT-pFL, HFED, and EFLM. All experiments in this example are implemented on a single 3080Ti GPU using PyTorch 2.5.1.

[0161] Experiment 1, for a fixed number of clients:

[0162] First, assume that there are 10 clients participating in the training, and all algorithms go through 50 rounds of global communication, with 10 local training iterations before each global communication. In order to simulate the heterogeneity of the data, each client holds only five randomly selected data categories. For algorithms designed specifically for the Metaverse, each category starts with 1,000 samples. After each round of global communication, 200 new samples are added until each category reaches 3,000 samples. After that, the amount of data will remain fixed and will be selected based on quality. Since the dataset used in this example is public, it is assumed that the quality of similar data is consistent, so no quality filtering is performed and random sampling is used instead. For other types of algorithms, each client initially holds 3,000 samples per category.

[0163] Figure 3(a) and Figure 3(b) show the accuracy changes of different algorithms on the MNIST and CIFAR10 datasets, respectively. The x-axis represents the number of global communication rounds, and the y-axis represents the model accuracy (the average accuracy of all client models). The shaded area represents the accuracy range of all clients during the current global communication. As shown in Figure 3(a), the model of the present invention performs best on the MNIST dataset and converges in about 30 rounds of global communication. Similarly, on the CIFAR-10 dataset, the model of the present invention also achieves the highest accuracy, but the convergence speed is not as fast as the MNIST dataset. This may be because CIFAR-10 consists of three-channel color images, which increases the training cost.

[0164] Table 1 shows the final test results of different methods on different data sets, indicating that the method of the present invention achieves the best performance on both data sets.

[0165] Table 1 Comparison of final test accuracy (%) on different datasets

[0166]

[0167] Test 2: Considering offline factors:

[0168] Consider a federated learning model with offline factors. Assume that a total of 30 clients participate in federated learning at the same time. Considering the offline factors, only 10 clients are allowed to participate in each round of training, and the probability of each client participating is related to the number of participations. Due to the uncertainty of client participation, 150 global communication rounds are set in this experiment, and the amount of data updates is set to 500 for the metadata-based method.

[0169] Figure 4(a) and Figure 4(b) show the final accuracy performance of different methods on the MNIST and CIFAR-10 datasets considering offline scenarios, respectively. Each node represents the average accuracy of the clients participating in the current global communication round. The red curve represents the method of the present invention. As can be seen from Figure 3, the method of the present invention is the best on both the MNIST and CIFAR-10 datasets. In addition, the convergence speed of the method of the present invention on the MNIST dataset is faster. Table 2 shows the final accuracy of different methods in offline scenarios.

[0170] Table 2 Accuracy of different models on MNIST and CIFAR-10 datasets in offline scenarios with client sampling rate of 1 / 3 (%)

[0171]

[0172] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a technician familiar with the technical field within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A personalized federated learning method for a metaverse, wherein the metaverse has a plurality of meta-users, characterized in that: include: Constructing a metaverse network consisting of a meta-server and a plurality of clients, wherein the meta-server is deployed with a global model, and the clients are connected to the meta-server and deploy a local model and correspond to the meta-users; Based on the initial private data of each client, the local model is trained individually to obtain the local model parameters of each local model; Constructing a knowledge coefficient matrix reflecting the similarity between different clients, and then obtaining the collaborative knowledge of each client based on the initial public data and public loss function of the meta server, thereby obtaining the personalized loss function of each client and the local gradient based on the knowledge coefficient matrix and the collaborative knowledge, and updating the local model parameters of the local model using a sparse distillation method for the local gradient and the collaborative knowledge; The meta-users are screened in consideration of their online communication capabilities, and the clients corresponding to the screened meta-users upload the acquired local gradients to the meta-server. The meta-server aggregates all uploaded local gradients to form a global gradient of the global model and the knowledge coefficient matrix, and then distributes the formed global gradient and knowledge coefficient matrix to each client; Perform equal-dimensional updating of the private data of each client and the public data of the meta-server in consideration of the dynamic characteristics of the meta-universe data, and obtain new private data of each client and new public data of the meta-server; Iterate from new private data and new public data to form a new global gradient and knowledge coefficient matrix and distribute it to each client until the iteration termination condition is reached.

2. The personalized federated learning method for the metaverse according to claim 1, characterized in that: The specific method of performing personalized training on the local model based on the initial private data of each client to obtain the local model parameters of each local model includes: The local model is trained individually using the stochastic gradient descent method. The individualized training is performed using the following formula: In the formula, is the local model parameter of meta-user u in the tth iteration; is the loss function of meta-user u in the tth iteration; is a real number space of dimension d; The private data of meta-user u in the tth iteration; is the i-th data in the private data of meta-user u in the t-th iteration; is the label of the i-th data in the private data of meta-user u in the t-th iteration; i is the index of the data.

3. The personalized federated learning method for the metaverse according to claim 2, characterized in that: The knowledge coefficient matrix reflecting the similarity between different clients The expression is: In the formula, c u′u is the similarity between any two clients u′ and client u; The expression of the client's collaborative knowledge is: In the formula, is the collaborative knowledge of meta-user u; l pub is the public loss function; The expression of the personalized loss function of the client is: In the formula, is the personalized loss function of meta-user u at the tth iteration; λ is a hyperparameter; KL(·,·) is the KL divergence; is the set of meta-users that are screened in the tth iteration; is the collaborative knowledge of meta-user u'.

4. The personalized federated learning method for the metaverse according to claim 3, characterized in that: The specific steps of updating the local model parameters of the local model by using the sparse distillation method for the local gradient and the collaborative knowledge include: In the formula, is the local model parameter of meta-user u in the t+1th iteration; η2 is the second learning rate; is the gradient sparsification; is the set of online meta-users in the tth iteration; is the local gradient of meta-user u on the 1st dimension, 2nd dimension, ..., dth dimension in the tth iteration.

5. The personalized federated learning method for the metaverse according to claim 4, characterized in that: The specific method of screening meta-users considering the online communication capabilities of users includes: Obtain the binary indicator related to whether the meta-user is online and the potential value of the meta-user's participation in communication. The calculation formulas are as follows: In the formula, is a binary indicator to determine whether the meta-user u is online in the tth iteration; is the potential value of the meta-user u’s participation in communication in the tth iteration; h us p us is the communication capability value of meta-user u; is the number of times meta-user u has participated in federated learning in the tth iteration; Based on the binary indicator, a primary screening is performed to determine whether the meta-user is online, and then a secondary screening is performed on the online meta-users screened out initially based on their potential values ​​for participating in communication. The secondary screening is performed by screening out meta-users whose potential values ​​for participating in communication are greater than or equal to a first set threshold, or by screening out meta-users whose potential values ​​for participating in communication are the largest before a second set threshold.

6. The personalized federated learning method for the metaverse according to claim 5, characterized in that: The specific steps of the client corresponding to the selected meta-user uploading the local gradient to the meta-server, the meta-server aggregating all uploaded local gradients and updating the global gradient of the global model and the knowledge coefficient matrix, and then distributing the global gradient and the knowledge coefficient matrix to each client include: The global gradient update formula of the global model is as follows: in, In the formula, is the local gradient of meta-user u in the tth iteration; g t is the global gradient of the tth iteration; The update formula of the knowledge coefficient matrix is ​​as follows: In the formula, and is the knowledge coefficient matrix that only contains the meta-users participating in federated learning in the tth iteration; is the set of meta-users participating in federated learning; η3 is the third learning rate.

7. The personalized federated learning method for the metaverse according to claim 1, characterized in that: The specific steps of performing equal-dimensional complementary updating on the private data of each client and the public data of the meta-server in consideration of the dynamic characteristics of the meta-universe data to obtain new private data of each client and new public data of the meta-server include: The current iteration number t is judged. If t<τ, the private data of each client and the public data of the meta server are updated using the following formula: In the formula, is the private data of meta-user u in the t+1th iteration; is the private data of meta-user u in the tth iteration; The newly added private data of meta-user u in the t+1th iteration; is the public data of the t+1th iteration; is the public data of the tth iteration; Newly added public data for the t+1th iteration; If t ≥ τ, calculate the private data of meta-user u in the tth iteration The personalized loss of each data in the filter is used to select the largest forward data, and recorded as the private data deleted by meta-user u in the tth iteration Calculate the deleted private data of meta-user u in the tth iteration The personalized loss of each data in the filter is used to select the largest forward data, and recorded as the deleted public data in the tth iteration 8. A personalized federated learning system for a metaverse, wherein the metaverse has a plurality of meta-users, characterized in that: It includes a grid construction module, a personalized training module, a local update module, a global update module, a data update module and an iteration module; The grid construction module is used to construct a metaverse network consisting of a meta server and a plurality of clients, wherein the meta server is deployed with a global model, and the clients are connected to the meta server and deployed with a local model and correspond to the meta users; The personalized training module is used to perform personalized training on the local model based on the initial private data of each client to obtain local model parameters of each local model; The local update module is used to construct a knowledge coefficient matrix reflecting the similarity between different clients, and then obtain the collaborative knowledge of each client based on the initial public data and public loss function of the meta server, thereby obtaining the personalized loss function of each client and the local gradient based on the knowledge coefficient matrix and the collaborative knowledge, and updating the local model parameters of the local model using a sparse distillation method for the local gradient and the collaborative knowledge; The global update module is used to screen meta-users in consideration of their online communication capabilities. The clients corresponding to the screened meta-users upload the acquired local gradients to the meta-server. The meta-server aggregates all uploaded local gradients to form the global gradient of the global model and the knowledge coefficient matrix, and then distributes the formed global gradient and knowledge coefficient matrix to each client. The data update module is used to perform equal-dimensional complementary updates on the private data of each client and the public data of the meta-server in consideration of the dynamic characteristics of the meta-universe data, so as to obtain new private data of each client and new public data of the meta-server; The iteration module is used to perform iterations from new private data and new public data to form a new global gradient and knowledge coefficient matrix and distribute it to each client until an iteration termination condition is reached.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

Images