Data completion method and device based on lightweight federal map learning

Abstract

The invention provides a data completion method and device based on lightweight federal map learning. The method comprises the following steps: acquiring data information of a user; establishing and training a local model at a user side; the server performs aggregation based on the local parameter information of different user sides to generate a global model; aggregating the data information of the user and the neighbor information to obtain global information; on the basis of the global model and the global information, missing data are predicted and complemented by using a federal map learning algorithm; and evaluating the accuracy of data completion by using the square root error and the mean absolute error, and judging whether the square root error and the mean absolute error converge or not. Through the lightweight federal map learning method, the local model of each user side can be trained by using the local data, so that the local features of the user data can be better captured. Meanwhile, the global model of the server side can integrate local information of different user sides, so that data completion is more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of data completion, and in particular to a data completion method and device based on lightweight federated graph learning. Background Art

[0002] As one of the core technologies in the field of artificial intelligence and machine learning, federated learning is committed to solving the problem of data silos, ensuring data privacy and security, and achieving efficient and accurate machine learning model training. More precisely, federated learning is a distributed machine learning architecture that combines privacy protection and secure encryption technology. Its purpose is to allow multiple participants to participate in model training without exposing individual privacy data. Its design purpose is to promote collaborative learning among multiple participants (computing nodes) to achieve a better global model while ensuring data privacy and legal compliance.

[0003] Among the diverse data sources owned by different users, there may be significant differences in data format and quality, which will affect the effectiveness of federated learning, resulting in poor model training results and reduced model prediction accuracy. Therefore, dealing with differences in data format, data quality, and graph structure among different user-side data owners has become a major challenge. Summary of the invention

[0004] The present invention aims to at least solve the technical problem of low prediction accuracy of the model in the prior art, and particularly innovatively proposes a data completion method and device based on lightweight federated graph learning.

[0005] In order to achieve the above-mentioned object of the present invention, the present invention provides a data completion method based on lightweight federated graph learning, the method comprising:

[0006] S1. Obtain data information between users and projects;

[0007] S2, initializing the parameters of the initial model at the user end and establishing a local model;

[0008] S3, inputting the data information into the local model, training the local model, obtaining local parameter information of the local model, and sending the local parameter information to the server;

[0009] S4, the server aggregates the local parameter information of different user terminals to generate a global model;

[0010] S5. The server uses an attention mechanism to obtain k most relevant neighbor users of the user based on the user's data information, obtains k neighbor information based on the k neighbor users, distributes the user's data information and neighbor information to the corresponding user terminal, and uses the user terminal to aggregate the user's data information and neighbor information to generate global information;

[0011] S6. The server predicts and completes the missing data using a federated graph learning algorithm based on the global model and the global information uploaded by the user.

[0012] S7. Evaluate the accuracy of data completion using the square root error and the mean absolute error to determine whether the square root error and the mean absolute error have converged;

[0013] If yes, then stop updating the global model;

[0014] If not, repeat steps S3 to S7, re-establish the local model based on the k neighbor information and existing data information obtained by the user end, upload the local model parameters to the server for re-aggregation, and reconstruct the global model until the square root error and the mean absolute error converge.

[0015] As an optional embodiment of the present invention, optionally, in step S5, using the attention mechanism to obtain the k most relevant neighbor users of the user includes:

[0016] S501, linearly changing the characteristics of each node in the data information;

[0017] S502, processing the linearly changed node using a nonlinear activation function to obtain an attention score;

[0018] S503, normalizing the attention score to obtain a weight coefficient;

[0019] S504, performing weighted summation with neighboring nodes using the weight coefficient, and averaging the results of the weighted summation;

[0020] S505. Based on the average result, use torch.topk to obtain the top k neighbor users with the highest attention scores.

[0021] As an optional embodiment of the present invention, optionally, the expression for obtaining the top k neighbor users with the highest attention scores is:

[0022] h′ i =Wh i

[0023] E ij =LeakyReLU(α[h′i ||h′ j ])

[0024]

[0025] Among them, h′ i represents the i-th node feature after linear change, W represents the learnable weight matrix, and h i represents the i-th node feature, E ij represents the similarity coefficient between feature nodes i and j, LeakyReLU(·) represents the activation function, α represents the attention mechanism parameter vector, [||] represents the connection operation, and h j ′ represents the jth node feature after linear change, α ij represents the correlation between the features of the i-th node and the j-th node after normalization, softmax(·) represents the normalization function, exp(·) represents the exponential function, k represents the neighbor node k, N i represents the node domain represented by all nodes adjacent to node i, E ik represents the similarity coefficient between feature node i and its feature neighbor node k, h i ″ represents the i-th node feature after the weighted summation of single-head attention features, K represents the number of attention heads, and h i ″ k Represents the average feature of the i-th node after weighted summation of multi-head attention features.

[0026] As an optional embodiment of the present invention, optionally, in step S5, noise needs to be added while aggregating the user's data information and neighbor information, and the noise is used to protect the user's privacy.

[0027] As an optional embodiment of the present invention, optionally, the expression of the square root error in step S7 is:

[0028]

[0029] Among them, h′ i represents the RMSE square root error, n represents the number of predictions, i represents the i-th node, represents the predicted value of the i-th node, y i Represents the true value of the i-th node;

[0030] The expression of the mean absolute error is:

[0031]

[0032] Among them, MAE means mean absolute error, and || means taking the absolute value.

[0033] As an optional embodiment of the present invention, optionally, generating the global model in S4 further includes:

[0034] S401, performing parameter initialization operation on the parameters of the global model, wherein the initialization parameters include labels, the total number of samples, gradient information of items, gradient information of users, cumulative loss, number of item appearances, and number of user appearances;

[0035] S402, traversing the initialization parameters, and calculating the weighted sum of item gradients and the weighted sum of user gradients, as well as the cumulative loss of the global model and the global model gradient based on each parameter information.

[0036] As an optional embodiment of the present invention, optionally, the expression of the weighted sum of the item gradients is:

[0037]

[0038] Among them, grad_item(x) represents the gradient information of the x-th item, y represents the y-th item, items represents the set of items involved in the training in this batch, item_grad(y) represents the gradient information of the y-th item, and num y It indicates the number of times the yth item appears in the current batch, and count_item(x) indicates the number of times the xth item appears in different batches;

[0039] The expression of the weighted sum of user gradients is:

[0040]

[0041] Among them, grad_user(x) represents the gradient information of the x-th user, users represents the set of users participating in the training in this batch, user_grad(y) represents the gradient information of the y-th user, and count_user(x) represents the number of times the x-th user appears in different batches;

[0042] The expression of the cumulative loss is:

[0043]

[0044] Among them, loss represents the cumulative loss, num represents the number of items in a batch, x represents the xth batch, and loss x Represents the loss of a user in the xth batch;

[0045] The expression of the global model gradient is:

[0046]

[0047] Among them, grad_model(x) represents the model gradient of the xth layer of a batch, model_grad(x) represents the model gradient of the xth layer of a batch, and num y Represents the number of samples in the current batch y, and num represents the total number of samples in all batches.

[0048] On the other hand, the present invention further provides a data completion device based on lightweight federated graph learning, including the data completion method based on lightweight federated graph learning;

[0049] The device also includes:

[0050] Data information extraction module, used for data information between users and projects;

[0051] Parameter initialization module, used to initialize parameters of local models and global models;

[0052] A user aggregation module is used to aggregate local parameter information of different user terminals;

[0053] A user-side module, connected to the data information extraction module and the parameter initialization module, for training the local model and sending the local parameter information of the local model to the server module;

[0054] The server module is connected to the user aggregation module and the parameter initialization module, and is used to perform weighted aggregation on the parameters of the local models to generate a global model, and distribute the parameter information of the global model to each user through a distribution unit.

[0055] As another optional embodiment of the present invention, optionally, the user terminal module includes:

[0056] A storage unit, connected to the data information extraction module, and used to store the user's data information and the data information supplemented by the data supplement module;

[0057] A data completion module is connected to the storage unit and is used to update the parameter information of the local model and complete the data information of the user using the local data of the user.

[0058] As another optional embodiment of the present invention, optionally, the server module includes:

[0059] A model aggregation unit, connected to the user aggregation module, for aggregating the user's data information and the user's neighbor information to obtain global information;

[0060] A distribution unit is connected to the user aggregation module and is used to distribute the parameter information of the global model to each user.

[0061] Beneficial effects of the present invention: The present invention uses a lightweight federated graph learning method so that the local model of each user terminal can be trained using local data, thereby better capturing the local features of the user data. At the same time, the global model on the server side can integrate the local information of different user terminals, making data completion more accurate and improving data prediction accuracy. Since the local model of each user terminal is trained locally, there is no need to upload the original data to the server, thereby effectively protecting the user's privacy information. The server side only processes the aggregated parameter information, further reducing the risk of privacy leakage. The lightweight federated graph learning method reduces the computational burden of the global model through steps such as parameter initialization, user aggregation, and model aggregation, making the data completion process more efficient. At the same time, each user terminal can process the training of local models in parallel, further improving the overall computing efficiency.

[0062] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0064] Figure 1 This is a flow chart of a data completion method based on lightweight federated graph learning in Example 1 of the present invention.

[0065] Figure 2 It is a completion / prediction flow chart of a data completion method based on lightweight federated graph learning in Example 1 of the present invention.

[0066] Figure 3 It is a structural schematic diagram of a data completion device based on lightweight federated graph learning in Example 2 of the present invention.

[0067] Figure 4 It is a comparison diagram before and after the application of Example 1 of the present invention. DETAILED DESCRIPTION

[0068] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0069] Example 1

[0070] There are many stage graph learning methods at present, and each method has its own unique advantages. This embodiment adopts a lightweight design method, abandons the feature conversion and nonlinear activation in GCN, and does not consider the use of self-information for linking, thereby realizing the use of lightweight technology in the recommendation system.

[0071] Specifically, Figure 1 As shown, a data completion method based on lightweight federated graph learning includes:

[0072] S1. Obtain data information between users and projects;

[0073] It should be noted that the user's data information is only collected from the data information extraction module, and the data information is pre-processed and divided according to four specific ratios of training set: validation set: test set = 1:4:95 (2:8:90 / 3:12:25 / 4:16:80); the local model is trained by the training set, the validation set is used to adjust the model parameters, and the test set is used to evaluate the model performance. The data information extraction module can use a variety of data pre-processing techniques, such as normalization, standardization, missing value processing, etc., to ensure data quality.

[0074] S2, initializing the parameters of the initial model at the user end and establishing a local model;

[0075] It should be noted that the reason why a local model is established on the user side is that it can better protect the user's privacy information. The parameter initialization of the initial model can use random initialization or pre-trained model parameters as a starting point to accelerate the convergence of the model. The selection of the initial model can be based on a variety of machine learning algorithms, such as linear regression, support vector machine, neural network, etc., depending on the characteristics of the data and task requirements.

[0076] S3, inputting the data information into the local model, training the local model, obtaining local parameter information of the local model, and sending the local parameter information to the server;

[0077] It should be noted that the local model in the user end will be trained using the user's local data information. During the training process, the local model will continuously update its parameters to minimize the loss function. After the training is completed, the parameter information of the local model will be sent to the server module for aggregation of the global model.

[0078] S4, the server aggregates the local parameter information of different user terminals to generate a global model;

[0079] It should be noted that after receiving the local parameter information sent by each user terminal, the server will perform weighted aggregation on these parameters according to certain strategies (such as the number of users, data quality, etc.) to generate a global model. The aggregated global model parameter information will be distributed to each user through the distribution unit so that the local model of the user terminal can use this information for further training.

[0080] S5. The server uses an attention mechanism to obtain k most relevant neighbor users of the user based on the user's data information, obtains k neighbor information based on the k neighbor users, distributes the user's data information and neighbor information to the corresponding user terminal, and uses the user terminal to aggregate the user's data information and neighbor information to generate global information;

[0081] It should be noted that in step S5, the attention mechanism is used to identify neighbor users and neighbor information that are most relevant to the current user data. In this way, it can be ensured that the global model pays more attention to information that has an important impact on the prediction of the current user data during aggregation. The value of k can be adjusted according to the actual application scenario and the size of the data set to achieve the best aggregation effect. The attention mechanism used in this embodiment is a multi-head attention mechanism.

[0082] S6. The server predicts and completes the missing data using a federated graph learning algorithm based on the global model and the global information uploaded by the user.

[0083] It should be noted that in step S6, the federated graph learning algorithm is used to integrate the global model and the global information uploaded by the user to predict and complete the missing data. The algorithm constructs a graph structure in which nodes represent user data and edges represent the relationship between users, thereby capturing the complex correlation between data. Using technologies such as graph convolutional networks (GCN), the algorithm can propagate and aggregate information on the graph, thereby achieving accurate prediction and completion of missing data.

[0084] S7. Evaluate the accuracy of data completion using the square root error and the mean absolute error to determine whether the square root error and the mean absolute error have converged;

[0085] If yes, then stop updating the global model;

[0086] If not, repeat steps S3 to S7, re-establish the local model based on the k neighbor information and existing data information obtained by the user end, upload the local model parameters to the server for re-aggregation, and reconstruct the global model until the square root error and the mean absolute error converge.

[0087] It should be noted that in step S7, the square root error (RMSE) and the mean absolute error (MAE) are indicators for evaluating the accuracy of model prediction. RMSE can punish larger errors, while MAE assigns the same weight to all errors. By calculating these indicators, it can be determined whether the model's prediction of missing data reaches the expected accuracy. If both RMSE and MAE converge, that is, reach a stable low value, it means that the model is accurate enough and the reconstruction of the global model can be stopped. If these two indicators do not converge, it means that the model has not converged yet, then it is necessary to re-establish the global model, and then re-distribute the user's data information and neighbor information to the corresponding user end, the user end re-establishes the local model, and then sends the global information aggregated by the user end to the server, and the server re-constructs the global model until the performance of the global model meets the requirements. At this time, the k neighbor information is redistributed to the most relevant k neighbor users so that they can use this information to further train and optimize the local model. Repeat steps S3 to S7 until the prediction accuracy of the model reaches a satisfactory level.

[0088] In summary, this embodiment not only protects the privacy information of users, but also improves the accuracy and efficiency of data completion through a lightweight federated graph learning method. By establishing a local model and initializing parameters on the user side, and then sending the local parameter information to the server side for aggregation, a global model is generated, making the model training process more efficient. At the same time, the attention mechanism is used to obtain the most relevant neighbor users and information, further improving the prediction performance and accuracy of the model.

[0089] As an optional embodiment of the present invention, optionally, in step S5, using the attention mechanism to obtain the k most relevant neighbor users of the user includes:

[0090] S501, linearly changing the characteristics of each node in the data information;

[0091] It should be noted that in step S501, the features of each node in the data information are first linearly changed to enhance the model's ability to express features. The linear change is achieved through matrix multiplication, which can map the original features to a new feature space, thereby capturing deeper feature relationships. This process helps to improve the accuracy of the attention mechanism in identifying related neighbor users and information in subsequent steps.

[0092] S502, processing the linearly changed node using a nonlinear activation function to obtain an attention score;

[0093] It should be noted that in step S502, the node features after linear changes will be processed by a nonlinear activation function to introduce nonlinear factors so that the model can capture complex patterns and relationships in the data. Commonly used nonlinear activation functions include ReLU, Sigmoid, and Tanh. Through the processing of the activation function, each node will obtain an attention score, which reflects the importance of the node in the global information.

[0094] S503, normalizing the attention score to obtain a weight coefficient;

[0095] It should be noted that in step S503, the attention scores are normalized and converted into weight coefficients. The normalization process ensures that the sum of all weight coefficients is 1, so that these weight coefficients can be interpreted as probability distributions. In this way, the model can assign corresponding weights according to the importance of each node, ensuring that when aggregating neighbor information, more attention is paid to those nodes that have an important impact on the current user data prediction.

[0096] S504, performing weighted summation with neighboring nodes using the weight coefficient, and averaging the results of the weighted summation;

[0097] It should be noted that in step S504, by weighting and summing the weight coefficients with the neighbor nodes, an aggregation result that comprehensively considers the importance of each neighbor node can be obtained. This weighted summation method enables the model to pay more attention to those neighbor nodes that have an important impact on the current user data prediction, thereby improving the quality of aggregated information. Then, the weighted summation result is averaged to ensure that the aggregated information can smoothly reflect the contribution of each neighbor node and avoid affecting the overall aggregation effect due to the extreme values ​​of individual nodes.

[0098] S505. Based on the average result, use torch.topk to obtain the top k neighbor users with the highest attention scores.

[0099] like Figure 2 As shown, it should be noted that in step S505, based on the average result of the weighted sum, the torch.topk function is used to select the top k neighbor users with the highest attention scores. The torch.topk function can quickly find the largest k values ​​and their corresponding indexes from a set of data, which is used in this embodiment to determine the neighbor users most relevant to the current user data. In this way, it can be ensured that in subsequent steps, the model can focus on those neighbor users who are most valuable for predicting the current user data, thereby further improving the accuracy and efficiency of data completion.

[0100] As an optional embodiment of the present invention, optionally, the expression for obtaining the top k neighbor users with the highest attention scores is:

[0101] For each node feature h i Make a linear change:

[0102] h′ i =Wh i

[0103] Use non-linear activation function processing:

[0104] E ij =LeakyReLU(α[h′ i ||h′ j ])

[0105] Normalize the attention scores:

[0106]

[0107] Use the normalized attention coefficient to perform feature weighted summation on neighbor nodes, and then average the results of multi-head calculations:

[0108]

[0109] Among them, h′ i Represents the i-th node feature after linear change;

[0110] W represents the learnable weight matrix;

[0111] h i represents the i-th node feature;

[0112] E ij Represents the similarity coefficient between feature nodes i and j;

[0113] LeakyReLU(·) represents the activation function;

[0114] α represents the attention mechanism parameter vector;

[0115] [||] indicates a connection operation;

[0116] h j ′ represents the jth node feature after linear change;

[0117] α ij Represents the correlation between the features of the i-th node and the j-th node after normalization;

[0118] softmax(·) represents the normalization function;

[0119] exp(·) represents the exponential function;

[0120] k represents neighbor node k;

[0121] N i represents the node domain represented by all nodes adjacent to node i;

[0122] E ik Represents the similarity coefficient between feature node i and its feature neighbor node k;

[0123] h i ″ represents the i-th node feature after weighted summation of single-head attention features;

[0124] K represents the number of attention heads;

[0125] Represents the average feature of the i-th node after weighted summation of multi-head attention features.

[0126] It should also be noted that in general neighbor aggregation, most of the aggregation operations are performed using a fully connected network, but in reality, not all neighbors have a great influence on the current user node. Therefore, this embodiment uses the attention mechanism of the graph attention network GAT to calculate the attention coefficient between the current node and each neighbor user, and selects the most relevant top k neighbor node information from the attention coefficients of all neighbors for aggregation operations, thereby reducing the computational cost and ensuring the stability of accuracy. At the same time, in order to further improve the generalization ability of the attention mechanism, this embodiment adopts a multi-head attention mechanism, which can quickly adapt to different application scenarios.

[0127] As an optional embodiment of the present invention, optionally, in step S5, noise needs to be added while aggregating the user's data information and neighbor information, and the noise is used to protect the user's privacy.

[0128] It should be noted that the noise used in the embodiment is Laplace noise, which is an effective means of protecting user privacy. By adding Laplace noise in the data aggregation process, user data can be effectively prevented from being reversely inferred, thereby protecting the user's privacy information. Laplace noise is a random noise with a specific distribution, which is characterized by high local sensitivity and can conceal the details of the data to a certain extent while maintaining the overall distribution characteristics of the data. In practical applications, the parameters of Laplace noise can be adjusted according to the sensitivity of the data and the privacy protection requirements to achieve the best privacy protection effect.

[0129] As an optional embodiment of the present invention, optionally, the expression of the square root error in step S7 is:

[0130]

[0131] Among them, h′ i Represents RMSE square root error;

[0132] n represents the number of predictions;

[0133] i represents the i-th node;

[0134] Represents the predicted value of the i-th node;

[0135] y i Represents the true value of the i-th node;

[0136] The expression of the mean absolute error is:

[0137]

[0138] Among them, MAE means mean absolute error;

[0139] | | means taking the absolute value.

[0140] As an optional embodiment of the present invention, optionally, generating the global model in S4 further includes:

[0141] S401, performing parameter initialization operation on the parameters of the global model, wherein the initialization parameters include labels, the total number of samples, gradient information of items, gradient information of users, cumulative loss, number of item appearances, and number of user appearances;

[0142] It should be noted that the parameter initialization operation in step S401 is a basic step in building a global model, and its purpose is to provide a good starting point for the model so that it can converge to the optimal solution more quickly in the subsequent training process. The initialization parameters include but are not limited to the following: label, total number of samples, gradient information of items, gradient information of users, cumulative loss, number of item appearances, and number of user appearances.

[0143] S402, traversing the initialization parameters, and calculating the weighted sum of item gradients and the weighted sum of user gradients, as well as the cumulative loss of the global model and the global model gradient based on each parameter information.

[0144] It should be noted that the traversal and calculation of the initialization parameters in step S402 is to further optimize the performance of the global model. By traversing each parameter information, the weighted sum of item gradients and the weighted sum of user gradients can be calculated, which helps to better understand the relationship between items and users. At the same time, calculating the cumulative loss of the global model and the global model gradient can provide an important basis for subsequent model optimization.

[0145] As an optional embodiment of the present invention, optionally, the expression of the weighted sum of the item gradients is:

[0146]

[0147] Among them, grad_item(x) represents the gradient information of the x-th item;

[0148] y represents the yth item;

[0149] Items represents the set of items involved in training in this batch;

[0150] item_grad(y) represents the gradient information of the y-th item;

[0151] num y The number of times the yth item in the current batch appears;

[0152] count_item(x) indicates the number of times the xth item appears in different batches;

[0153] The expression of the weighted sum of user gradients is:

[0154]

[0155] Among them, grad_user(x) represents the gradient information of the x-th user;

[0156] users represents the set of users participating in the training in this batch;

[0157] user_grad(y) represents the gradient information of the y-th user;

[0158] count_user(x) indicates the number of times the xth user appears in different batches;

[0159] The expression of the cumulative loss is:

[0160]

[0161] Among them, loss represents the cumulative loss;

[0162] num represents the number of items in a batch;

[0163] x represents the xth batch;

[0164] loss x Indicates the loss of a user in the xth batch;

[0165] The expression of the global model gradient is:

[0166]

[0167] Among them, grad_model(x) represents the model gradient of the xth layer of a batch;

[0168] model_grad(x) represents the model gradient of the xth layer of a batch;

[0169] num y Indicates the number of samples in the current y-th batch;

[0170] num represents the total number of samples in all batches.

[0171] like Figure 4 As shown in FIG. 1 , the comparison chart of the prediction results after the data completion method based on lightweight federated graph learning in this embodiment is applied (mainly considering the two evaluation indicators of RMSE and MAE). 1 represents the federated graph learning method before application, 2 represents the method when the neighbor information is not considered when the present invention is applied (missing the user aggregation module), and 3 represents the method when the information of k neighbors is considered when the present invention is applied (applying the user aggregation module). Figure 4 It can be seen that the application of this embodiment will produce a very good effect on the prediction effect and greatly improve the accuracy of the prediction data.

[0172] It can be seen from the technical solution of this embodiment that this embodiment provides a data completion method based on lightweight federated graph learning, which is specifically used for data completion in various scenarios. It can predict and complete data information for different incomplete data, and solve the problem of missing graph data information in the real world such as modern recommendation systems.

[0173] Example 2

[0174] like Figure 3 A data completion device based on lightweight federated graph learning is shown, including the data completion method based on lightweight federated graph learning;

[0175] The device also includes:

[0176] Data information extraction module, used to obtain data information between users and projects;

[0177] It should be noted that the data information extraction module is a data acquisition module that divides the data set according to a certain ratio, extracts information stored in the corresponding array according to different information positions, extracts its topological information according to the data situation, and obtains its adjacency matrix, etc.

[0178] Parameter initialization module, used to initialize parameters of local models and global models;

[0179] It should be noted that the parameter initialization module is used to initialize the initial model parameter information, so as to speed up the data prediction during subsequent data completion; the parameter initialization module mainly includes the following parameters: embedding size embed_size (specifies the dimension of the embedding size), learning rate lr (controls the update step size of the model parameters), batch size user_batch (the number of users processed in each training / testing), laplace_lambda (using Laplace noise to protect privacy), clipping threshold clip (preventing gradient explosion), negative sampling size sample_negative (controls the number of negative samples paired with each positive sample), verification frequency valid_step (performs a verification after a specified number of trainings to ensure the performance of the evaluation model on the verification set), L2 regularization coefficient weight_decay (prevents overfitting), layers_n model layers (controls the depth of node information propagation), etc.

[0180] A user aggregation module is used to aggregate local parameter information of different user terminals;

[0181] It should be noted that the user aggregation module uses a multi-head attention mechanism to select the top-ranked neighbor information for user information aggregation based on the calculated weights, thereby enhancing the richness of node information. The parameter initialization model is used to initialize the parameters, and the attention mechanism GAT is used to calculate the corresponding weights in combination with the neighbor user data obtained from the server. After that, several neighbor information with higher weights are selected for user aggregation to further enrich the node information. Finally, these selected k neighbor information are passed to the allocation unit.

[0182] A user-side module, connected to the data information extraction module and the parameter initialization module, for training the local model and sending the local parameter information of the local model to the server module;

[0183] It should be noted that the user-side module collects user data information and trains model parameters locally to ensure that model parameters between different user terminals remain independent while protecting the privacy of data information. This module uses a parameter initialization unit to initialize model parameters. In addition, the user-side module is responsible for collecting data information and training model parameters locally to ensure the independence of model parameters and data privacy. The core components of this module include a data completion unit, which completes existing data by performing feature transformation to ensure that the data volume meets the requirements; and a storage unit for storing original data and the final data results after completion processing.

[0184] The server module is connected to the user aggregation module and the parameter initialization module, and is used to perform weighted aggregation on the parameters of the local models to generate a global model, and distribute the parameter information of the global model to each user through a distribution unit.

[0185] It should be noted that the server module is responsible for weighted aggregation of data information submitted by the user end to build a global model, and distributes the parameter information of the global model to each user end through the distribution unit. This module uses the parameter initialization module to initialize the parameters, then performs weighted aggregation, integrates the user end data to form a global model, and distributes the model parameter information to the user end through the distribution unit. The core components of the server module include the model aggregation unit, which integrates the model parameters from different user ends into a unified model parameter and stores it on the server end; and the distribution unit, which is responsible for distributing the aggregated global model and the most relevant k neighbors selected according to user characteristics and neighbor information to each node.

[0186] As another optional embodiment of the present invention, optionally, the user terminal module includes:

[0187] A storage unit, connected to the data information extraction module, and used to store the user's data information and the data information supplemented by the data supplement module;

[0188] It should be noted that the storage unit is used to store the original user data information received by the data information extraction module, as well as the final overall data result after data completion.

[0189] A data completion module is connected to the storage unit and is used to update the parameter information of the local model and complete the data information of the user using the local data of the user.

[0190] It should be noted that the data completion module switches the global model to evaluation mode based on the local data of the user end, updates the local model parameters and other information, disables training operations such as dropout, and then obtains the corresponding item embedding vector. It uses the similarity between the embedding vectors to perform a dot product between the item embedding vector and the user feature vector, and finally obtains the prediction result for completion operation.

[0191] As another optional embodiment of the present invention, optionally, the server module includes:

[0192] A model aggregation unit, connected to the user aggregation module, for aggregating the user's data information and the user's neighbor information to obtain global information;

[0193] It should be noted that the model aggregation unit aggregates model parameters from different user terminals into overall model parameter information and stores it in the server terminal.

[0194] A distribution unit is connected to the user aggregation module and is used to distribute the parameter information of the global model to each user.

[0195] It should be noted that the allocation unit splits the global model after the global model is aggregated, and allocates the split parameter information to the model parameter information belonging to the node, and transmits the user-aggregated result back to the user end.

[0196] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A data completion method based on lightweight federated graph learning, characterized in that: The method comprises: S1. Obtain data information between users and projects; S2, initializing the parameters of the initial model at the user end and establishing a local model; S3, inputting the data information into the local model, training the local model, obtaining local parameter information of the local model, and sending the local parameter information to the server; S4, the server aggregates the local parameter information of different user terminals to generate a global model; S5. The server uses an attention mechanism to obtain k most relevant neighbor users of the user based on the user's data information, obtains k neighbor information based on the k neighbor users, distributes the user's data information and neighbor information to the corresponding user terminal, and uses the user terminal to aggregate the user's data information and neighbor information to generate global information; S6. The server predicts and completes the missing data using a federated graph learning algorithm based on the global model and the global information uploaded by the user. S7. Evaluate the accuracy of data completion using the square root error and the mean absolute error to determine whether the square root error and the mean absolute error have converged; If yes, then stop updating the global model; If not, repeat steps S3 to S7, re-establish the local model based on the k neighbor information and existing data information obtained by the user end, upload the local model parameters to the server for re-aggregation, and reconstruct the global model until the square root error and the mean absolute error converge.

2. A data completion method based on lightweight federated graph learning as claimed in claim 1, characterized in that: In step S5, the attention mechanism is used to obtain the k most relevant neighbor users of the user, including: S501, linearly changing the characteristics of each node in the data information; S502, processing the linearly changed node using a nonlinear activation function to obtain an attention score; S503, normalizing the attention score to obtain a weight coefficient; S504, performing weighted summation with neighboring nodes using the weight coefficient, and averaging the results of the weighted summation; S505. Based on the average result, use torch.topk to obtain the top k neighbor users with the highest attention scores.

3. A data completion method based on lightweight federated graph learning as claimed in claim 2, characterized in that: The expression for obtaining the top k neighbor users with the highest attention scores is: h′ i =Wh i TEACHER ij =LeakyReLU(α[h i ′||h j ′]) Among them, h i ′ represents the i-th node feature after linear change, W represents the learnable weight matrix, h i represents the i-th node feature, E ij represents the similarity coefficient between feature nodes i and j, LeakyReLU(·) represents the activation function, α represents the attention mechanism parameter vector, [||] represents the connection operation, and h j ′ represents the jth node feature after linear change, α ij represents the correlation between the features of the i-th node and the j-th node after normalization, softmax(·) represents the normalization function, exp(·) represents the exponential function, k represents the neighbor node k, N i represents the node domain represented by all nodes adjacent to node i, E ik represents the similarity coefficient between feature node i and its feature neighbor node k, h i ″ represents the i-th node feature after the weighted summation of single-head attention features, K represents the number of attention heads, Represents the average feature of the i-th node after weighted summation of multi-head attention features.

4. The data completion method based on lightweight federated graph learning according to claim 1, characterized in that: In step S5, noise needs to be added while aggregating the user's data information and neighbor information, and the noise is used to protect the user's privacy.

5. The data completion method based on lightweight federated graph learning according to claim 1, characterized in that: In step S7, the expression of the square root error is: Among them, h i ′ represents the RMSE square root error, n represents the number of predictions, i represents the i-th node, represents the predicted value of the i-th node, y i Represents the true value of the i-th node; The expression of the mean absolute error is: Among them, MAE means mean absolute error, and || means taking the absolute value.

6. A data completion method based on lightweight federated graph learning as claimed in claim 1, characterized in that: Generating the global model in S4 further includes: S401, performing parameter initialization operation on the parameters of the global model, wherein the initialization parameters include labels, the total number of samples, gradient information of items, gradient information of users, cumulative loss, number of item appearances, and number of user appearances; S402, traversing the initialization parameters, and calculating the weighted sum of item gradients and the weighted sum of user gradients, as well as the cumulative loss of the global model and the global model gradient based on each parameter information.

7. A data completion method based on lightweight federated graph learning as claimed in claim 6, characterized in that: The expression of the weighted sum of the item gradients is: Among them, grad_item(x) represents the gradient information of the x-th item, y represents the y-th item, items represents the set of items involved in the training in this batch, item_grad(y) represents the gradient information of the y-th item, and num y It indicates the number of times the yth item appears in the current batch, and count_item(x) indicates the number of times the xth item appears in different batches; The expression of the weighted sum of user gradients is: Among them, grad_user(x) represents the gradient information of the x-th user, users represents the set of users participating in the training in this batch, user_grad(y) represents the gradient information of the y-th user, and count_user(x) represents the number of times the x-th user appears in different batches; The expression of the cumulative loss is: Among them, loss represents the cumulative loss, num represents the number of items in a batch, x represents the xth batch, and loss x Indicates the loss of a user in the xth batch; The expression of the global model gradient is: Among them, grad_model(x) represents the model gradient of the xth layer of a batch, model_grad(x) represents the model gradient of the xth layer of a batch, and num y Represents the number of samples in the current batch y, and num represents the total number of samples in all batches.

8. A data completion device based on lightweight federated graph learning, characterized in that: It includes the data completion method based on lightweight federated graph learning as described in any one of claims 1 to 7; The device also includes: Data information extraction module, used to obtain data information between users and projects; Parameter initialization module, used to initialize parameters of local models and global models; A user aggregation module is used to aggregate local parameter information of different user terminals; A user-side module, connected to the data information extraction module and the parameter initialization module, for training the local model and sending the local parameter information of the local model to the server module; The server module is connected to the user aggregation module and the parameter initialization module, and is used to perform weighted aggregation on the parameters of the local models to generate a global model, and distribute the parameter information of the global model to each user through a distribution unit.

9. The data completion device based on lightweight federated graph learning according to claim 8, characterized in that: The user end module comprises: A storage unit, connected to the data information extraction module, and used to store the user's data information and the data information supplemented by the data supplement module; A data completion module is connected to the storage unit and is used to update the parameter information of the local model and complete the data information of the user using the local data of the user.

10. The data completion device based on lightweight federated graph learning according to claim 8, characterized in that: The server module comprises: A model aggregation unit, connected to the user aggregation module, for aggregating the user's data information and the user's neighbor information to obtain global information; A distribution unit is connected to the user aggregation module and is used to distribute the parameter information of the global model to each user.

Images