A Model Heterogeneous Federated Recommendation Method and System with Computing Power Adaptability

Abstract

The present invention discloses a model heterogeneous federated recommendation method and system with computing power adaptability, belonging to the fields of recommendation systems and federated learning. The central server initializes the item matrix, and the user side initializes the user matrix; the central server randomly selects several user sides to participate in federated training; the user side determines the dimension of all item latent feature vectors participating in training according to its own computing power condition, randomly extracts all item latent feature vectors related to its own items in this dimension from the item matrix, and randomly extracts the user latent feature vectors of all local users in this dimension, and trains the local model according to the extracted item latent feature vectors and user latent feature vectors; updates the user matrix / item matrix by updating the gradients of the user / item latent feature vectors obtained after training. The present invention can adaptively select a model with a suitable size according to the computing power of the devices participating in training, and can achieve a balance between the recommendation effect of the model and the complexity of the model.

Description

Technical Field

[0001] The present invention relates to the fields of recommendation systems and federated learning, and particularly to a method and system for model heterogeneous federated recommendation with computing power adaptability. Background Art

[0002] A recommendation system is a machine learning-based information filtering system that predicts user preferences by learning the user's historical behavior (i.e., the historical interactions generated between the user and the items), and recommends suitable items to the user accordingly. Currently, recommendation systems have been widely applied in many fields such as e-commerce, online movie viewing, and social networks, and have brought huge benefits to many industries. However, since the recommendation system needs to use the user's historical interaction records as training data, it brings potential risks to the user's privacy.

[0003] Federated learning technology can eliminate this risk to a certain extent. In the federated learning scenario, multiple clients train the same model under the coordination of a central server. In each round of training, the central server randomly selects some clients to participate in the training and shares the current model parameters with these participating clients. Subsequently, the participating clients train the received model parameters based on their local training data and send the gradients back to the central server. Finally, the central server aggregates the gradients sent back by all participating clients and updates the model parameters according to the aggregated gradients. Throughout the entire process of federated learning, the data of each client is always stored locally and is not shared with others (including the central server). Therefore, federated learning can effectively protect the privacy data of each client while completing model training.

[0004] Combining federated learning technology with a recommendation system can effectively protect the user's privacy data while providing a recommendation service to the user. In federated recommendation, each user stores their historical interaction data with the items on their own user-side device (which may be a computer, tablet, mobile phone, or other Internet of Things device), and allows their user-side device to participate in the federated training of the recommendation model. Currently, most of the research or applications related to federated recommendation still stay in homogeneous recommendation models, that is, all user-side devices jointly train and use the same recommendation model. However, different user-side devices have different computing capabilities. If we use a simpler recommendation model for low-computing-power devices on the one hand, it will limit the expressive ability of the model, and on the other hand, if we only consider high-computing-power devices and use a more complex model, it will cause the computing time of low-computing-power devices to be too long, affecting the training and operation of the overall recommendation system.

[0005] Regarding the above-mentioned model heterogeneity problem, although there are already some studies, the scenarios of these studies are often as follows: 1) Each training participant has its own training task, and the training tasks of different participants are different; 2) For the models corresponding to these participants, some are shared by all, and some are private and designed for their own training tasks. This method requires designing different training tasks for the private parts according to the different computing powers of the training participants, and the scheme is complex and not universal. Summary of the Invention

[0006] To solve the above technical problems, the present invention proposes a computing power adaptive model heterogeneous federated recommendation method and system, which can adaptively select a model of a suitable size according to the computing power of the devices participating in the training, and can achieve a perfect balance between the model recommendation effect and the model complexity. The model heterogeneity described in the present invention refers to different sizes of the same model, which is different from the existing heterogeneity method between different models.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] An object of the present invention is to provide a computing power adaptive model heterogeneous federated recommendation method, including the following steps:

[0009] Step 1: The central server randomly initializes an item matrix composed of all project latent feature vectors, and the user side randomly initializes a user matrix composed of local user latent feature vectors related to its own projects.

[0010] Step 2: The central server randomly selects several user sides to participate in the federated training, and sends the item matrix to each selected user side.

[0011] Step 3: Each user side determines the dimension k of all project latent feature vectors participating in the training according to its own computing power condition i , and then randomly extracts k i dimensions of all project latent feature vectors related to its own projects from the item matrix sent by the central server, and randomly extracts k i dimensions of all local user latent feature vectors of local users from the local user latent feature vectors, and trains a local model according to the extracted project latent feature vectors and user latent feature vectors.

[0012] Step 4: Each user side participating in the training updates the gradient using the trained user latent feature vector, directly updates it to the user matrix maintained locally, and at the same time uploads the updated gradient of the trained project latent feature vector to the central server; the central server aggregates the updated gradients of the project latent feature vectors uploaded by all user sides participating in the training, and updates the item matrix.

[0013] Step 5: Loop through Step 2 to Step 4 until the training ends. Each client estimates the recommended degree scores of each item based on the user matrix maintained locally and the item matrix sent by the central server, and selects the top-ranked items as the recommendation results.

[0014] Further, in Step 3, randomly extract all item latent feature vectors related to its own item with dimension k i from the item matrix sent by the central server, specifically:

[0015] Each client determines the dimension k of all item latent feature vectors participating in the training according to its own computing power condition i , and randomly selects k i index values from [0, k i - 1] to form the feature filtering layer parameter ind. The feature filtering layer is expressed as follows:

[0016]

[0017] Among them, F(V) represents the item matrix V trained with the parameters extracted from the item matrix V sent by the central server ′ , k i represents the dimension of the item latent feature vectors participating in the training determined by the i-th client according to its own computing power condition, and k v represents the dimension of each item latent feature vector in the item matrix V sent by the central server. select_index(V, ind, dim = 1) represents extracting the item latent feature vectors from the column vectors of the item matrix V according to the feature filtering layer parameter ind, and dim = 1 is used to represent the column vectors with dimension k v .

[0018] Further, when looping through Step 2 to Step 5 for iterative training, randomly extract item latent feature vectors and user latent feature vectors in each round of training.

[0019] Further, the number k i of index values is the optimal value obtained according to the training cut-off time specified by the central server, specifically:

[0020] The current client generates an item matrix with dimension m × k i and a user matrix with dimension n × k i (k i < k v ) according to the training cut-off time specified by the server, performs matrix multiplication by simulating the training situation, and obtains the maximum k i value that can complete the matrix multiplication calculation within the specified training cut-off time, denoted as k i_max ; randomly select from [0, kv -1] i_max The index values ​​constitute the feature filter layer parameter ind.

[0021] Furthermore, the step 4 is specifically as follows:

[0022] On the local user side, k randomly extracted from the local user latent feature vector before training i After training, k user latent feature vectors of all local users with dimension i The user feature matrix gradient U of dimension grad , the user feature matrix gradient U grad One-to-one correspondence with the dimensions of the extracted user latent feature vectors, the user matrix is ​​optimized and updated parameters directly on the local user side;

[0023] At the same time, k randomly selected items from the project matrix sent from the central server before training are i The index value of the dimension and the training item latent feature vector update gradient V grad Upload them to the central server together, and fill the update gradient corresponding to the missing index value in the central server with 0, which is expressed as:

[0024]

[0025] The Scatter(.) function indicates that the index value distribution is restored, and the remaining blanks are filled with 0; R(V grad ) indicates that the gradient V of the project latent feature vector is updated according to the feature filter layer parameter ind grad Restore to the original dimension k v ;

[0026] The central server aggregates all the updated gradients of the latent feature vectors of the items after the padded dimensions and updates the item matrix.

[0027] Furthermore, each user terminal generates a partially ordered relation triple based on the locally stored user historical interaction data for use in each round of training process.

[0028] Furthermore, the central server uses average aggregation to aggregate the project latent feature vectors uploaded by all users participating in the training to update the gradients.

[0029] Furthermore, after obtaining the estimated recommendation scores of each item in step 5, the items that have been interacted with are removed, and then several items with high rankings are selected as recommendation results.

[0030] Another object of the present invention is to provide a computing power adaptive model heterogeneous federated recommendation system for implementing the above-mentioned model heterogeneous federated recommendation method.

[0031] The beneficial effects of the present invention are as follows:

[0032] In traditional federated recommendation systems, all participating client devices often use recommendation models with exactly the same structure. Considering the computing power differences among devices participating in federated recommendation, the present invention introduces the operation of "dimension extraction" on the basis of traditional federated recommendation, enabling each client device participating in federated recommendation training to freely select the dimension of the model to be used according to its own device computing power. Compared with traditional methods, the present invention can achieve a similar level of recommendation effect while reducing the computing power requirements of user devices, with a simple solution and strong versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 FIG. shows the overall workflow diagram of a computing power adaptive model heterogeneous federated recommendation method shown in the present invention.

[0034] Figure 2 FIG. shows the client workflow diagram of a computing power adaptive model heterogeneous federated recommendation method shown in the present invention.

[0035] Figure 3 FIG. shows the feature filtering process diagram of a computing power adaptive model heterogeneous federated recommendation method shown in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] The present invention will be further described in detail below in conjunction with the drawings and embodiments. It should be noted that the following embodiments are intended to facilitate the understanding of the present invention and do not impose any limitations on it.

[0037] In addition, the drawings are only schematic diagrams of the present invention. Some of the block diagrams shown in the drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0038] The overall workflow diagram of the present invention is as shown in Figure 1 FIG., which is applicable to the scenario of federated recommendation using the matrix factorization algorithm. First, the matrix factorization algorithm will be briefly described below, and the symbols and formulas required below will be defined and explained.

[0039] Assume that there are a total of n users and m items, and the predicted ratings of each user for each item form an n-row and m-column matrix R n×m , where the value in the i-th row and j-th column is R ijDenote the predicted score of user \(i\) for item \(j\). By sorting the predicted scores of each user for all items, items with higher score values can be recommended to the corresponding users. The matrix factorization algorithm aims to factorize the rating matrix \(R\). n×m into the product of two matrices:

[0040] R n×m = U n×k × (V m×k ) T

[0041] where \(U\) represents the user feature matrix, \(V\) represents the item feature matrix, \(k\) represents the dimension of the latent feature vectors of users and items. The score of user \(i\) for item \(j\) can be transformed into the product of the latent feature vector of user \(i\) and the latent feature vector of item \(j\), that is, \(R\). ij = U 1×k × (V 1×k ) T to obtain the corresponding item scores. Finally, substituting the ground truth and predicted values into the loss function, and using backpropagation to solve the gradients \(U\) of the user feature matrix grad and the gradients \(V\) of the item feature matrix grad , optimize the matrix eigenvalues, and gradually converge to obtain the result.

[0042] In the actual scenario of federated recommendation, there are various user-side devices participating in federated training, such as computers, tablets, mobile phones, or other Internet of Things devices, etc. The computing capabilities of different types of devices often vary greatly, which causes existing federated recommendation systems to often not be able to fully utilize their performance in order to accommodate low-computing-power devices. To address this deficiency, the present invention provides a computing power adaptive model heterogeneous federated recommendation method and system.

[0043] The present invention takes federated recommendation as the application scenario. In the federated recommendation scenario, due to the differences in the performance of each user-side device, some low-computing-power users generally have problems such as long training time and low model training degree when training the model. The present invention provides a federated recommendation method and system based on feature extraction to adapt to different device performances, which is applicable to most algorithms based on matrix factorization, and has no special requirements for the loss function and optimization method, has strong adaptability, and can improve the training efficiency and reduce the load of the user-side device while maintaining the original training effect. The federated recommendation training model based on Bayesian personalized ranking algorithm, stochastic gradient descent optimization algorithm, and additional feature extraction will be described in detail below. The same symbols are used to follow the description in the matrix factorization algorithm part above.

[0044] As Figure 2 shown, the computing power adaptive model heterogeneous federated recommendation method mainly includes the following steps:

[0045] (1) The central server maintains the project feature matrix V, and each client maintains the user feature matrix U related to its own data. The central server stipulates the average training time for one round of training according to the task requirements and sends it to all clients. At the same time, it sends the dimension k of the project latent feature vector stipulated by the central server. v . Each client determines whether it can complete the forward propagation loss calculation and backward propagation gradient calculation processes within the stipulated time according to its own computing power and the scale of model data. If it cannot be achieved, then on the basis of k v dimensions, appropriately reduce the user latent feature vector of this client to dimension k i , and randomly select k v index values from [0, k i - 1] to form the feature filtering layer parameter ind. The feature filtering layer is expressed as follows:

[0046]

[0047] It means to extract the corresponding index values from the input matrix V according to the feature filtering layer parameter ind in the column dimension and recombine them into the new input matrix F(V). The clients with higher computing power can keep the feature filtering layer as the same mapping function.

[0048] In this embodiment, after each client determines the model dimension (that is, the dimension of its own user latent feature vector) according to its own computing power condition, each project latent feature vector extracted from the original project matrix is only different from the original matrix in dimension, and the number of projects is the same. The select_index(V, ind, dim = 1) in the above formula means to extract the project latent feature vector from the column vector of the project matrix V according to the feature filtering layer parameter ind, and dim = 1 is used to represent the column vector of dimension k v , and dim = 0 is used to represent the row vector of different projects. In addition, the dimension of the user latent feature vector needs to be the same as the dimension of the extracted project latent feature vector, and different index values can be randomly extracted during each round of training.

[0049] As Figure 3 shown, for the m×k - dimensional project feature matrix V maintained by the central server, m represents the number of projects, and k represents the dimension of the project latent feature vector; for the feature filtering layer parameter ind = {1, x, y} composed of k i = 3 index values, the new input matrix F(V) is formed by extracting the first, x, and y columns of data from the original project feature matrix V through the formula F(V) = select_index(V, ind, dim = 1). In this embodiment, the value of k i is the optimal value obtained according to the training deadline stipulated by the central server. Specifically: the current client generates m×k according to the training deadline stipulated by the server - side.i The item matrix of dimensions and the n×k i user matrix (k i <k v ), perform matrix multiplication by simulating the training situation, and obtain the maximum k that can complete the matrix multiplication calculation within the specified training deadline i value, denoted as k i_max , which is used as the parameter of the current client feature filtering layer.

[0050] (2) Initialize the user feature matrix parameters at each user end and generate a partial order relation triple; initialize the item feature matrix parameters at the central server end.

[0051] (3) The central server randomly selects several user ends to participate in the training and sends the complete item feature matrix V to each selected user end. The user end inputs the received item feature matrix into the feature filtering layer to generate a new item feature matrix V ′ = F(V). Calculate the predicted value using the partial order relation triple and substitute it into the loss function to obtain the loss value loss.

[0052] (4) Use loss backpropagation to obtain the user feature matrix gradient U grad and the item feature matrix gradient V grad , U grad directly optimizes and updates the parameters of the user feature matrix at the user end, and V grad needs to be restored to the initial dimension according to the parameters in the feature filtering layer:

[0053]

[0054] The Scatter_ function represents restoring according to the distribution of the original column index values, and filling the remaining blank parts with 0. The restored item feature matrix gradient is sent back to the central server.

[0055] In this embodiment, when updating the user matrix according to the local user feature matrix gradient U grad , since the user feature matrix gradient U grad corresponds one-to-one with the dimensions of the extracted user latent feature vectors, the parameters of the user matrix can be directly optimized and updated at the local user end.

[0056] The present invention can be used for item recommendation for different types of user ends. For example, the recommended items included in user ends A and B are different, but the central server includes all the item latent feature vectors related to the items of A and B themselves. In local training, user ends A and B can extract the item latent feature vectors related to their own items from all the item latent feature vectors to participate in the training, and the gradients of the item latent feature vectors not related to their own items are 0.

[0057] (5) The central server collects and integrates the gradients of the item feature matrices uploaded by all user clients participating in the training, averages and aggregates the gradients of the recovered latent feature vectors of all items, and updates the parameters of the item feature matrix.

[0058] Repeat the above training process for several rounds until the model converges. After the model converges, projects can be recommended to a certain user based on this model.

[0059] In this embodiment, the method of recommending projects is as follows:

[0060] i. The central server sends the latent feature vectors of all items to the user client;

[0061] ii. After receiving the latent feature vectors of all items sent by the central server, this user client estimates the preference score of the user for each item based on the latent feature vectors of all items and the latent feature vector of the user maintained locally;

[0062] iii. According to the historical interaction records between the user and the items, the interacted items are excluded, several items with the highest estimated preference scores are found, and finally these items are recommended to this user.

[0063] When conducting simulation tests in the implementation of the present invention, an equal number of simulated user clients are constructed according to the number of data users. Each user client contains the data of one user, and an equal number of partial order relation triples are constructed according to the number of implicitly interacted items as simulated training data. Each round of training in federated training includes dividing all users into several batches, each batch contains 256 simulated user clients, and each user client uses all the partial order relation triples in its own simulated training data for training.

[0064] In the field of recommendation system research, the MovieLens dataset is a dataset about movie ratings and is relatively well-known. This dataset contains the rating records of users for movies. Excluding the ratings of movies by the users in this dataset, this dataset can be regarded as an implicit feedback dataset.

[0065] It is found after conducting experiments on the MovieLens data that there is almost no performance degradation in the adaptive adjustment of the item matrix compared to the original dimension, and instead, the average accuracy rate in the first round of training is higher than the original training method, rising from 6% to 8%, an increase of two percentage points. In subsequent tests, after 130 rounds of training, both methods reach convergence. The final accuracy rate of the method of the present invention is stable at about 56%, and the final accuracy rate of the original method is stable at about 57%, with a very small difference and within an acceptable range.

[0066] Corresponding to the embodiment of the above-mentioned model heterogeneous federated recommendation method with computing power self-adaptation, the present application also provides an embodiment of a model heterogeneous federated recommendation system with computing power self-adaptation, which completes tasks such as data acquisition, local training, and central server parameter update, and realizes the full automation of the recommendation system.

[0067] The model heterogeneous federated recommendation system described above includes:

[0068] A user model, which is located at each local user side and is used to update the gradient according to the uploaded item latent feature vector to maintain the local user matrix;

[0069] An item model, which is located at the central server side and is used to update the gradient according to the trained user latent feature vector to maintain the item matrix;

[0070] A data module, which is located at each user side and is used to generate a partial order relation triple based on the items interacted with by the current user and the items not interacted with;

[0071] A performance measurement module, which is used to determine the dimension k of all item latent feature vectors participating in training according to the computing power condition of the user side itself i ;

[0072] A data extraction and restoration module, which is used to randomly extract all item latent feature vectors related to its own items with dimension k from the item matrix sent by the central server, and randomly extract all user latent feature vectors of local users with dimension k from the local user latent feature vectors; and, it is used to restore the updated gradients of the user latent feature vector and the item latent feature vector obtained by training to the original dimension; i All item latent feature vectors related to its own items with dimension k are randomly extracted from the item matrix sent by the central server, and all user latent feature vectors of local users with dimension k are randomly extracted from the local user latent feature vectors; and, it is used to restore the updated gradients of the user latent feature vector and the item latent feature vector obtained by training to the original dimension; i All item latent feature vectors related to its own items with dimension k are randomly extracted from the item matrix sent by the central server, and all user latent feature vectors of local users with dimension k are randomly extracted from the local user latent feature vectors; and, it is used to restore the updated gradients of the user latent feature vector and the item latent feature vector obtained by training to the original dimension;

[0073] A training module, which is located at each user side and is used to train the local model according to the extracted item latent feature vector and user latent feature vector to obtain the updated gradient of the user latent feature vector and the updated gradient of the item latent feature vector;

[0074] In a specific implementation of the invention, the number k of index values generated by the performance measurement module i is the optimal value obtained according to the training deadline specified by the central server side. Specifically: the current client generates an m×k i matrix and an n×k i matrix (k i <k, and the values in the matrix are random) according to the training deadline specified by the server side, simulates the training situation to perform matrix multiplication, and obtains the maximum k i value that can complete the matrix multiplication calculation within the specified training deadline, denoted as k i_max and used as the parameter of the feature filtering layer of the current client;

[0075] A query module, which is used to estimate the recommended degree scores of each project according to the user matrix maintained locally and the project matrix sent by the central server, and select the top-ranked projects as the recommended results.

[0076] Regarding the system in the above embodiments, the specific ways in which each unit or module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0077] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions in the method embodiments. The system embodiments described above are only illustrative, and each module therein may or may not be physically separated. In addition, in the present invention, each functional module may be integrated in a processing unit, or each module may exist physically alone, or two or more modules may be integrated in a unit. The above integrated modules or units may be implemented in the form of hardware or in the form of software functional units, and some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present application.

[0078] The above embodiments have described in detail the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, supplements, equivalent replacements, etc. made within the scope of the principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A model heterogeneous federated recommendation method with computing power adaptability, characterized in that, it includes the following steps: Step 1: The central server randomly initializes the item matrix composed of all item latent feature vectors, and the client randomly initializes the user matrix composed of local user latent feature vectors related to its own items; Step 2: The central server randomly selects several clients to participate in federated training, and sends the item matrix to each selected client; Step 3: Each client determines the dimension k of all project latent feature vectors participating in the training according to its own computing power conditions i , and then randomly extracts k i -dimensional all project latent feature vectors related to its own projects from the project matrix sent by the central server, and randomly extracts k i -dimensional all user latent feature vectors of local users from the local user latent feature vectors, and trains the local model according to the extracted project latent feature vectors and user latent feature vectors; Randomly extract all the item latent feature vectors related to its own items with k i dimensions from the item matrix sent by the central server, specifically: Each client determines the dimension k of the latent feature vectors of all projects participating in the training according to its own computing power conditions i , and randomly selects k v indices from [0, k i - 1] to form the feature filtering layer parameter ind. The feature filtering layer is represented as follows: Among them, F(V) represents the item matrix V' trained with parameters extracted from the item matrix V sent by the central server, and k i represents the dimension participated in training determined by the i-th client according to its own computing power condition, and k v represents the dimension of each item latent feature vector in the item matrix V sent by the central server. select_index(V, ind, dim = 1) represents extracting the item latent feature vector from the column vectors of the item matrix V according to the feature filtering layer parameter ind, and dim = 1 is used to represent k v column vectors of the dimension; Step 4: Each client participating in training updates the gradient using the trained user latent feature vector, directly updates it to the user matrix maintained locally, and at the same time uploads the updated gradient of the item latent feature vector obtained from training to the central server; The central server aggregates the updated gradients of the item latent feature vectors uploaded by all clients participating in training, and updates the item matrix; Step 5: Repeat Step 2 to Step 4 until the training ends. Each client estimates the recommendation degree scores of each item based on the user matrix maintained locally and the item matrix sent by the central server, and selects the items with the top rankings as the recommendation results.

2. The model heterogeneous federated recommendation method with computing power adaptability according to claim 1, characterized in that, When iteratively training by repeating Step 2 to Step 5, the item latent feature vector and the user latent feature vector are randomly selected in each round of training.

3. The model heterogeneous federated recommendation method with computing power adaptability according to claim 1, characterized in that, The number k of index values i is the optimal value obtained according to the training deadline specified by the central server, specifically: The current client generates an item matrix of m×k i dimensions and a user matrix of n×k i (k i <k v ) according to the training deadline specified by the server, performs matrix multiplication by simulating the training situation, and obtains the maximum k i value that can complete the matrix multiplication calculation within the specified training deadline, denoted as k i_max ; randomly select k v index values from [0, k i_max -1] in the item matrix sent from the central server to form the feature filtering layer parameter ind.

4. The model heterogeneous federated recommendation method with computing power adaptability according to claim 1, characterized in that, The specific content of Step 4 is as follows: At the local client side, according to the user latent feature vectors of all local users with k i dimensions randomly extracted from the local user latent feature vectors before training, after training, a user feature matrix gradient U i with k grad dimensions is generated. The user feature matrix gradient U grad corresponds one-to-one with the dimensions of the extracted user latent feature vectors, and directly optimizes and updates the parameters of the user matrix at the local client side; Meanwhile, the index values of k i dimensions randomly sampled from the item matrix sent by the central server before training are uploaded to the central server together with the updated gradient V grad of the item latent feature vector. At the central server, the updated gradient corresponding to the missing index value is filled with 0, which is expressed as: Among them, the Scatter(.) function represents restoring according to the index value distribution, and the remaining blank parts are filled with 0; R(V grad ) represents updating the gradient V of the item latent feature vector according to the feature filtering layer parameter ind grad to the original dimension k v ; The central server aggregates the updated gradients of all item latent feature vectors after dimension complementation, and updates the item matrix.

5. The model heterogeneous federated recommendation method with computing power adaptability according to claim 1, characterized in that, Each client generates a partial order relation triple according to the user historical interaction data stored locally for each round of training process.

6. The model heterogeneous federated recommendation method with computing power adaptability according to claim 1, characterized in that, The central server aggregates the updated gradients of the item latent feature vectors uploaded by all clients participating in training by using the average aggregation method.

7. The model heterogeneous federated recommendation method with computing power adaptability according to claim 1, characterized in that, After obtaining the recommendation degree scores of each item after estimation in Step 5, the interacted items are excluded, and then several items with the top rankings are selected as the recommendation results.

8. A model heterogeneous federated recommendation system with computing power adaptability for implementing the model heterogeneous federated recommendation method described in claim 1, characterized in that, The model heterogeneous federated recommendation system includes: A user model, which is located at each local client and is used to maintain the local user matrix according to the updated gradient of the uploaded item latent feature vector; An item model, which is located at the central server side and is used to maintain the item matrix according to the updated gradient of the trained user latent feature vector; A data module, which is located on each client and is used to generate a partial order relation triple based on the items interacted by the current user and the items not interacted; A performance measurement module, which is used to determine the dimension k of all project latent feature vectors participating in training according to the computing power conditions of the user side i ; A data extraction and recovery module, which is used to randomly extract all item latent feature vectors related to its own item with k dimensions from the item matrix sent by the central server, and randomly extract user latent feature vectors of all local users with k dimensions from the local user latent feature vectors; and, used to restore the updated gradients of the user latent feature vectors and the updated gradients of the item latent feature vectors obtained by training to the original dimensions; i All item latent feature vectors related to its own item with k dimensions, and all user latent feature vectors of local users with k dimensions randomly extracted from the local user latent feature vectors; and, used to restore the updated gradients of the user latent feature vectors and the updated gradients of the item latent feature vectors obtained by training to the original dimensions; i All item latent feature vectors related to its own item with k dimensions, and all user latent feature vectors of local users with k dimensions randomly extracted from the local user latent feature vectors; and, used to restore the updated gradients of the user latent feature vectors and the updated gradients of the item latent feature vectors obtained by training to the original dimensions; Randomly extract all item latent feature vectors related to its own items with k i dimensions from the item matrix sent by the central server, specifically: Each client determines the dimension k of the latent feature vectors of all projects participating in the training according to its own computing power conditions i , randomly selects k from the project matrix sent by the central server in the range of [0, k v -1] to form the feature filtering layer parameter ind, and the feature filtering layer is represented as follows: i ​ Among them, F(V) represents the item matrix V' trained with parameters extracted from the item matrix V sent by the central server, and k i represents the dimension for participation in training determined by the i-th client according to its own computing power condition, k v represents the dimension of each item latent feature vector in the item matrix V sent by the central server. select_index(V, ind, dim = 1) represents extracting the item latent feature vector from the column vectors of the item matrix V according to the feature filtering layer parameter ind, and dim = 1 is used to represent k v column vectors of the dimension; A training module, which is located on each client and is used to train a local model according to the extracted item latent feature vectors and user latent feature vectors to obtain the user latent feature vector update gradient and the item latent feature vector update gradient; A query module, which is used to estimate the recommended degree scores of each item according to the user matrix maintained locally and the item matrix sent by the central server, and select the items with the top rankings as the recommended results.

9. The computing power adaptive model heterogeneous federated recommendation system according to claim 8, wherein, the optimal value obtained by the performance measurement module according to the training deadline specified by the central server is specifically: The current client generates an item matrix of m×k i dimensions and a user matrix of n×k i according to the training deadline specified by the server side, and performs matrix multiplication by simulating the training situation for k i <k v , and obtains the maximum k i value that can complete the matrix multiplication calculation within the specified training deadline as the optimal value.

Images