Adaptive aggregation federated recommendation system and method with structure and training phase perception

Abstract

This invention discloses a structure- and training-phase-aware adaptive aggregation federated recommendation system and method, belonging to the technical field of federated learning and recommendation systems. The client employs structure-responsive low-rank parameter updates, locally updating the item embedding matrix as a low-rank matrix product, uploading only the lightweight trainable matrix portion. The server calculates the federated gradient norm to determine key learning stages, dynamically adjusting aggregation coefficients based on the determination results, and only enabling performance-based weight adjustments during key stages to perform weighted aggregation of the low-rank matrix. This invention reduces communication overhead by over 90%; the standard deviation decreases from 0.089 to 0.063, significantly improving training stability; the recommendation performance HR@10 remains above 94.2% of the baseline method, and it is fully compatible with homomorphic encrypted secure aggregation protocols. The attached figure is a flowchart of the method of this invention.

Description

Technical Field

[0001] This invention belongs to the technical field of federated learning and recommendation systems, specifically relating to an adaptive aggregation federated recommendation system and method that is aware of the structure and training phase. Background Technology

[0002] In existing federated recommendation systems, clients frequently need to upload complete model parameters, resulting in significant communication overhead. Furthermore, the non-independent and identically distributed nature of client data causes drastic fluctuations during training, hindering convergence. While existing structure compression methods can reduce the number of uplink parameters, downlink transmission still requires a full-rank matrix and is incompatible with homomorphic encrypted secure aggregation protocols, adding additional computational burden. Existing training phase-aware methods can adjust for early, high-information phases, but they do not consider parameter structure characteristics, resulting in ineffective reduction of overall communication overhead and a tendency to cause over-smoothing in non-critical phases. Both of these approaches have their limitations and lack a unified framework, failing to simultaneously achieve communication efficiency, training stability, and security compatibility. Summary of the Invention

[0003] Technical problems to be solved

[0004] The technical problem this invention aims to solve is to simultaneously reduce communication overhead, improve training stability, and ensure full compatibility with homomorphic encryption secure aggregation protocols in a federated recommendation system. Existing technologies cannot simultaneously address the downlink and security compatibility deficiencies of structure compression and the communication efficiency deficiencies perceived during the training phase, thus limiting the overall system performance.

[0005] Technical solution

[0006] The technical solution of this invention is as follows: The client adopts a structurally responsive low-rank parameter update mechanism, which represents the local update of the item embedding matrix as a low-rank matrix product, and only uploads the lightweight trainable matrix part; The server calculates the federated gradient norm and determines whether it is currently in a critical learning stage, and dynamically adjusts the aggregation coefficients according to the determination result, and only enables weight adjustment based on performance score in critical stages; The low-rank matrix part is weighted and aggregated to obtain the global model update, forming a unified closed loop of structural response and training stage awareness.

[0007] Beneficial effects

[0008] Compared with the prior art, this invention achieves collaborative optimization by unifying structural response and training phase perception: communication overhead is reduced by more than 90%; the standard deviation is reduced from 0.089 to 0.063, significantly improving training stability; the recommended performance HR@10 remains above 94.2% of the baseline method; and it is fully compatible with homomorphic encrypted secure aggregation protocol. This solves the problem that communication, security and stability cannot be balanced due to the fragmented modeling of the prior art, and produces beneficial effects that the prior art could not have anticipated. Attached Figure Description

[0009] Attached Figure Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0010] This embodiment provides an adaptive aggregation federated recommendation system and method that is aware of the structure and training phase. The specific implementation is as follows (corresponding to the flowchart in the attached figures): The server first initializes the global low-rank matrix B. (0) And broadcast fixed matrix B (t) (See attached left side). The client receives B. (t) Then, perform a low-rank parameter update, representing the local update of the item embedding matrix as follows: Perform local training while calculating performance scores. Change in loss The client will use a lightweight matrix. The performance score and the change in loss are uploaded to the server.

[0011] After receiving the uploaded parameters, the server calculates the federated gradient norm FGN(t) and performs a critical learning period determination: if Then we enter the critical stage and activate the dynamic aggregation coefficient. Otherwise, proceed to the non-critical stage and use standard aggregation (see the right branch in the attached diagram). Then, perform weighted aggregation and update the global model B. (t+1) The system will then return the statistics and proceed to the next round (see the arrow below the attached image).

[0012] The above process was validated on datasets such as MovieLens-1M and Pinterest, showing a significant reduction in communication rounds and the number of transmitted parameters, resulting in a smoother loss curve. In different embodiments, low-rank dimension... =8 to =32、 =0.01、 Stable results were achieved when the value was 0.3.

Claims

1. A structure- and training-phase-aware adaptive aggregation federated recommendation system and method, characterized in that, Includes the following steps: The client employs a structured reactive low-rank parameter update mechanism, which parameterizes the local update of the item embedding matrix into a low-rank matrix product form. ,in A fixed low-rank matrix for global server broadcasting. The client uses a lightweight matrix that can be trained locally, and only the part of the lightweight trainable matrix is ​​uploaded to achieve communication compression; the server calculates the federated gradient norm and determines whether the current training round is in a critical learning stage based on a preset threshold. The server dynamically adjusts the aggregation coefficient of each client based on the discrimination results. When the client is in a critical learning phase, weight adjustment based on performance scores is enabled; otherwise, it degenerates into standard aggregation. The server performs weighted aggregation on the lightweight trainable matrix portion to obtain a global model update. The aforementioned structural response mechanism and the perception mechanism during the training phase form a unified closed loop, enabling structural adaptive weights only during the phase with the highest information content, thereby simultaneously addressing issues of communication overhead, training instability, and security aggregation compatibility.

2. The method according to claim 1, characterized in that, The federated gradient norm FGN(t) is calculated using gradient information uploaded by all clients and is used to characterize the overall amount of information in the current training round.

3. The method according to claim 1, characterized in that, The criteria for determining the key learning stage are as follows: ,in This is a preset threshold.

4. The method according to claim 1, characterized in that, In the weight adjustment based on performance scores, the aggregation coefficient ,in Provide the client with a performance score based on the local model output. To adjust the hyperparameters.

5. The method according to claim 1, characterized in that, The weighted aggregation supports a homomorphic cryptographic secure aggregation protocol, and the operations involve only addition and scalar multiplication of low-rank matrices.

6. The method according to claim 1, characterized in that, low-rank dimension The value ranges from 8 to 32, which is the threshold for the critical learning stage. The value range is from 0.005 to 0.

02.

7. A structure- and training-phase-aware adaptive aggregation federated recommendation system and method, characterized in that, include: Multiple clients are configured to use a structure-responsive low-rank parameter update mechanism, which parameterizes the local update of the item embedding matrix into a low-rank matrix product form and uploads only the lightweight trainable matrix portion. The server is configured to calculate the federated gradient norm and determine whether the current training round is in a critical learning stage based on a preset threshold. Based on the determination result, the aggregation coefficient of each client is dynamically adjusted, and the lightweight trainable matrix portion is weighted and aggregated to obtain a global model update. The aforementioned client and server form a unified closed loop of structural response mechanism and training phase perception mechanism.

8. The method according to claim 7, characterized in that, The server includes a stage discrimination module, a weight adjustment module, and an aggregation module.

9. The method according to claim 7, characterized in that, The client includes a low-rank parameterization module and an upload module.

Images