Data processing method and device for federated learning-based large language model

Abstract

The application is suitable for the field of artificial intelligence technology, and provides a data processing method and device for a large language model based on federated learning, wherein the method comprises: decomposing a large language model to obtain an adapter and a simulator; performing layer compression processing on the simulator to obtain a compressed simulator, and deploying the adapter and the compressed simulator to multiple clients; obtaining adapter parameters uploaded by each client after training the adapter based on a local data set; performing global aggregation on general adapter parameters in the adapter parameters, grouping and aggregating personalized adapter parameters in the adapter parameters to obtain aggregated adapter parameters; and outputting the aggregated adapter parameters to the clients. The scheme can realize privacy protection and retain individual user preferences.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a data processing method and apparatus for a large language model based on federated learning. Background Technology

[0002] Large Language Models (LLMs) have demonstrated exceptional capabilities in handling a wide range of open-ended tasks, prompting the field of recommender systems to begin leveraging their extensive contextual understanding and generalization abilities.

[0003] Existing LLM-based recommendation models typically employ a centralized fine-tuning approach to achieve better performance, but this paradigm has two problems:

[0004] 1) Threatens user privacy;

[0005] 2) Ignoring individual user preferences. Summary of the Invention

[0006] This application provides a data processing method and apparatus for large language models based on federated learning, in order to solve the problems that the centralized fine-tuning method of large language models in the prior art threatens user privacy and ignores individual user preferences.

[0007] The first aspect of this application provides a data processing method for a large language model based on federated learning, applied to a server, including:

[0008] The large language model is decomposed to obtain the adapter and simulator;

[0009] The simulator is subjected to layer compression to obtain a compressed simulator, and the adapter and the compressed simulator are deployed to multiple clients.

[0010] Obtain the adapter parameters uploaded by each client after training the adapter based on the local dataset;

[0011] Perform global aggregation on the general adapter parameters in the adapter parameters, and group and aggregate the personalized adapter parameters in the adapter parameters to obtain the aggregated adapter parameters;

[0012] The aggregated adapter parameters are output to the client.

[0013] A second aspect of this application provides a data processing method for a large language model based on federated learning, applied to a client, including:

[0014] Obtain the adapter and compression simulator obtained by decomposing the large language model from the server;

[0015] The adapter is trained based on the local dataset to obtain updated adapter parameters, and the adapter parameters are then uploaded to the server.

[0016] Obtain the aggregated adapter parameters sent by the server; the aggregated adapter parameters are obtained by the server performing global aggregation based on the general adapter parameters in the adapter parameters and group aggregation based on the personalized adapter parameters in the adapter parameters.

[0017] The adapter is updated based on the aggregated adapter parameters until the large language model converges.

[0018] A third aspect of this application provides a data processing apparatus for a large language model based on federated learning, comprising:

[0019] The decomposition module is used to decompose a large language model to obtain an adapter and a simulator;

[0020] A compression module is used to perform layer compression on the simulator to obtain a compressed simulator, and to deploy the adapter and the compressed simulator to multiple clients;

[0021] The first acquisition module is used to acquire the adapter parameters uploaded by each client after training the adapter based on the local dataset;

[0022] The aggregation module is used to perform global aggregation on the general adapter parameters in the adapter parameters and to group and aggregate the personalized adapter parameters in the adapter parameters to obtain aggregated adapter parameters.

[0023] The output module is used to output the aggregated adapter parameters to the client.

[0024] A fourth aspect of this application provides a data processing apparatus for a large language model based on federated learning, comprising:

[0025] The second acquisition module is used to acquire the adapter and compression simulator obtained by decomposing the large language model sent by the server.

[0026] The first update module is used to train the adapter based on the local dataset, obtain the updated adapter parameters, and upload the adapter parameters to the server;

[0027] The third acquisition module is used to acquire the aggregated adapter parameters sent by the server; the aggregated adapter parameters are obtained by the server performing global aggregation based on the general adapter parameters in the adapter parameters and group aggregation based on the personalized adapter parameters in the adapter parameters.

[0028] The second update module is used to update the adapter according to the aggregated adapter parameters until the large language model converges.

[0029] A fifth aspect of this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the methods described in the first and second aspects.

[0030] A sixth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the methods described in the first and second aspects.

[0031] The seventh aspect of this application provides a computer program product that, when run on a computer device, causes the computer device to perform the steps of the methods described in the first and second aspects.

[0032] As can be seen from the above, in this embodiment, the large language model is decomposed to obtain an adapter and a simulator. The simulator is then subjected to layer compression to obtain a compressed simulator. The adapter and the compressed simulator are deployed to multiple clients. The adapter parameters uploaded by each client after training the adapter on the local dataset are obtained. Global aggregation is performed on the general adapter parameters in the adapter parameters, and personalized adapter parameters in the adapter parameters are grouped and aggregated to obtain aggregated adapter parameters. The aggregated adapter parameters are output to the client, so that the client only stores and updates the lightweight proxy model (adapter and compressed simulator), isolates the original large language model, and does not need to transmit the original large language model parameters or sensitive user data. This fundamentally avoids the leakage of the original large language model and the exposure of user data, achieving privacy protection. Furthermore, it can obtain the adapter parameters uploaded by each client based on local data, perform global aggregation on the general adapter parameters in the adapter parameters, and group and aggregate the personalized adapter parameters in the adapter parameters, retaining the personalized parameters of each client and preserving individual user preferences. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This application provides a flowchart of a data processing method for a large language model based on federated learning. Figure 1 ;

[0035] Figure 2 This is an architecture diagram of PFedLRec provided in the embodiments of this application;

[0036] Figure 3 This is a schematic diagram of the steps under the PFedLRec architecture provided in the embodiments of this application;

[0037] Figure 4 This application provides a flowchart of a data processing method for a large language model based on federated learning. Figure 2 ;

[0038] Figure 5 This application provides a structure of a data processing method apparatus for a large language model based on federated learning, as illustrated in an embodiment of this application. Figure 1 ;

[0039] Figure 6 This application provides a structure of a data processing method apparatus for a large language model based on federated learning, as illustrated in an embodiment of this application. Figure 2 ;

[0040] Figure 7 This is a structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0041] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0042] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0043] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0044] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0045] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0046] In specific implementations, the computer devices described in the embodiments of this application include, but are not limited to, other portable devices such as mobile phones, laptop computers, or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0047] The following discussion describes computer devices including displays and touch-sensitive surfaces. However, it should be understood that computer devices may include one or more other physical user interface devices such as physical keyboards, mice, and / or joysticks.

[0048] Computer devices support a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disc burning applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, exercise support applications, photo management applications, digital camera applications, digital camcorder applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0049] Various applications that can run on a computer device can use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the computer device can be adjusted and / or changed between and / or within applications. In this way, the common physical architecture of the computer device (e.g., the touch-sensitive surface) can support various applications with user interfaces that are intuitive and transparent to the user.

[0050] It should be understood that the sequence number of each step in this embodiment does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0051] Large Language Models (LLMs) have demonstrated outstanding performance in various natural language processing tasks due to their excellent generalization ability and rich contextual understanding. However, since there is a significant gap between the pre-trained task and the current recommendation task, fine-tuning LLMs to better adapt them to recommendation tasks has become a key requirement for improving performance.

[0052] Some technologies (such as TALL-Rec and P5) use a centralized server to collect all raw user interaction data and then fine-tune the LLM.

[0053] A centralized paradigm is adopted, collecting and transmitting users' personal historical behavior (such as click and purchase records) to a central server for adaptation. However, this process may compromise user privacy and potentially violate privacy regulations. Furthermore, when sharing the same recommendation model, global model sharing leads to the averaging of user characteristic diversity, making it difficult to capture fine-grained individual preferences, and potentially compromising user characteristic diversity.

[0054] To address these issues, federated recommendation systems have emerged as an innovative paradigm, where users transmit model parameters rather than sensitive raw data to the server.

[0055] Some methods typically use LLM as a feature extractor in traditional recommendation models, neglecting its inherent reasoning capabilities and powerful text processing and generation abilities. Therefore, some strategies in this application's embodiments directly use LLM as a recommendation engine, leveraging its text processing and reasoning advantages for a deeper understanding of user preferences. Furthermore, traditional models rely on pre-generated embeddings, requiring the uploading of sensitive user behavior data. These embeddings pose privacy risks during transmission or require frequent updates as data changes, thus compromising the security of the federated learning system.

[0056] In implementation, during the recommendation process based on LLM, given an item set I = {i1, i2, ..., i...} n A time-sequential interaction sequence S = {i1, i2, ..., i} of a user. n The itemset contains object data that users can interact with. By capturing behavioral signals such as clicks or purchases through the itemset and the user's interaction sequence, the goal is to predict the next item in the sequence and model user preferences over time.

[0057] To align the recommendation process with the text generation capabilities of the LLM, the model is tuned using instructions. The historical interaction sequence S of each user is used as input, and subsequent interaction items are used as output to construct the data D={(x,y)}. This bridges the recommendation target with the pre-trained LLM capabilities, and the optimization task is as follows:

[0058]

[0059] where x and y represent the instruction input and instruction output in the instruction data, and y t is the t-th token of y, and y<t represents the sequence of tokens before y t and θ are the model parameters.

[0060] In Federated Recommendation (FR), user interaction sequences are stored locally to protect privacy. Each client c holds a local dataset Dc={(Su),∀u∈c}, where Su is the interaction history of user u. The goal is to collaboratively train an optimal global model parameter without sharing private data :[[]]END]]

[0061]

[0062] where L i (·) represents the loss function of the recommendation task on each client, and ω i is the corresponding weight. The server initializes and distributes the global parameters, and the clients perform local updates and return the parameters for aggregation, iterating until convergence.

[0063] Therefore, it is proposed to combine the LLM-based recommendation model with federated learning to provide stronger privacy protection and personalization capabilities.

[0064] When introducing the LLM into FR, two key challenges arise:

[0065] 1) High overhead: The large number of parameters in the LLM in the client leads to high computational and communication costs, especially for resource-constrained client devices;

[0066] 2) Performance degradation: The heterogeneity of client users hinders effective knowledge generalization and modeling, and traditional personalization strategies are not compatible with the LLM, reducing the recommendation accuracy.

[0067] Therefore, in some embodiments of this application, PFedLRec (Personalized Federated LLM Recommendation Framework) is proposed, an approach for efficiently deploying LLM-based federated recommendations and enhancing personalization.

[0068] Here, P stands for Personalized, Fed stands for Federated, L stands for Lightweight, and Rec stands for Recommendation. Combined, it refers to a "federated personalized recommendation framework based on a lightweight large language model".

[0069] In some implementations, a two-stage training paradigm can be followed: global collaborative training and local training, which includes three key steps:

[0070] Each client first deploys a lightweight server proxy for privacy and efficiency. It is then aligned with the original model, and LLM knowledge preservation and coarse-grained personalization are achieved through context-aware grouping. Finally, fine-grained personalization is provided through retrieval-enhanced memory-injection hints to fine-tune the strategy, resulting in effective federated recommendations.

[0071] In other words, this application proposes an efficient personalized federated LLM recommendation framework (PFedLRec), which involves the intersection of federated learning, large language models, and personalized recommendation systems. Specifically, it relates to a system and method that deploys a lightweight large language model on resource-constrained clients to achieve efficient and personalized recommendations while protecting user privacy. This lightweight federated personalized recommendation framework both protects privacy and leverages the contextual understanding capabilities of LLM.

[0072] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0073] See Figure 1 , Figure 1 This application provides a flowchart of a data processing method for a large language model based on federated learning. Figure 1 .like Figure 1 As shown, a data processing method for a large language model based on federated learning is applied to a server. The method includes the following steps:

[0074] Step 101: Decompose the large language model to obtain the adapter and simulator.

[0075] Many LLMs remain closed-source due to high training costs, privacy concerns, and commercial protections. In personalized recommendations, API-based access is insufficient because domain-specific data needs to be fine-tuned to capture user preferences and provide accurate context-aware recommendations.

[0076] To address these challenges, embodiments of this application introduce a novel segmentation strategy called Offsite-tuning into federated personalized recommendation systems. This approach is well-suited for federated personalized recommendation under resource constraints and data heterogeneity.

[0077] Optionally, the large language model is decomposed to obtain an adapter and a simulator, including:

[0078] The large language model is decomposed into an adapter formed by a set number of layers close to the input and output, and a simulator formed by the remaining parts.

[0079] "Layer" specifically refers to the Transformer Decoder Block in a large language model.

[0080] Optionally, the number of layers in the adapter can be, for example, 6 layers, 5 layers, etc. The number can be set, for example, 3 layers, 2 layers, etc.

[0081] This embodiment of the application decomposes the complete large language model M into two components: an adapter A, which contains a small number of layers close to the input and output; and a simulator. This includes the rest. Therefore, M = A∪ .

[0082] To mitigate the performance degradation caused by emulator compression, the server employs knowledge distillation technology to reduce file size. and The performance gap between them. Optionally, before deploying the adapter and compression emulator to multiple clients, the following is also included:

[0083] The compressed simulator is subjected to knowledge distillation, and the output of the knowledge-distilled compressed simulator is aligned with the uncompressed simulator in the large language model.

[0084] An implementation process is shown in the following formula:

[0085]

[0086] in, It is Knowledge Distillation Loss, used to measure the compressed simulator. Compared to the original simulator in the large language model The performance difference between them; N is the number of samples used to calculate the loss, i.e., the total number of input samples used for training; x i It is the i-th input sample. For the compressed simulator, the input x i The output; For the original simulator, the input x iThe output of is used as the target of knowledge distillation; λ is a hyperparameter used to balance the weights of the two terms in the loss function. The first term indicates the mean square error of the simulator output; the second term indicates the difference in output of the large language model M under the parameters before and after compression. It uses compressed simulator parameters. and adapter parameters The output of the complete large language model M to input x; Is it using the original simulator parameters? and adapter parameters The complete large language model M outputs on input x.

[0087] Step 102: Perform layer compression processing on the simulator to obtain a compressed simulator, and deploy the adapter and the compressed simulator to multiple clients.

[0088] To further simplify client deployment, the server compresses the emulator by uniformly removing layers before deployment. Client deployment adapter A and the compressed emulator are shown. Only A is updated to enable local personalized feature training.

[0089] Optionally, the simulator is subjected to layer compression to obtain a compressed simulator, including:

[0090] The layers in the simulator are uniformly deleted according to a set ratio to obtain the compressed simulator.

[0091] This setting percentage is, for example, 20%.

[0092] Adapter A can extract low-level features such as grammatical and lexical patterns through the input layer to support general language understanding, while capturing global context and task-specific semantics through the output layer to reflect user preferences.

[0093] Based on this, adapter A is decomposed into Ageneral (general adapter parameters) and Apersonal (personalized adapter parameters) to represent general and personalized features respectively, achieving a finer balance between generalization and personalization. The trainable parameters of the adapter, simulator, and compressed simulator are represented by the general adapter parameters W. Ag Personalized adapter parameters W Ap Simulator parameters and compression simulator parameters To improve efficiency, embodiments of this application apply the LoRA (Low-Rank Adaptation) method to the adapter and simulator, obtaining ALoRA (including AgLoRA corresponding to general adapter parameters and ApLoRA corresponding to personalized adapter parameters), ELoRA, and... This enables efficient LLM-based federated recommendations.

[0094] The adapter and compression simulator need to be deployed to multiple clients to enable client-server collaboration for model training. Specifically, the training strategy could involve the server first decomposing and compressing the full-scale LLM before training begins. (Client-side model) The (compression simulator) is initialized through knowledge distillation on public datasets such as Alpaca.

[0095] Step 103: Obtain the adapter parameters uploaded by each client after training the adapter based on the local dataset.

[0096] The client uploads the updated adapter parameters W after local training. ALoRA (Including the W corresponding to the personalized adapter parameters) ApLoRA W corresponding to the general adapter parameters AgLoRA The parameters are sent to the server. The server aggregates these parameters and uses them... and aggregation of W ALoRA Distillation is performed, guided by a complete large language model. This process updates the compression parameters to better approximate the performance of the original model, while reducing the discrepancies caused by compression.

[0097] The distillation process can strategically update and maintain W. ALoRA Freezing achieves two key objectives: (1) preserving the performance characteristics of the complete model; and (2) maintaining adaptability to client-specific data distributions.

[0098] Step 104: Perform global aggregation on the general adapter parameters in the adapter parameters, and group and aggregate the personalized adapter parameters in the adapter parameters to obtain the aggregated adapter parameters.

[0099] In federated learning, server-side parameter aggregation faces the challenge of heterogeneous client-side data, limiting the effectiveness of traditional averaging methods for personalization.

[0100] This application addresses this issue through a context-based grouping mechanism using a similarity-based clustering client. This method preserves personalization and global generalization by processing adapter parameters separately: a general approach captures shared features while a personal approach retains contextual information.

[0101] Optionally, step 104 involves grouping and aggregating the personalized adapter parameters in the adapter parameters, including:

[0102] The clients are grouped based on contextual similarity.

[0103] The personalized adapter parameters are clustered according to the grouping of the client, and the clustered personalized adapter parameters are aggregated within the group.

[0104] Before aggregation, the server uses the uploaded W Ap Parameter similarity analysis is used in an adaptive grouping client. These parameters are first mapped to dense vectors to construct a retrieval table. W Ap Parameters are processed based on similarity S i,j Clustering was performed, and the optimal number of clusters K for each round of aggregation was determined using the elbow method and contour scores.

[0105]

[0106]

[0107] During the aggregation process, W Ag Perform global aggregation, W Ap Aggregate within each cluster (i.e., aggregate within a group):

[0108]

[0109]

[0110] Finally, the server returns the updated parameters to the client for the next round of training:

[0111]

[0112] Among them, S ij It is the cosine similarity between clients i and j, used to measure their personalization parameter W. ApLoRA Similarity; is the LoRA parameter of Apersonal uploaded by the i-th client; N is the total number of clients participating in federated learning; K is the final number of clusters selected, determined by maximizing the elbow method and silhouette coefficient; Elbow(k) is the elbow method function, used to evaluate the change of clustering error with k (usually k corresponding to the inflection point is selected); Silhouette(k) is the silhouette coefficient function, which measures the compactness and separation of the clustering results (the closer the value is to 1, the better the clustering effect), and can dynamically determine the number of groups K in each round of aggregation to avoid the bias caused by fixed groups; It is the LoRA parameter of the global Ageneral after the (t+1)th round update, which is shared by all clients; is the Ageneral parameter of the i-th client in round t; N is the total number of clients, which can be averaged over the global parameters of all clients to achieve global knowledge sharing; The LoRA parameter of Apersonal for the k-th cluster after aggregation in round t+1 is only returned to clients within that cluster; C k It is the set of client indexes contained in the k-th cluster; |C k | is the number of clients in the k-th cluster; It is the personalized parameter of the i-th client in the t-th round. It can take the average of the LoRA parameters of the client Apersonal within the same cluster, preserve the similarity within the group, and distinguish the personalized features of different clusters. This means that the server returns a set of parameters for all K clusters, and the client receives the corresponding personalized parameters according to its own cluster, realizing a hybrid update strategy of "global sharing plus intra-group personalization".

[0113] Step 105: Output the aggregated adapter parameters to the client.

[0114] The above process, combined with Figure 2 As shown, to achieve efficient and personalized LLM-based federated recommendations, this application proposes PFedLRec. An overview of PFedLRec is as follows... Figure 2 As shown, it comprises two main phases: a global phase of collaborative training and a post-training phase following federated learning. The first phase aims to preserve existing knowledge while achieving generalization across groups and coarse-grained personalization. The second phase focuses on fine-grained personalization, enhancing user modeling tailored to each local need.

[0115] Combination Figure 3 As shown, the specific steps involved in the implementation of each part are broken down in detail:

[0116] Step 1: Simulator Compression and Alignment. Efficient federated recommendations are achieved by compressing the LLM while maintaining its performance through alignment.

[0117] Step 2: Federated training with contextual grouping. Utilize adapter components for multi-granularity aggregation, balancing user similarity and global generalization;

[0118] Step 3: Fine-grained user modeling. Utilize application retrieval to access relevant local records, enhancing LLM personalization and user preference modeling.

[0119] In this embodiment, the large language model is decomposed to obtain an adapter and a simulator. The simulator is then compressed to obtain a compressed simulator. The adapter and compressed simulator are deployed to multiple clients. The adapter parameters uploaded by each client after training the adapter on its local dataset are obtained. Global aggregation is performed on the general adapter parameters in the adapter parameters, and personalized adapter parameters in the adapter parameters are grouped and aggregated to obtain aggregated adapter parameters. The aggregated adapter parameters are output to the clients, allowing the clients to store and update only the lightweight proxy model (adapter and compressed simulator), isolating the original large language model. There is no need to transmit the original large language model parameters or sensitive user data, fundamentally avoiding the leakage of the original large language model and the exposure of user data, thus achieving privacy protection. Furthermore, it is possible to obtain the adapter parameters uploaded by each client based on local data, perform global aggregation on the general adapter parameters in the adapter parameters, and group and aggregate the personalized adapter parameters in the adapter parameters, retaining the personalized parameters of each client and preserving individual user preferences.

[0120] This application also provides different implementations of a data processing method for a large language model based on federated learning in its embodiments.

[0121] See Figure 4 , Figure 4 This is the flow chart of the data processing method for a large language model based on federated learning provided in the embodiments of this application. Figure 2 .like Figure 4 As shown, a data processing method for large language models based on federated learning is applied to the client side. This method includes the following steps:

[0122] Step 401: Obtain the adapter and compression simulator obtained by decomposing the large language model from the server.

[0123] Step 402: Train the adapter based on the local dataset to obtain updated adapter parameters, and upload the adapter parameters to the server.

[0124] Optionally, training the adapter based on the local dataset to obtain updated adapter parameters includes:

[0125] Freeze the compression simulator;

[0126] The adapter is trained based on the local dataset to obtain updated adapter parameters.

[0127] Client-side deployment of lightweight model M=[Ag, Ap, Ag (A general) and Ap (A personal) form adapter A. Represents a compressed simulator that maintains [data / capacity] during local training. Freeze. Adapter W only. ALoRA Update (see formula below):

[0128]

[0129] Where [M] is the total number of clients participating in federated learning, and m represents the m-th client; This represents a client-specific loss function. For the corresponding loss weights, These are adapter parameters. These are the simulator parameters; λ is the hyperparameter of the regularization term, which controls the strength of the parameter deviation constraint. This is the current LoRA adapter parameter vector to be optimized; These are the LoRA parameters initialized at the start of the t-th round of federated learning; It is the square of the L2 norm, used to measure the deviation of the parameter from the initial value.

[0130] Regularization limits the adapter parameters from deviating from their initial values ​​in each round, mitigating performance discrepancies caused by local client adaptation.

[0131] This design reduces storage and computing costs for resource-constrained clients.

[0132] Step 403: Obtain the aggregated adapter parameters sent by the server.

[0133] The aggregated adapter parameters are obtained by the server performing global aggregation based on the general adapter parameters in the adapter parameters and grouping aggregation based on the personalized adapter parameters in the adapter parameters.

[0134] Step 404: Update the adapter according to the aggregated adapter parameters, and iteratively execute the above process until the large language model converges.

[0135] Specifically, iteratively executing the above process may involve performing steps 402-403 and updating the adapter based on the aggregated adapter parameters to perform iterative training of the large language model until it converges. This allows for adjustments to the large language model locally on the client side.

[0136] In one example, a recommendation system is formed by using a large language model for information recommendation. After the large language model converges, recommendation results can be generated and output based on the user's input text.

[0137] In implementation, each client uses the updated Start a new local training session and iteratively optimize W. ALoRAThis continues until convergence. This ensures global model consistency within the federated framework while maintaining local adaptability.

[0138] After receiving the global model, each client uses search enhancement suggestions and the global model to perform local fine-grained user modeling, enhancing client-level personalization.

[0139] To implement enhanced search suggestions based on local user modeling, the method further includes:

[0140] Retrieve the top M items from the local knowledge base that are similar to historical interaction data; M ≥ 2;

[0141] Based on the first M items, construct enhanced prompt information;

[0142] The enhanced prompt information is encoded into a persistent memory vector, and the persistent memory vector is injected into the prompt word template of the converged large language model to obtain the adjusted large language model.

[0143] The first M items refer to historical interaction items, such as historical browsing data items, historical information receiving data items, and historical information output data items, like historical movie browsing, historical song playback, and historical voice calls. Based on the first M items, item information can be extracted to construct enhanced prompts.

[0144] Enhanced prompts are personalized prompts added to the prompts in the large language model, making them more relevant to the current client's habits.

[0145] For each client, perform retrieval enhancement hints based on local user modeling after federated learning is complete.

[0146] Global model training uses user item interactions to protect privacy, while local modeling incorporates richer personal data for better personalization. However, the LLM context length constraint limits the length of the behavior sequence and the number of candidate items included in the prompt, reducing recommendation performance. This application addresses this issue through two-stage prompt tuning with enhanced retrieval capabilities.

[0147] Phase 1: Retrieval Enhancement Prompt Design. This application's embodiment constructs local projects and user profiles to form a knowledge base. It implements personalized enhancement of prompt words, enabling the large language model to better integrate user-specific information for information recommendation.

[0148] Table 1: Local Projects

[0149]

[0150] Table 2: User Profile

[0151]

[0152] For each user, the BM25 retrieval algorithm can be used to retrieve the top K similar items from their top N interactive items in the knowledge base. Recommendation suggestions are then constructed based on these retrieved items.

[0153] Instruction: Considering that User 1 has already interacted with ML100K items, where each item is represented as item_id(score=, genre=, similar items=item_id(score)). `score` represents the user's preference for the item, ranging from 0 (dislike) to 5 (strongly like). `genre` indicates the category to which the item belongs. `similar items` represents the three most similar items. Based on the above information, recommend items for the user.

[0154] Input: User 1 has interacted with item 1036 (Rating=4, Type=Action, Adventure, Similar items=['Item 1018(4)', 'Item 1181(3)', 'Item 1011(4)'])...

[0155] Output: Item 1181.

[0156] Phase 2: Personalized Cueing for Memory Enhancement. Given the relatively structured and static attributes in user profiles, this embodiment parameterizes the structured user profile into persistent memory, enabling direct integration into training. This memory is injected into the cue words from Phase 1, allowing for full utilization of user preferences while alleviating context length constraints.

[0157] The above implementation process divides the overall learning process into a global phase of collaborative training and a local phase after training. In the first phase, a locally compressed proxy model deployment is introduced to prevent leakage of the original LLM, thereby ensuring privacy protection. Furthermore, a two-level aggregation strategy during collaborative training aligns the proxy with the original model, preserving its capabilities while obtaining general representations and coarse-grained personalization. In the second phase, embodiments of this application enhance fine-grained personalization capabilities by enhancing prompts to retrieve additional relevant historical information and user profiles.

[0158] To demonstrate the effectiveness of PFedLRec, this application's embodiments conducted extensive experiments on multiple recommendation benchmark datasets. These numerous experiments on benchmark datasets demonstrate the effectiveness of PFedLRec.

[0159] Experimental results show that PFedLRec can significantly improve the performance of LLMRec while maintaining its privacy and enhanced personalization. The main contributions of the embodiments of this application are summarized as follows:

[0160] PFedLRec implements efficient LLM-based federated recommendation, enhancing personalization while leveraging the contextual reasoning capabilities of LLM and maintaining privacy and efficiency.

[0161] The framework employs two steps: federated learning and local post-training on the client side, and uses a lightweight client agent to implement personalized recommendations.

[0162] Extensive evaluations on multiple datasets demonstrate that PFedLRec consistently improves recommendation accuracy and computational efficiency across various settings.

[0163] The specific experimental and in-depth analysis process includes, for example, the following:

[0164] 1. Ablation research.

[0165] To analyze the impact of each component in PFedLRec, embodiments of this application removed the grouping process and local user modeling, referred to as "w / o GP", "w / o PT", and "w / o GP&PT", respectively. The results show that: (1) in LLM-based federated recommendation, both grouping and prompts are crucial for performance optimization; (2) prompts contribute more significantly because they leverage local personal data to enhance personalization.

[0166] 2. Adaptive grouping effect.

[0167] This application's embodiments found that the variation of the optimal cluster number K across rounds reflects dynamic user behavior patterns. This prompted this application's embodiments to employ an adaptive clustering method, which outperforms fixed grouping (K=2,3) on both HR@10 and NDCG@10 metrics. The results confirm that dynamic clustering better captures changes in data distribution, resulting in superior recommendation performance.

[0168] This application's embodiments demonstrate that PFedLRec significantly reduces trainable parameters, GPU memory, and computational costs while maintaining acceptable accuracy, enabling resource-constrained clients to deploy LLM-based recommendations. Results confirm that PFedLRec successfully balances the performance and efficiency of federated LLM deployments.

[0169] This embodiment enhances the privacy protection of PFedLRec by incorporating local differential privacy, setting the noise intensity δ=[0,0.1,0.2,0.3,0.4,0.5]. Performance degrades as the noise intensity δ increases. To achieve a trade-off between recommendation accuracy and privacy protection, this embodiment can set δ=0.2, at which point performance still outperforms most baselines, achieving an enhanced privacy-protected PFedLRec.

[0170] This application proposes PFedLRec, providing an efficient framework for LLM-based recommendation in federated learning. By strategically adjusting the LLM architecture, PFedLRec achieves scalable deployment on resource-constrained clients. The two-stage training optimization of this application's embodiments optimizes performance: (1) two-level aggregation balances personalization and global generalization; (2) retrieval enhancement hints utilize local data to achieve fine-grained personalization. Extensive experiments demonstrate superior performance, and privacy-preserving integration confirms robustness.

[0171] See Figure 5 , Figure 5 This application provides the structure of a data processing device for a large language model based on federated learning, as illustrated in an embodiment. Figure 1 For ease of explanation, only the parts relevant to the embodiments of this application are shown. The data processing device for the large language model in the federated learning can be a server.

[0172] The data processing device 500 for a large language model based on federated learning includes:

[0173] The decomposition module 501 is used to decompose the large language model to obtain the adapter and simulator;

[0174] Compression module 502 is used to perform layer compression processing on the simulator to obtain a compressed simulator, and to deploy the adapter and the compressed simulator to multiple clients;

[0175] The first acquisition module 503 is used to acquire the adapter parameters uploaded by each client after training the adapter based on the local dataset;

[0176] Aggregation module 504 is used to perform global aggregation on the general adapter parameters in the adapter parameters and to group and aggregate the personalized adapter parameters in the adapter parameters to obtain aggregated adapter parameters;

[0177] Output module 505 is used to output the aggregated adapter parameters to the client.

[0178] Optionally, the decomposition module 501 is specifically used for:

[0179] The large language model is decomposed into an adapter formed by a set number of layers close to the input and output, and a simulator formed by the remaining parts.

[0180] Optionally, the compression module 502 is specifically used for:

[0181] The layers in the simulator are uniformly deleted according to a set ratio to obtain the compressed simulator.

[0182] Optionally, aggregation module 504 is specifically used for:

[0183] The clients are grouped based on contextual similarity.

[0184] The personalized adapter parameters are clustered according to the grouping of the client, and the clustered personalized adapter parameters are aggregated within the group.

[0185] Optionally, the compression module 502 is also specifically used for:

[0186] The compressed simulator is subjected to knowledge distillation, and the output of the knowledge-distilled compressed simulator is aligned with the uncompressed simulator in the large language model.

[0187] The data processing apparatus for a large language model based on federated learning provided in this application embodiment can implement all the processes of the above-described data processing method for a large language model based on federated learning, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0188] See Figure 6 , Figure 6 This application provides the structure of a data processing device for a large language model based on federated learning, as illustrated in an embodiment. Figure 2 For ease of explanation, only the parts relevant to the embodiments of this application are shown. The data processing device for the large language model of the federated learning can be a client.

[0189] A data processing device 600 for a large language model based on federated learning includes:

[0190] The second acquisition module 601 is used to acquire the adapter and compression simulator obtained by decomposing the large language model sent by the server.

[0191] The first update module 602 is used to train the adapter based on the local dataset, obtain the updated adapter parameters, and upload the adapter parameters to the server;

[0192] The third acquisition module 603 is used to acquire the aggregated adapter parameters sent by the server; the aggregated adapter parameters are obtained by the server performing global aggregation based on the general adapter parameters in the adapter parameters and group aggregation based on the personalized adapter parameters in the adapter parameters.

[0193] The second update module 604 is used to update the adapter according to the aggregated adapter parameters, and iteratively execute the above process until the large language model converges.

[0194] Optionally, the device further includes:

[0195] A fine-tuning module is used to retrieve the top M items similar to historical interaction data from the local knowledge base; M≥2; based on the top M items, enhance prompt information is constructed; the enhance prompt information is encoded into a persistent memory vector, and the persistent memory vector is injected into the prompt word template of the converged large language model to obtain the adjusted large language model.

[0196] Optionally, the first update module 602 is specifically used for:

[0197] Freeze the compression simulator;

[0198] The adapter is trained based on the local dataset to obtain updated adapter parameters.

[0199] The data processing apparatus for a large language model based on federated learning provided in this application embodiment can implement all the processes of the above-described data processing method for a large language model based on federated learning, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0200] Figure 7 This is a structural diagram of a computer device provided in an embodiment of this application. As shown in the figure, the computer device 7 of this embodiment includes: at least one processor 70 ( Figure 7 (Only one is shown in the diagram), memory 71, and computer program 72 stored in said memory 71 and executable on said at least one processor 70, which, when executed, implements the steps in any of the above method embodiments.

[0201] The computer device 7 can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device 7 may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 7 This is merely an example of computer device 7 and does not constitute a limitation on computer device 7. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device may also include input / output devices, network access devices, buses, etc.

[0202] The processor 70 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0203] The memory 71 can be an internal storage unit of the computer device 7, such as a hard disk or RAM of the computer device 7. The memory 71 can also be an external storage device of the computer device 7, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device 7. Furthermore, the memory 71 can include both internal and external storage units of the computer device 7. The memory 71 is used to store the computer program and other programs and data required by the computer device. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0204] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0205] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0206] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0207] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0208] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0209] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0210] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0211] The methods described in this application can be implemented in whole or in part by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the various method embodiments described above.

[0212] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A data processing method of a large language model based on federated learning, applied to a server, characterized in that, The method comprises the following steps: decomposing a large language model to obtain an adapter and a simulator; wherein the adapter comprises general adapter parameters and personalized adapter parameters, the general adapter parameters are used to represent general features, and the personalized adapter parameters are used to represent personalized features; the converged large language model is used to generate a recommended result based on user input text and output; performing layer compression processing on the simulator to obtain a compressed simulator, and deploying the adapter and the compressed simulator to a plurality of clients; obtaining adapter parameters uploaded by each client after training the adapter based on a local data set; the uploaded adapter parameters comprise updated general adapter parameters and updated personalized adapter parameters; adaptively grouping the clients according to the similarity of the personalized adapter parameters, performing global aggregation on the general adapter parameters in the adapter parameters, and aggregating the personalized adapter parameters in the adapter parameters in the groups where the personalized adapter parameters are located respectively to obtain aggregated adapter parameters; the global aggregation comprises averaging the general adapter parameters of all clients; and the in-group aggregation comprises averaging the personalized adapter parameters of the clients in the same group; outputting the aggregated adapter parameters to the clients.

2. The method of claim 1, wherein, The decomposing a large language model to obtain an adapter and a simulator comprises: decomposing the large language model into an adapter formed by a set number of layers close to the input and the output, and a simulator formed by the remaining part.

3. The method of claim 1, wherein, The performing layer compression processing on the simulator to obtain a compressed simulator comprises: uniformly deleting layers in the simulator according to a set proportion to obtain the compressed simulator.

4. The method of claim 1, wherein, The grouping and aggregating the personalized adapter parameters in the adapter parameters comprises: grouping the clients according to context similarity; clustering the personalized adapter parameters according to the grouping of the clients, and aggregating the clustered personalized adapter parameters in the groups.

5. The method of claim 1, wherein, Before the deploying the adapter and the compressed simulator to a plurality of clients, the method further comprises: performing knowledge distillation on the compressed simulator, and aligning the output of the knowledge-distilled compressed simulator with the simulator in the large language model that is not compressed.

6. A data processing method of a large language model based on federated learning, applied to a client, characterized in that, The method comprises the following steps: obtaining an adapter and a compressed simulator obtained by decomposing a large language model and sent by a server; the adapter comprises general adapter parameters and personalized adapter parameters, the general adapter parameters are used to represent general features, and the personalized adapter parameters are used to represent personalized features; training the adapter based on a local data set to obtain updated adapter parameters, and uploading the adapter parameters to the server; the uploaded adapter parameters comprise updated general adapter parameters and updated personalized adapter parameters; obtaining aggregated adapter parameters sent by the server; The post-aggregation adapter parameters are used for adaptive grouping of the clients by the server according to the similarity of the personalized adapter parameters, and global aggregation based on the general adapter parameters in the adapter parameters and post-aggregation based on the personalized adapter parameters in the adapter parameters are performed; The global aggregation includes averaging the general adapter parameters of all clients, and the group aggregation includes averaging the personalized adapter parameters of the clients in the same group; The adapter is updated according to the post-aggregation adapter parameters until the large language model converges; and the converged large language model is used for generating a recommended result based on user input text and outputting.

7. The method of claim 6, wherein, The method further comprises: Retrieving the top M items similar to the historical interaction data from the local knowledge base; M≥1; Based on the top M items, constructing enhanced prompt information; Encoding the enhanced prompt information into a long-term memory vector, and injecting the long-term memory vector into a prompt word template of the converged large language model to obtain an adjusted large language model.

8. The method of claim 6, wherein, The training of the adapter based on the local data set to obtain updated adapter parameters comprises: Freezing the compressed simulator; Training the adapter based on the local data set to obtain updated adapter parameters.

9. A data processing apparatus of a large language model based on federated learning, characterized by, Comprise: A decomposition module is configured to decompose a large language model to obtain an adapter and a simulator; wherein the adapter comprises general adapter parameters and personalized adapter parameters, the general adapter parameters are used to represent general features, and the personalized adapter parameters are used to represent personalized features; the converged large language model is used to generate a recommended result based on user input text and output; A compression module is configured to perform layer compression processing on the simulator to obtain a compressed simulator, and deploy the adapter and the compressed simulator to a plurality of clients; A first acquisition module is configured to acquire adapter parameters uploaded by each client after training the adapter based on a local data set; the uploaded adapter parameters comprise updated general adapter parameters and updated personalized adapter parameters; An aggregation module is configured to adaptively group the clients according to the similarity of the personalized adapter parameters, perform global aggregation on the general adapter parameters in the adapter parameters, and aggregate the personalized adapter parameters in the adapter parameters in the group where each is located to obtain post-aggregation adapter parameters; the global aggregation includes averaging the general adapter parameters of all clients; and the group aggregation includes averaging the personalized adapter parameters of the clients in the same group; An output module is configured to output the post-aggregation adapter parameters to the clients.

10. A data processing apparatus of a large language model based on federated learning, characterized by, Comprise: A second acquisition module is configured to acquire an adapter and a compressed simulator obtained by decomposing a large language model by a server; The adapter comprises general adapter parameters and personalized adapter parameters, the general adapter parameters are used to represent general features, and the personalized adapter parameters are used to represent personalized features; The first updating module is configured to train the adapter based on the local dataset, obtain updated adapter parameters, and upload the adapter parameters to the server; The uploaded adapter parameters include updated general adapter parameters and updated personalized adapter parameters; The third obtaining module is configured to obtain aggregated adapter parameters sent by the server; The aggregated adapter parameters are obtained by performing global aggregation based on the general adapter parameters in the adapter parameters and performing in-group aggregation based on the personalized adapter parameters in the adapter parameters after adaptive grouping of the clients by the server according to the similarity of the personalized adapter parameters; The global aggregation includes averaging the general adapter parameters of all clients; and the in-group aggregation includes averaging the personalized adapter parameters of the clients in the same group; The second updating module is configured to update the adapter according to the aggregated adapter parameters until the large language model converges; and the converged large language model is configured to generate a recommended result based on a user input text and output the recommended result.

Images