Big language model data processing method and device based on federal learning

Abstract

The invention is suitable for the technical field of artificial intelligence, and provides a data processing method and device for a big language model based on federal learning, and the method comprises the steps: decomposing the big language model, and obtaining an adapter and a simulator; performing layer compression processing on the simulator to obtain a compression simulator, and deploying the adapter and the compression simulator to a plurality of clients; obtaining adapter parameters uploaded by each client after training the adapter based on a local data set; performing global aggregation on universal adapter parameters in the adapter parameters, and performing grouping aggregation on personalized adapter parameters in the adapter parameters to obtain aggregated adapter parameters; and outputting the aggregated adapter parameters to the client. According to the scheme, privacy protection can be realized, and individual user preferences are reserved.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular relates to a data processing method and device for a large language model based on federated learning. Background Art

[0002] Large Language Models (LLMs) have demonstrated excellent capabilities in handling various open-ended tasks, prompting the recommendation system field to begin leveraging their extensive context understanding and generalization capabilities.

[0003] Existing LLM-based recommendation models usually adopt a centralized fine-tuning approach to achieve better performance, but this paradigm has two problems: 1) Threaten user privacy; 2) Ignore individual user preferences. Summary of the Invention

[0004] The embodiments of the present application provide a data processing method and device for a large language model based on federated learning to solve the problem in the prior art that the centralized fine-tuning method of large language models threatens user privacy and ignores individual user preferences.

[0005] A first aspect of an embodiment of the present application provides a data processing method for a large language model based on federated learning, which is applied to a server and includes: Decompose the large language model to obtain adapters and simulators; Performing layer compression processing on the simulator to obtain a compressed simulator, and deploying the adapter and the compressed simulator to multiple clients; Obtaining adapter parameters uploaded by each client after training the adapter based on a local data set; performing global aggregation on the common adapter parameters in the adapter parameters, grouping and aggregating the personalized adapter parameters in the adapter parameters to obtain aggregated adapter parameters; The aggregated adapter parameters are output to the client.

[0006] A second aspect of the embodiments of the present application provides a data processing method for a large language model based on federated learning, which is applied to a client and includes: Obtaining the adapter and compression simulator obtained by decomposing the large language model sent by the server; Training the adapter based on the local data set to obtain updated adapter parameters, and uploading the adapter parameters to the server; Acquire aggregated adapter parameters sent by the server; the aggregated adapter parameters are obtained by the server performing global aggregation based on the general adapter parameters in the adapter parameters and group aggregation based on the personalized adapter parameters in the adapter parameters; The adapter is updated according to the aggregated adapter parameters until the large language model converges.

[0007] A third aspect of the embodiments of the present application provides a data processing device for a large language model based on federated learning, comprising: Decomposition module, used to decompose the large language model to obtain adapters and simulators; A compression module, configured to perform layer compression processing on the simulator to obtain a compressed simulator, and deploy the adapter and the compressed simulator to multiple clients; A first acquisition module is used to obtain adapter parameters uploaded by each client after training the adapter based on a local data set; an aggregation module, configured to perform global aggregation on the common adapter parameters in the adapter parameters, and group and aggregate the personalized adapter parameters in the adapter parameters to obtain aggregated adapter parameters; An output module is used to output the aggregated adapter parameters to the client.

[0008] A fourth aspect of the embodiments of the present application provides a data processing device for a large language model based on federated learning, comprising: The second acquisition module is used to obtain the adapter and compression simulator obtained by decomposing the large language model sent by the server; A first updating module is configured to train the adapter based on a local data set to obtain updated adapter parameters and upload the adapter parameters to the server; A third acquisition module is configured to acquire aggregated adapter parameters sent by the server; the aggregated adapter parameters are obtained by the server performing global aggregation based on the general adapter parameters in the adapter parameters and group aggregation based on the personalized adapter parameters in the adapter parameters; The second updating module is configured to update the adapter according to the aggregated adapter parameters until the large language model converges.

[0009] The fifth aspect of an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first and second aspects when executing the computer program.

[0010] A sixth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method described in the first and second aspects are implemented.

[0011] The seventh aspect of the present application provides a computer program product, which, when executed on a computer device, enables the computer device to execute the steps of the methods described in the first and second aspects above.

[0012] As can be seen from the above, an embodiment of the present application decomposes a large language model to obtain an adapter and a simulator, performs layer compression processing on the simulator to obtain a compressed simulator, deploys the adapter and the compressed simulator to multiple clients, obtains the adapter parameters uploaded by each client after training the adapter based on a local data set, performs global aggregation on the general adapter parameters in the adapter parameters, groups and aggregates the personalized adapter parameters in the adapter parameters to obtain aggregated adapter parameters, and outputs the aggregated adapter parameters to the client, so that the client only stores and updates the lightweight proxy model (adapter and compressed simulator), isolates the original large language model, and does not need to transmit the original large language model parameters or user sensitive data, fundamentally avoiding the leakage of the original large language model and the exposure of user data, achieving privacy protection, and being able to obtain the adapter parameters uploaded by each client based on local data, perform global aggregation on the general adapter parameters in the adapter parameters, group and aggregate the personalized adapter parameters in the adapter parameters, retain the personalized parameters of each client, and retain individual user preferences. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0014] Figure 1 This is a process of a data processing method for a large language model based on federated learning provided in an embodiment of the present application. Figure 1 ; Figure 2 This is an architectural diagram of the PFedLRec provided in an embodiment of the present application; Figure 3 This is a schematic diagram of the steps under the PFedLRec architecture provided in an embodiment of the present application; Figure 4 This is a process of a data processing method for a large language model based on federated learning provided in an embodiment of the present application. Figure 2 ; Figure 5 This is the structure of a data processing method and device for a large language model based on federated learning provided in the embodiment of the present application. Figure 1 ; Figure 6 This is the structure of a data processing method and device for a large language model based on federated learning provided in the embodiment of the present application. Figure 2 ; Figure 7 This is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0016] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0017] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0018] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0019] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0020] In specific implementations, the computer devices described in the embodiments of the present application include, but are not limited to, other portable devices such as mobile phones, laptop computers, or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or touch pads). It should also be understood that in some embodiments, the device is not a portable communication device, but rather a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or touch pads).

[0021] In the following discussion, a computer device including a display and a touch-sensitive surface is described. However, it should be understood that a computer device may include one or more other physical user interface devices such as a physical keyboard, mouse, and / or joystick.

[0022] The computer device supports various applications, such as one or more of the following: a drawing application, a presentation application, a word processing application, a website creation application, a disk burning application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a workout support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and / or a digital video player application.

[0023] Various applications that can be executed on a computer device can use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the computer device can be adjusted and / or changed between applications and / or within a respective application. In this way, the common physical architecture of the computer device (e.g., the touch-sensitive surface) can support a variety of applications with user interfaces that are intuitive and transparent to the user.

[0024] It should be understood that the size of the serial numbers of each step in this embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of this application.

[0025] Large language models (LLMs) have demonstrated excellent performance in various natural language processing tasks due to their excellent generalization and rich contextual understanding capabilities. However, due to the significant gap between pre-trained tasks and current recommendation tasks, fine-tuning LLMs to better adapt to recommendation tasks has become a key requirement for improving performance.

[0026] In some technologies (such as TALL-Rec and P5), a centralized server is used to collect all users' original interaction data and then fine-tune the LLM.

[0027] Using a centralized paradigm, the personal historical behaviors of users (such as click and purchase records) are collected and transmitted to a central server for adaptation. However, this process may compromise user privacy and may violate privacy protection regulations. In addition, when sharing the same recommendation model, global model sharing leads to the averaging of user feature diversity, making it difficult to capture individual fine-grained preferences, and the diversity of user features may be damaged.

[0028] To address these issues, the federated recommendation system emerges as an innovative paradigm where users transmit model parameters rather than sensitive raw data to the server.

[0029] In some methods, the LLM is usually used as a feature extractor for traditional recommendation models, ignoring its inherent reasoning ability and powerful text processing and generation capabilities. Therefore, some strategies in the embodiments of this application directly use the LLM as a recommendation engine, leveraging its text processing and reasoning advantages to gain a deeper understanding of user preferences. In addition, traditional models rely on pre-generated embeddings and need to upload users' sensitive behavior data. These embeddings pose a risk of privacy leakage during transmission or need to be updated frequently as the data changes, thus compromising the security of the federated learning system.

[0030] In implementation, during the recommendation process based on the LLM, given an item set I = {i1, i2,..., i n} and a user's chronological interaction sequence S = {i1, i2,..., i n}, the item set contains object data that users can interact with, and behavior signals such as clicks or purchases are captured through the item set and the user's interaction sequence. The goal is to predict the next item in the sequence and model user preferences over time.

[0031] To align the recommendation process with the text generation ability of the LLM, instructions are applied to tune the model. The historical interaction sequence S of each user is used as the input, and the subsequent interaction items are used as the output to construct the data D = {(x, y)}. This bridges the recommendation goal with the pre-trained LLM capabilities, and the optimization task is:

[0032] where x and y represent the instruction input and instruction output in the instruction data, y t is the t-th token of y, y<t represents the sequence of tokens before y t and θ is the model parameter.

[0033] In Federated Recommendation (FR), user interaction sequences are stored locally to protect privacy. Each client c holds a local dataset Dc={(Su),∀u∈c}, where Su is the interaction history of user u. The goal is to collaboratively train an optimal global model parameter without sharing private data. :

[0034] Among them L i (·) represents the loss function of the recommendation task on each client, ω i The server initializes and distributes global parameters, and the client performs local updates and returns parameters for aggregation, iterating until convergence.

[0035] Therefore, it is proposed to combine the LLM-based recommendation model with federated learning to provide stronger privacy protection and personalization capabilities.

[0036] When introducing LLM into FR, two key challenges arise: 1) High overhead: LLM in the client leads to large computation and communication costs due to the large number of parameters, especially for resource-constrained client devices; 2) Performance degradation: Client user heterogeneity hinders effective knowledge generalization and modeling, and traditional personalization strategies are incompatible with LLM, which reduces recommendation accuracy.

[0037] Therefore, in some embodiments of the present application, PFedLRec (Personalized Federated LLM Recommendation Framework) is proposed, which is a method for efficiently deploying LLM-based federated recommendations and enhancing personalization.

[0038] P stands for Personalized, Fed stands for Federated, L stands for Lightweight, and Rec stands for Recommendation. Together, they refer to the "federated personalized recommendation framework for lightweight large language models."

[0039] In some implementations, a two-stage training paradigm can be followed: global collaborative training and local training, which consists of three key steps: Each client first deploys a lightweight server agent to achieve privacy and efficiency. It then aligns with the original model and achieves LLM knowledge preservation and coarse-grained personalization through context-aware grouping. Finally, a retrieval-enhanced memory-injected hint fine-tuning strategy provides fine-grained personalization, resulting in effective federated recommendations.

[0040] Specifically, the embodiments of this application propose an efficient personalized federated LLM recommendation framework (PFedLRec), which addresses the intersection of federated learning, large language models, and personalized recommendation systems. In particular, the present invention relates to a system and method for deploying lightweight large language models on resource-constrained clients to achieve efficient, personalized recommendations while protecting user privacy. This lightweight federated personalized recommendation framework is designed to both protect privacy and leverage the contextual understanding capabilities of LLMs.

[0041] In order to illustrate the technical solution described in this application, specific embodiments are provided below.

[0042] See also Figure 1 , Figure 1 This is a process of a data processing method for a large language model based on federated learning provided in an embodiment of the present application. Figure 1 .like Figure 1 As shown, a data processing method for a large language model based on federated learning is applied to a server. The method includes the following steps: Step 101: Decompose the large language model to obtain an adapter and a simulator.

[0043] Many LLMs remain closed source due to high training costs, privacy concerns, and commercial protection. In personalized recommendation, API-based access is not sufficient because domain-specific data needs to be fine-tuned to capture user preferences and provide accurate context-aware recommendations.

[0044] To address these challenges, the present invention introduces a novel segmentation strategy called Offsite-tuning into a federated personalized recommendation system. This approach is well-suited for federated personalized recommendations under resource constraints and data heterogeneity.

[0045] Optionally, the large language model is decomposed to obtain adapters and simulators, including: The large language model is decomposed into an adapter formed by a set number of layers close to the input and output, and a simulator formed by the remaining parts.

[0046] "Layer" specifically refers to the Transformer Decoder Block in the large language model.

[0047] Optionally, the number of layers of the adapter is, for example, 6 layers, 5 layers, etc. The set number is, for example, 3, 2, etc.

[0048] The embodiment of the present application decomposes the complete large language model M into two components: an adapter A, which contains a small number of layers close to the input and output; and a simulator , including the rest. So M=A∪ .

[0049] To alleviate the performance degradation caused by simulator compression, the server uses knowledge distillation technology to reduce and Optionally, before deploying the adapter and compression simulator to multiple clients, also include: Knowledge distillation is performed on the compression simulator, and the output of the compression simulator after knowledge distillation is aligned with the uncompressed simulator in the large language model.

[0050] An implementation process is shown below:

[0051] in, is the knowledge distillation loss, which is used to measure the compressed simulator Compared with the original simulator in the large language model The performance difference between them; N is the number of samples used to calculate the loss, that is, the total number of input samples involved in training; x i is the i-th input sample, The compressed simulator input x i Output; For the original simulator input x i The output of is used as the goal of knowledge distillation; λ is a hyperparameter used to balance the weights of the two terms in the loss function. The first function term indicates the mean square error of the simulator output; the second function term indicates the output difference of the large language model M under the parameters before and after compression; Is to use the compressed simulator parameters and adapter parameters The output of the complete large language model M for the input x; Is to use the original simulator parameters and adapter parameters The output of the complete large language model M for input x.

[0052] Step 102: perform layer compression processing on the simulator to obtain a compressed simulator, and deploy the adapter and the compressed simulator to multiple clients.

[0053] To further simplify client deployment, the server compresses the simulator by evenly removing layers before deployment. Client deploys adapter A and compresses the simulator , only A is updated to achieve local personalized feature training.

[0054] Optionally, performing layer compression processing on the simulator to obtain a compressed simulator includes: The layers in the simulator are evenly deleted according to a set ratio to obtain the compression simulator.

[0055] The set ratio is, for example, 20%.

[0056] Adapter A can be an input layer that extracts low-level features such as grammatical and lexical patterns to support general language understanding, while the output layer captures global context and task-specific semantics to reflect user preferences.

[0057] Based on this, the adapter A is decomposed into Ageneral (general adapter parameters) and Apersonal (personalized adapter parameters) to represent general and personalized features respectively, achieving a more refined balance between generalization and personalization. The trainable parameters of the adapter, simulator, and compressed simulator are represented as general adapter parameters W Ag , personalized adapter parameters W Ap , simulator parameters and compression simulator parameters To improve efficiency, the present embodiment applies the LoRA (Low-Rank Adaptation) method to the adapter and simulator to obtain ALoRA (including AgLoRA corresponding to the general adapter parameters and ApLoRA corresponding to the personalized adapter parameters), ELoRA and , to achieve efficient LLM-based federated recommendation.

[0058] The adapter and compression simulator need to be deployed to multiple clients to implement model training with the client and server. The training strategy can be that before training begins, the server first decomposes and compresses the full-scale LLM. Client Model (Compression Simulator) initialized by knowledge distillation on public datasets such as Alpaca.

[0059] Step 103: Acquire adapter parameters uploaded by each client after training the adapter based on the local data set.

[0060] The client uploads the updated adapter parameters W after local training ALoRA (Including W corresponding to personalized adapter parameters ApLoRA W corresponding to the general adapter parameters AgLoRA ) to the server. The server aggregates these parameters and uses and aggregated W ALoRA Perform distillation, guided by the full large language model. This process updates the compression parameters to better approximate the original model performance while reducing the gap caused by compression.

[0061] The distillation process can be strategically updated to maintain W ALoRA Freezing,achieves two key goals: (1) preserving the performance characteristics of the,full model; (2) maintaining adaptability to the client’s specific data,distribution.

[0062] Step 104 : performing global aggregation on the common adapter parameters in the adapter parameters, and grouping and aggregating the personalized adapter parameters in the adapter parameters to obtain aggregated adapter parameters.

[0063] In federated learning, server-side parameter aggregation faces the challenge of client-side data heterogeneity, which limits the effectiveness of traditional averaging methods for personalization.

[0064] The present invention solves this problem through a context grouping mechanism based on similarity clustering clients. The method preserves personalization and global generalization by processing adapter parameters separately: Ageneral captures shared features while Apersonal preserves context information.

[0065] Optionally, in step 104, grouping and aggregating the personalized adapter parameters in the adapter parameters includes: grouping the clients according to context similarity; The personalized adapter parameters are clustered according to the grouping of the clients, and the clustered personalized adapter parameters are aggregated within the group.

[0066] Before aggregation, the server uses the uploaded W Ap The similarity analysis of the parameters is used to adaptively group the clients. These parameters are first mapped to dense vectors to construct a search table. Ap The parameters are based on the similarity S i,j Clustering, using the elbow method and silhouette score to determine the optimal number of clusters K for each round of aggregation:

[0067]

[0068] During the polymerization process, W Ag Perform global aggregation, W Ap Aggregate within each cluster (i.e., aggregate within groups):

[0069]

[0070] Finally, the server returns the updated parameters to the client for the next round of training:

[0071] Among them, Sij is the cosine similarity between clients i and j, which is used to measure their personalization parameters W ApLoRA similarity; is the LoRA parameter of Apersonal uploaded by the i-th client; N is the total number of clients participating in federated learning; K is the number of clusters finally selected, determined by maximizing the elbow method (Elbow) and the silhouette coefficient (Silhouette); Elbow(k) is the elbow method function used to evaluate the change of clustering error with k (usually the k corresponding to the inflection point is selected); Silhouette(k) is the silhouette coefficient function, which measures the compactness and separation of the clustering results (the closer the value is to 1, the better the clustering effect). It can dynamically determine the number of groups K in each aggregation round to avoid deviations caused by fixed groupings; It is the global Ageneral LoRA parameter after the t+1th round update, shared by all clients; is the Ageneral parameter of the i-th client in the t-th round; N is the total number of clients, and the global parameters of all clients can be averaged to achieve global knowledge sharing; is the LoRA parameter of Apersonal after the aggregation of the kth cluster in the t+1th round, which is only returned to the clients in the cluster; C k is the client index set contained in the k-th cluster; |C k ∣ is the number of clients in the kth cluster; is the personalized parameter of the i-th client in the t-th round. The LoRA parameters of the clients Apersonal in the same cluster can be averaged to preserve the similarity within the group while distinguishing the personalized features of different clusters; It means that the server returns the parameter sets of all K clusters, and the client receives the corresponding personalized parameters according to the cluster it belongs to, realizing the hybrid update strategy of "global sharing plus in-group personalization".

[0072] Step 105: Output the aggregated adapter parameters to the client.

[0073] The above process, combined with Figure 2 As shown, in order to achieve efficient and personalized LLM-based federated recommendation, the embodiment of the present application proposes PFedLRec. The overview of PFedLRec is as follows Figure 2 As shown in Figure 3, the proposed method consists of two main phases: a global phase of collaborative training and a post-training phase after federated learning. The first phase aims to preserve old knowledge while achieving generalization between groups and coarse-grained personalization. The second phase focuses on fine-grained personalization, enhancing user modeling tailored to each local need.

[0074] Combine Figure 3 As shown, the specific steps involved in the implementation of each part are broken down in detail: Step 1: Simulator compression and alignment. Efficient federated recommendation is achieved by compressing LLMs while maintaining their performance through alignment. Step 2: Federated training with contextual grouping. Utilize adapter components for multi-granularity aggregation, balancing user similarity and global generalization. Step 3: Fine-grained user modeling. The application retrieves and accesses relevant local records to improve LLM personalization and user preference modeling.

[0075] In an embodiment of the present application, a large language model is decomposed to obtain an adapter and a simulator, the simulator is layer-compressed to obtain a compressed simulator, the adapter and the compressed simulator are deployed to multiple clients, the adapter parameters uploaded by each client after training the adapter based on a local data set are obtained, global aggregation is performed on the general adapter parameters in the adapter parameters, the personalized adapter parameters in the adapter parameters are grouped and aggregated to obtain aggregated adapter parameters, and the aggregated adapter parameters are output to the client, so that the client only stores and updates the lightweight proxy model (adapter and compressed simulator), isolates the original large language model, and does not need to transmit the original large language model parameters or user sensitive data, fundamentally avoiding the leakage of the original large language model and the exposure of user data, achieving privacy protection, and being able to obtain the adapter parameters uploaded by each client based on local data, perform global aggregation on the general adapter parameters in the adapter parameters, group and aggregate the personalized adapter parameters in the adapter parameters, retain the personalized parameters of each client, and retain individual user preferences.

[0076] The embodiments of this application also provide different implementation methods of the data processing method of a large language model based on federated learning.

[0077] See also Figure 4 , Figure 4 This is the process of the data processing method of the large language model based on federated learning provided in the embodiment of the application Figure 2 .like Figure 4 As shown in FIG, a data processing method for a large language model based on federated learning is applied to a client. The method includes the following steps: Step 401: Obtain the adapter and compression simulator sent by the server, which are obtained by decomposing the large language model.

[0078] Step 402: Train the adapter based on the local data set to obtain updated adapter parameters, and upload the adapter parameters to the server.

[0079] Optionally, the training the adapter based on the local data set to obtain updated adapter parameters includes: freezing the compression simulator; The adapter is trained based on the local data set to obtain updated adapter parameters.

[0080] Client deployment lightweight model M=[Ag, , Ap], Ag (Ageneral) and Ap (Apersonal) form the adapter A, Represents a compressed simulator, which is maintained during local training Frozen. Adapter only W ALoRA Update (see formula below):

[0081] Where [M] is the total number of clients participating in federated learning, and m represents the mth client; represents the client-specific loss function, is the corresponding loss weight, are adapter parameters, is a simulator parameter; λ is a hyperparameter of the regularization term, which controls the strength of the parameter deviation constraint; is the LoRA adapter parameter vector to be optimized; is the LoRA parameter initialized at the beginning of the tth round of federated learning; It is the square of the L2 norm, which is used to measure the deviation of the parameters from the initial value.

[0082] The regularization term limits the adapter parameters from deviating from the initial values ​​in each round, alleviating the performance divergence caused by local client adaptation.

[0083] This design reduces storage and computation costs for resource-constrained clients.

[0084] Step 403: Acquire the aggregated adapter parameters sent by the server.

[0085] The aggregated adapter parameters are obtained after the server performs global aggregation based on the common adapter parameters in the adapter parameters and performs group aggregation based on the personalized adapter parameters in the adapter parameters.

[0086] Step 404: Update the adapter according to the aggregated adapter parameters, and iterate the above process until the large language model converges.

[0087] The iterative execution of the above process may specifically refer to executing steps 402 to 403 and updating the adapter according to the aggregated adapter parameters to perform cyclic iterative training of the large language model until the large language model converges, thereby enabling adjustment of the large language model locally on the client.

[0088] In one example, a recommendation system is formed by using a large language model to recommend information. After the large language model converges, it can generate and output recommendation results based on the text input by the user.

[0089] In implementation, each client uses the updated Start new local training and iteratively optimize W ALoRA Until convergence. This ensures global model consistency in the federated framework while maintaining local adaptability.

[0090] After receiving the global model, each client uses retrieval-enhanced hints together with the global model to perform local fine-grained user modeling, enhancing client-level personalization.

[0091] To implement retrieval enhancement prompts for local user modeling, the method further includes: Retrieve the top M items similar to historical interaction data from the local knowledge base; M ≥ 2; Constructing enhanced prompt information based on the first M items; The enhanced prompt information is encoded into a persistent memory vector, and the persistent memory vector is injected into the prompt word template of the converged large language model to obtain the adjusted large language model.

[0092] The first M items refer to historical interaction items, such as historical browsing data items, historical information received data items, and historical information output data items, such as historical movie browsing, historical song playback, and historical outgoing voice calls. Based on the first M items, item information can be extracted to construct enhanced prompt information.

[0093] Enhanced prompt information is personalized prompt word information that is more in line with the current client habits and is added to the prompt words of the large language model.

[0094] For each client, retrieval enhancement prompts based on local user modeling are performed after federated learning is completed.

[0095] Global model training uses user item interactions to protect privacy, while local modeling incorporates richer personal data for better personalization. However, the LLM context length limit constrains the length of action sequences and the candidate items included in prompts, reducing recommendation performance. This embodiment of the application addresses this issue through retrieval-enhanced two-stage prompt tuning.

[0096] Phase 1: Search Enhancement Prompt Design. This embodiment of the application builds local projects and user profiles to form a knowledge base. This achieves personalized enhancement of prompt words, enabling the large language model to better integrate user personalized information to implement information recommendations.

[0097] Table 1: Local projects

[0098] Table 2: User Profile

[0099] For each user, the BM25 retrieval algorithm can be used to retrieve the top K similar items of their top N interactive items from the knowledge base. Recommendation prompt words are constructed based on these retrieved items.

[0100] Instructions: Consider user 1, who has interacted with ML100K items, where each item is represented by item_id(score=<item_id>, genre=<item_id>, similar items=<item_id>(score)). The score represents the user's preference for the item, ranging from 0 (dislike) to 5 (very much like). The genre represents the category to which the item belongs. Similar items represents the three most similar items. Based on this information, recommend items to the user.

[0101] Input: User 1 has interacted with item 1036 (rating = 4, type = action, adventure, similar items = ['item 1018 (4)', 'item 1181 (3)', 'item 1011 (4)'])....

[0102] Output: Item 1181.

[0103] Phase 2: Personalized Prompts for Memory Enhancement. Given the relatively structured and static attributes of user profiles, this embodiment parameterizes the structured user profiles into persistent memory, enabling direct integration into training. This memory is injected into the prompt words in Phase 1, allowing for full utilization of user preferences while alleviating context length constraints.

[0104] In the above implementation, the overall learning process is divided into a global phase of collaborative training and a local phase after training. In the first phase, a local compressed proxy model deployment is introduced to prevent leakage of the original LLM, thereby ensuring privacy protection. In addition, a two-level aggregation strategy during collaborative training aligns the proxy with the original model, preserving its capabilities while achieving universal representation and coarse-grained personalization. In the second phase, embodiments of the present application enhance fine-grained personalization capabilities by enhancing prompts to retrieve additional relevant historical information and user profiles.

[0105] To demonstrate the effectiveness of PFedLRec, the present invention conducts extensive experiments on multiple recommended benchmark datasets. Extensive experiments on the benchmark datasets demonstrate the effectiveness of PFedLRec.

[0106] Experimental results show that PFedLRec can significantly improve the performance of LLMRec while maintaining its privacy and enhancing personalization. The main contributions of the embodiments of this application are summarized as follows: PFedLRec implements efficient LLM-based federated recommendation with enhanced personalization while leveraging the contextual reasoning capabilities of LLM and maintaining privacy and efficiency.

[0107] The framework adopts two steps: federated learning and client-side local post-training, and uses a lightweight client agent to achieve personalized recommendations.

[0108] Extensive evaluations on multiple datasets demonstrate that PFedLRec achieves consistent improvements in recommendation accuracy and computational efficiency under different settings.

[0109] The specific experimental and in-depth analysis process includes the following: 1. Ablation study.

[0110] To analyze the impact of each component in PFedLRec, this embodiment removes the grouping process and local user modeling, respectively, referred to as "without GP," "without PT," and "without GP & PT." The results show that: (1) in LLM-based federated recommendation, both grouping and prompting are crucial for optimizing performance; (2) prompting contributes more significantly because it leverages local personal data to enhance personalization.

[0111] 2. Adaptive grouping effect.

[0112] This example demonstrates that the optimal number of clusters, K, varies between rounds, reflecting dynamic user behavior patterns. This leads to the use of an adaptive clustering approach in this example, which outperforms fixed clustering (K=2, 3) in both HR@10 and NDCG@10 metrics. The results confirm that dynamic clustering better captures changes in data distribution, resulting in superior recommendation performance.

[0113] The present invention demonstrates that PFedLRec significantly reduces the number of trainable parameters, GPU memory, and computational cost while maintaining acceptable accuracy, enabling resource-constrained clients to deploy LLM-based recommendations. The results confirm that PFedLRec successfully balances the performance and efficiency of federated LLM deployments.

[0114] This embodiment enhances the privacy protection of PFedLRec by incorporating local differential privacy, setting the noise intensity δ to [0, 0.1, 0.2, 0.3, 0.4, 0.5]. As the noise intensity δ increases, performance degrades. To achieve a trade-off between recommendation accuracy and privacy protection, this embodiment can set δ to 0.2, which still outperforms most baselines, achieving PFedLRec with enhanced privacy protection.

[0115] This embodiment of the present application proposes PFedLRec, a highly efficient framework for LLM-based recommendations in federated learning. By strategically adjusting the LLM architecture, PFedLRec enables scalable deployment on resource-constrained clients. This embodiment of the present application utilizes a two-stage training approach to optimize performance: (1) two-level aggregation balances personalization and global generalization; (2) retrieval-enhanced prompts leverage local data for fine-grained personalization. Extensive experiments demonstrate superior performance, and privacy-preserving integration confirms robustness.

[0116] See also Figure 5 , Figure 5 This is the structure of a data processing device based on a large language model of federated learning provided in the embodiment of the present application. Figure 1 For ease of explanation, only the parts related to the embodiment of the present application are shown. The data processing device of the large language model of the federated learning can be optionally a server.

[0117] The data processing device 500 of the large language model based on federated learning includes: Decomposition module 501, used to decompose the large language model to obtain adapters and simulators; A compression module 502 is configured to perform layer compression processing on the simulator to obtain a compressed simulator, and deploy the adapter and the compressed simulator to multiple clients; A first acquisition module 503 is configured to acquire adapter parameters uploaded by each client after training the adapter based on a local data set; an aggregation module 504 for performing global aggregation on the common adapter parameters in the adapter parameters, and grouping and aggregating the personalized adapter parameters in the adapter parameters to obtain aggregated adapter parameters; The output module 505 is configured to output the aggregated adapter parameters to the client.

[0118] Optionally, the decomposition module 501 is specifically configured to: The large language model is decomposed into an adapter formed by a set number of layers close to the input and output, and a simulator formed by the remaining parts.

[0119] Optionally, the compression module 502 is specifically configured to: The layers in the simulator are evenly deleted according to a set ratio to obtain the compression simulator.

[0120] Optionally, the aggregation module 504 is specifically configured to: grouping the clients according to context similarity; The personalized adapter parameters are clustered according to the grouping of the clients, and the clustered personalized adapter parameters are aggregated within the group.

[0121] Optionally, the compression module 502 is further configured to: Knowledge distillation is performed on the compression simulator, and the output of the compression simulator after knowledge distillation is aligned with the uncompressed simulator in the large language model.

[0122] The data processing device for a large language model based on federated learning provided in the embodiment of the present application can implement the various processes of the embodiment of the data processing method for a large language model based on federated learning, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0123] See also Figure 6 , Figure 6 This is the structure of a data processing device based on a large language model of federated learning provided in the embodiment of the present application. Figure 2 For ease of explanation, only the parts related to the embodiment of the present application are shown. The data processing device of the large language model of the federated learning can be selected as the client.

[0124] The data processing device 600 of a large language model based on federated learning includes: The second acquisition module 601 is used to acquire the adapter and compression simulator obtained by decomposing the large language model sent by the server; A first updating module 602 is configured to train the adapter based on a local data set to obtain updated adapter parameters and upload the adapter parameters to the server; A third acquisition module 603 is configured to acquire aggregated adapter parameters sent by the server; the aggregated adapter parameters are obtained by the server performing global aggregation based on the general adapter parameters in the adapter parameters and group aggregation based on the personalized adapter parameters in the adapter parameters; The second updating module 604 is configured to update the adapter according to the aggregated adapter parameters and iteratively execute the above process until the large language model converges.

[0125] Optionally, the device further comprises: The fine-tuning module is configured to retrieve the top M items similar to the historical interaction data from the local knowledge base, where M is greater than or equal to 2; construct enhanced prompt information based on the top M items; encode the enhanced prompt information into a persistent memory vector, and inject the persistent memory vector into the prompt word template of the converged large language model to obtain the adjusted large language model.

[0126] Optionally, the first updating module 602 is specifically configured to: freezing the compression simulator; The adapter is trained based on the local data set to obtain updated adapter parameters.

[0127] The data processing device for a large language model based on federated learning provided in the embodiment of the present application can implement the various processes of the embodiment of the data processing method for a large language model based on federated learning, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0128] Figure 7 is a structural diagram of a computer device provided in an embodiment of the present application. As shown in the figure, the computer device 7 of this embodiment includes: at least one processor 70 ( Figure 7 Only one is shown in the figure), a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, wherein the processor 70 implements the steps of any of the above-mentioned method embodiments when executing the computer program 72.

[0129] The computer device 7 may be a desktop computer, a notebook computer, a PDA, a cloud server or other computing devices. The computer device 7 may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that Figure 7 It is only an example of the computer device 7 and does not constitute a limitation of the computer device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0130] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0131] The memory 71 can be an internal storage unit of the computer device 7, such as the hard drive or memory of the computer device 7. The memory 71 can also be an external storage device of the computer device 7, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the computer device 7. Furthermore, the memory 71 can include both the internal storage unit of the computer device 7 and an external storage device. The memory 71 is used to store the computer program and other programs and data required by the computer device. The memory 71 can also be used to temporarily store data that has been output or is about to be output.

[0132] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0133] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0134] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of the apparatus or unit, which can be electrical, mechanical or other forms.

[0136] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0137] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0138] If the integrated module / unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the process steps in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content of the computer-readable medium can be appropriately increased or decreased based on the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0139] The present application implements all or part of the processes in the above-mentioned embodiment methods, and may also be implemented through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0140] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A data processing method for a large language model based on federated learning, applied to a server, characterized in that: include: Decompose the large language model to obtain adapters and simulators; Performing layer compression processing on the simulator to obtain a compressed simulator, and deploying the adapter and the compressed simulator to multiple clients; Obtaining adapter parameters uploaded by each client after training the adapter based on a local data set; performing global aggregation on the common adapter parameters in the adapter parameters, grouping and aggregating the personalized adapter parameters in the adapter parameters to obtain aggregated adapter parameters; The aggregated adapter parameters are output to the client.

2. The method according to claim 1, characterized in that Decomposing the large language model to obtain the adapter and simulator includes: The large language model is decomposed into an adapter formed by a set number of layers close to the input and output, and a simulator formed by the remaining parts.

3. The method according to claim 1, characterized in that The layer-compression processing is performed on the simulator to obtain a compressed simulator, comprising: The layers in the simulator are evenly deleted according to a set ratio to obtain the compression simulator.

4. The method according to claim 1, wherein The grouping and aggregating the personalized adapter parameters in the adapter parameters includes: grouping the clients according to context similarity; The personalized adapter parameters are clustered according to the grouping of the clients, and the clustered personalized adapter parameters are aggregated within the group.

5. The method according to claim 1, wherein Before deploying the adapter and the compression simulator to multiple clients, the method further includes: Knowledge distillation is performed on the compression simulator, and the output of the compression simulator after knowledge distillation is aligned with the uncompressed simulator in the large language model.

6. A data processing method for a large language model based on federated learning, applied to a client, characterized in that: include: Obtaining the adapter and compression simulator obtained by decomposing the large language model sent by the server; Training the adapter based on the local data set to obtain updated adapter parameters, and uploading the adapter parameters to the server; Obtaining the aggregated adapter parameters sent by the server; The aggregated adapter parameters are obtained by the server performing global aggregation based on the general adapter parameters in the adapter parameters and grouping aggregation based on the personalized adapter parameters in the adapter parameters; The adapter is updated according to the aggregated adapter parameters until the large language model converges.

7. The method according to claim 6, characterized in that The method further comprises: Retrieve the top M items similar to historical interaction data from the local knowledge base; M ≥ 1; Constructing enhanced prompt information based on the first M items; The enhanced prompt information is encoded into a persistent memory vector, and the persistent memory vector is injected into the prompt word template of the converged large language model to obtain the adjusted large language model.

8. The method according to claim 6, characterized in that The step of training the adapter based on the local data set to obtain updated adapter parameters includes: freezing the compression simulator; The adapter is trained based on the local data set to obtain updated adapter parameters.

9. A data processing device for a large language model based on federated learning, characterized in that: include: Decomposition module, used to decompose the large language model to obtain adapters and simulators; A compression module, configured to perform layer compression processing on the simulator to obtain a compressed simulator, and deploy the adapter and the compressed simulator to multiple clients; A first acquisition module is used to obtain adapter parameters uploaded by each client after training the adapter based on a local data set; an aggregation module, configured to perform global aggregation on the common adapter parameters in the adapter parameters, and group and aggregate the personalized adapter parameters in the adapter parameters to obtain aggregated adapter parameters; An output module is used to output the aggregated adapter parameters to the client.

10. A data processing device for a large language model based on federated learning, characterized in that: include: The second acquisition module is used to obtain the adapter and compression simulator obtained by decomposing the large language model sent by the server; A first updating module is configured to train the adapter based on a local data set to obtain updated adapter parameters and upload the adapter parameters to the server; A third acquisition module, configured to acquire the aggregated adapter parameters sent by the server; The aggregated adapter parameters are obtained by the server performing global aggregation based on the general adapter parameters in the adapter parameters and grouping aggregation based on the personalized adapter parameters in the adapter parameters; The second updating module is configured to update the adapter according to the aggregated adapter parameters until the large language model converges.

Images