AIGC field customization framework based on edge collaborative fine tuning

Abstract

The invention relates to the technical field of computer artificial intelligence, in particular to an AIGC field customization framework based on edge collaborative fine tuning, which comprises a cloud-driven AIGC layer, an edge network layer and a terminal user layer, the cloud-driven AIGC layer carries out pre-training on AIGC, the edge network layer carries out fine tuning on the pre-trained AIGC, and the terminal user layer carries out fine tuning on the pre-trained AIGC. The terminal user layer continues to finely tune the AIGC associated with the personal preference from the user Internet of Things device; the edge network layer performs fine adjustment on the pre-trained AIGC model by using industry-specific data; the method has the advantages that the lightweight fine tuning technology is adopted to promote edge nodes to participate in cooperative fine tuning, how different scales of models execute distributed mutual learning on the Gossip network is mainly introduced, and the method has the advantages that the user personalized AIGC model is finely tuned by using the individual Internet of Things equipment of the user; different edge devices are promoted to be added into the fine adjustment process of the pre-trained AIGC model, and meanwhile privacy and fairness of all parties are guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of computer artificial intelligence, and specifically to an AIGC field customized framework based on edge collaborative fine-tuning. Background Art

[0002] Large Language Model (LLM) can generate high-quality text by training with massive data, such as natural language generation, machine translation, and text summarization. However, in order to improve the performance and accuracy of the model, LLM training requires a lot of computing resources, such as GPU and TPU. Specifically, LLM models usually need to use more than 10 billion parameters and perform millions of training iterations on large-scale data sets, which consumes thousands of hours of computing time and a lot of storage space. Therefore, although the LLM model has made significant breakthroughs in the field of Artificial Intelligent Generative Content (AIGC), it has also become one of the representatives of computing resource-intensive tasks, and the demand for computing resources is huge;

[0003] With the popularity of artificial intelligence generated content (AIGC), the number of users has exploded, which has brought unprecedented demand for preference diversity. Users' demand for personalized and professional content generation is also increasing. However, using open source LLM models to fine-tune to meet the needs of one's own professional field often faces some challenges. First, fine-tuning an open source LLM model requires a lot of computing resources. For ordinary users, it is difficult to have enough computing resources to support such a fine-tuning process. Traditional client devices often cannot meet the needs of LLM fine-tuning, which makes it very difficult for users to perform professional fine-tuning independently. Secondly, the adaptability and generalization ability of open source LLM models are also a problem. Although LLM has been trained on large-scale data sets, these training data may not be applicable to the user's professional field. An open source model still needs to be fine-tuned to meet the needs of a specific field, which requires domain expertise and data. There will be problems such as missing data sets and reduced original model capabilities;

[0004] Federated Learning is a distributed machine learning method that aims to improve the performance of the global model by training models on multiple local devices or edge devices to achieve data privacy and security. In traditional machine learning methods, it is usually necessary to centralize the data set to a central server for model training. However, this centralized approach has some problems. First, the centralized storage of large-scale data may involve privacy issues, especially for personal data containing sensitive information. Second, transferring data to a central server may also bring problems such as network latency and bandwidth consumption. Federated learning pushes model training to local devices, so that data is always kept locally and local models are trained on the devices. Local devices can be smartphones, sensors, edge servers, etc. Each device uses local data to train the model, and then transmits only the updated model parameters back to the central server. The central server integrates the received parameter updates to generate a new global model, which is distributed to each device again for the next round of training. This process is repeated until the global model reaches the predetermined performance indicators. The advantages of federated learning are mainly reflected in data privacy and security. Since the data is always kept on the local device and not transmitted to the central server, the user's privacy can be effectively protected, which is very beneficial for some institutions with privacy protection needs, such as hospitals, banks, etc. In addition, federated learning can also reduce network communication overhead and speed up model training; at the same time, federated learning also has a certain degree of fault tolerance, even if some devices are offline or interrupted, the entire system can still operate normally;

[0005] Therefore, using federated learning to perform edge collaborative training of LLM is a very meaningful solution, but there is still a problem of insufficient training resources. Because for federated learning, each edge device still needs to deploy a complete LLM for fine-tuning, which will result in a lot of computing overhead, so a high-performance fine-tuning method is essential.

[0006] Currently, methods for fine-tuning LLM on smaller, domain-specific datasets can be roughly divided into two categories: adapter-based fine-tuning and task-oriented fine-tuning. Adapter-based fine-tuning uses neural adapters or modular components to enhance the performance of LLM on domain-specific tasks without major modifications to the internal parameters of LLM. These adapters are usually integrated into existing LLM architectures, allowing task-specific learning while keeping the original model basically intact; task-oriented fine-tuning focuses on modifying the internal parameters of LLM to improve alignment with specific tasks. However, due to hardware limitations and potential performance degradation, it may be impractical to fully update all parameters of LLM; therefore, the challenge facing researchers is to determine which parameters need to be changed in the huge parameter space, or to efficiently update a portion of them; these two methods enable LLM to be customized for specific tasks or domains, providing flexibility and efficiency to handle specialized applications.

[0007] For edge collaborative fine-tuning, since the data of each client is mostly heterogeneous and updating the parameters of the same LLM may lead to problems such as context learning degradation and catastrophic forgetting, it is better to choose parameter-efficient adapter fine-tuning, which aims to add a small number of additional parameters to the LLM to achieve better performance in specific tasks. Usually, these additional parameters are encoded in simple modules to guide the adaptation of the language model to the target domain or task. The advantages of adding modules include: (1) small number of parameters and simplicity; (2) scalability to the original language model; (3) flexibility in sequential training on each specific domain. Summary of the invention

[0008] In order to solve the above technical problems, the technical solution provided by the present invention is: a customized AIGC field framework based on edge collaborative fine-tuning, including a cloud-driven AIGC layer, an edge network layer, and an end-user layer, characterized in that:

[0009] Cloud-driven AIGC layer: The cloud-driven AIGC layer refers to pre-training a large AIGC model for content generation tasks through companies with powerful computing resources such as Amazon and Meta, or implementing a basic general content generation AIGC model based on an open source large model;

[0010] Edge network layer: Fine-tune the pre-trained AIGC model in a specialized way at the edge layer. Relying on the unique data of different industries, fine-tune the model according to the professional needs of the industry to achieve an industry-customized AIGC model.

[0011] End-user layer: Continue to fine-tune the AIGC model in fine granularity, and fine-tune the AIGC model based on the collected personal Internet devices of users to personalize the preferences of users, and finally realize the user's personal AIGC model;

[0012] The cloud-driven AIGC layer pre-trains AIGC, the edge network layer fine-tunes the pre-trained AIGC, and the end-user layer continues to fine-tune the AIGC associated with the personal preferences from the user's IoT device; the AIGC is first trained with computing power provided by major server manufacturers, and then provides content generation services; the edge network layer uses industry-proprietary data to fine-tune the pre-trained AIGC model to achieve industry specialization in vertical fields; the end-user layer uses the user's personal IoT device to fine-tune the user's personalized AIGC model.

[0013] As an improvement, the edge-driven collaborative fine-tuning performs the following steps:

[0014] Step 1: Considering the heterogeneous resources of candidate edge devices, IoT devices are divided into subgroups, and an initial model of adaptive scale is deployed. Then, local training is performed based on the original data to iteratively optimize the local model. Deep Mutual Learning (DML), as an online knowledge distillation algorithm, will be deployed in the edge network for knowledge exchange between edge devices, and a series of machine learning algorithms, including stochastic gradient descent and batch gradient descent, can be used to update the local model by minimizing the loss function between the label value and the inference result.

[0015] Step 2: Considering that edge devices usually have more limited computing and communication resources than data center servers, however, in federated learning, local model parameter sharing and transmission requires a lot of communication bandwidth and resource consumption;

[0016] Step 3: The edge server receives updated parameters from terminal devices dispersed in different subgroups through the edge network; the edge server aggregates the collected parameters and forms a global model in the current communication round after parameter alignment;

[0017] As an improvement, the AIGC model operating system performs the following steps:

[0018] Step 1: The model information published on the public distributed ledger increases the risk of user privacy leakage; therefore, the present invention further enhances privacy protection by adding Laplace noise Lap(1 / ∈) to each dimension of the gradient, which is called ∈-differential privacy (DP); the present invention ensures that it is difficult to infer private information from the shared model according to ∈-differential privacy, which can prevent the adversary from inferring the information of the prover P' from the blockchain;

[0019] Step 2: In order to verify the quality of fine-tuning, the present invention proposes a peer-to-peer sub-Gossip consensus verification scheme without the need for a centralized coordinator; the clients participating in the AIGC fine-tuning task are defined as two groups, namely, prover P and verifier V; first, P updates the model parameters and information as evidence of AIGC, which can be used to prove the contribution to collaborative fine-tuning. Then, the global model needs to be verified by consensus algorithms such as Proof of Work (PoW) and Proof of Stake (PoS) to verify the quality of AIGC. In this process, participants in P randomly select a sub-Gossip network S from V to prevent differential attacks against updates, and only broadcast their local models to participants in the subset S, verifying the model quality through a public data set. As a consensus node in the blockchain, V can evaluate the performance of AIGC and broadcast it to the edge network; if successfully verified, some of the fine-tuned model parameters can also be used as part of the base model for further fine-tuning;

[0020] Step 3: In the system model, the edge server recruits heterogeneous terminal clients to perform collaborative fine-tuning through federated learning; the interaction between the edge server and the terminal clients is modeled.

[0021] As an improvement, the edge-driven collaborative fine-tuning execution step three specifically includes: the aggregated parameters are only a small part of the local model parameters, most of which are frozen to reduce the communication budget in the edge network; after several aggregations, the global model achieves the expected inference accuracy and model iteration termination.

[0022] The advantages of the present invention compared with the prior art are:

[0023] 1. The present invention adopts lightweight fine-tuning technology to promote edge nodes to participate in collaborative fine-tuning, overcoming the fine-tuning bottleneck caused by limited computing resources;

[0024] Second, the privacy-enhanced collaborative fine-tuning scheme based on federated learning in this invention focuses on how models of different scales can perform distributed mutual learning on the Gossip network, solving the problem that the training progress is difficult to synchronize and the training effect is difficult to guarantee after the computing power resources are dispersed;

[0025] 3. This invention provides a tutorial overview of the verifiable model consensus and incentive design based on blockchain, and invents a practical and reliable incentive mechanism to promote different edge devices to join in the fine-tuning process of the pre-trained AIGC model, while ensuring the privacy and fairness of all parties, and realizing an effective new method for the specialization of AIGC in multiple fields and industries, laying a solid foundation for promoting the widespread use and safe and reliable use of AIGC models. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1The overall framework diagram of the end-edge cloud customized for the AIGC service of the present invention;

[0027] Figure 2 The invention implements edge-driven collaborative fine-tuning graphs for AIGC hierarchical deployment by federated learning;

[0028] Figure 3 A workflow diagram for parameter verification and incentive design of the AIGC model of the present invention; DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] The present application fully explains the problems of the prior art in the technical background, and there are no superordinate or general problems. The problems to be solved by the present application are described in detail in the following invention content and specific implementation methods, the technical problems to be solved by the technical scheme are clarified, and it is determined that the technical scheme of the present application has beneficial effects relative to the objective prior art.

[0031] See also Figure 1-Figure 3 , an embodiment provided by the present invention:

[0032] A customized AIGC framework based on edge collaborative fine-tuning, including cloud-driven AIGC layer, edge network layer, and end-user layer.

[0033] Cloud-driven AIGC layer: The cloud-driven AIGC layer refers to pre-training a large AIGC model for content generation tasks through companies with powerful computing resources such as Amazon and Meta, or implementing a basic general content generation AIGC model based on an open source large model;

[0034] Edge network layer: Fine-tune the pre-trained AIGC model in a specialized way at the edge layer. Relying on the unique data of different industries, fine-tune the model according to the professional needs of the industry to achieve an industry-customized AIGC model.

[0035] End-user layer: Continue to fine-tune the AIGC model in fine granularity, and fine-tune the AIGC model based on the collected personal Internet devices of users to personalize the preferences of users, and finally realize the user's personal AIGC model;

[0036] The cloud-driven AIGC layer pre-trains AIGC, the edge network layer fine-tunes the pre-trained AIGC, and the end-user layer continues to fine-tune the AIGC associated with the personal preferences from the user's IoT device; the AIGC is first trained with computing power provided by major server manufacturers, and then provides content generation services; the edge network layer uses industry-proprietary data to fine-tune the pre-trained AIGC model to achieve industry specialization in vertical fields; the end-user layer uses the user's personal IoT device to fine-tune the user's personalized AIGC model.

[0037] Federated learning further explains how AIGC layered deployment implements edge-driven collaborative fine-tuning:

[0038] Step 1: Considering the heterogeneous resources of candidate edge devices, IoT devices are divided into subgroups, and an initial model of adaptive scale is deployed. Then, local training is performed based on the original data to iteratively optimize the local model. Deep Mutual Learning (DML), as an online knowledge distillation algorithm, will be deployed in the edge network for knowledge exchange between edge devices, and a series of machine learning algorithms, including stochastic gradient descent and batch gradient descent, can be used to update the local model by minimizing the loss function between the label value and the inference result.

[0039] Step 2: Considering that edge devices usually have more limited computing and communication resources than data center servers, however, in federated learning, local model parameter sharing and transmission requires a lot of communication bandwidth and resource consumption; to address this challenge, the present invention adopts a personalized pruning method based on user prompts, considering the importance of gradient dimensions, and achieving efficient parameter updates. This method significantly reduces the size of the shared model during federated learning, thereby minimizing communication overhead and overall training time, while maintaining the same level of accuracy as full model migration;

[0040] Step 3: The edge server receives updated parameters from terminal devices scattered in different subgroups through the edge network; the edge server aggregates the collected parameters and forms a global model in the current communication round after parameter alignment. It is worth noting that the aggregated parameters are only a small part of the local model parameters, most of which are frozen to reduce the communication budget in the edge network. After several aggregations, the global model achieves the expected inference accuracy and model iteration termination;

[0041] The AIGC model parameter verification and stimulus design workflow overview is further explained in detail:

[0042] Step 1: The model information published on the public distributed ledger increases the risk of user privacy leakage; therefore, the present invention further enhances privacy protection by adding Laplace noise Lap(1 / ∈) to each dimension of the gradient, which is called ∈-differential privacy (DP); the present invention ensures that it is difficult to infer private information from the shared model according to ∈-differential privacy, which can prevent the adversary from inferring the information of the prover P' from the blockchain;

[0043] Step 2: In order to verify the quality of fine-tuning, the present invention proposes a peer-to-peer sub-Gossip consensus verification scheme without the need for a centralized coordinator; in the scheme of the present invention, the clients participating in the AIGC fine-tuning task are defined as two groups, namely, the prover P and the verifier V; first, P updates the model parameters and information as evidence of AIGC, which can be used to prove the contribution to collaborative fine-tuning; then, the global model needs to be verified through consensus algorithms such as proof of work (PoW) and proof of stake (PoS) to verify the quality of AIGC; in this process, participants in P randomly select a sub-Gossip network S from V to prevent differential attacks against updates, and only broadcast their local models to participants in the subset S, verifying the model quality through a public data set; as a consensus node in the blockchain, V can evaluate the performance of AIGC and broadcast it to the edge network. If successfully verified, some of the fine-tuned model parameters can also be used as part of the base model for further fine-tuning;

[0044] Step 3: In the system model of the present invention, the edge server recruits heterogeneous terminal clients to perform collaborative fine-tuning through federated learning; the interaction between the edge server and the terminal client is modeled, aiming to use reinforcement learning to establish a fair incentive structure for payment and reward distribution; the present invention trains a global model to minimize empirical risk under the communication budget constraint; all IoT devices associated with a specific edge network can become candidates for participating in the fine-tuning task, and the edge server selects a high-quality local model based on the AIGC's proof;

[0045] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope of the present application is indicated by the claims.

[0046] It should be understood that the present application is not limited to the precise structure described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The above-described embodiments of the present application do not constitute a limitation on the scope of protection of the present application.

Claims

1. A customized AIGC field framework based on edge collaborative fine-tuning, including a cloud-driven AIGC layer, an edge network layer, and an end-user layer, characterized in that: Cloud-driven AIGC layer: The cloud-driven AIGC layer refers to pre-training a large AIGC model for content generation tasks through companies with powerful computing resources such as Amazon and Meta, or implementing a basic general content generation AIGC model based on an open source large model; Edge network layer: Fine-tune the pre-trained AIGC model in a specialized way at the edge layer. Relying on the unique data of different industries, fine-tune the model according to the professional needs of the industry to achieve an industry-customized AIGC model. End-user layer: Continue to fine-tune the AIGC model in fine granularity, and fine-tune the AIGC model based on the collected personal Internet devices of users to personalize the preferences of users, and finally realize the user's personal AIGC model; The cloud-driven AIGC layer pre-trains AIGC, the edge network layer fine-tunes the pre-trained AIGC, and the end-user layer continues to fine-tune the AIGC associated with the personal preferences from the user's IoT device; the AIGC is first trained with computing power provided by major server manufacturers, and then provides content generation services; the edge network layer uses industry-proprietary data to fine-tune the pre-trained AIGC model to achieve industry specialization in vertical fields; the end-user layer uses the user's personal IoT device to fine-tune the user's personalized AIGC model.

2. According to claim 1, the AIGC field customized framework based on edge collaborative fine-tuning is characterized by: The edge-driven collaborative fine-tuning performs the following steps: Step 1: Considering the heterogeneous resources of candidate edge devices, IoT devices are divided into subgroups, and an initial model of adaptive scale is deployed, and then local training is performed based on the original data to iteratively optimize the local model; Deep Mutual Learning (DML), as an online knowledge distillation algorithm, will be deployed in the edge network for knowledge exchange between edge devices, and supplemented with a series of machine learning algorithms, including stochastic gradient descent and batch gradient descent, which can update the local model by minimizing the loss function between the label value and the inference result; Step 2: Considering that edge devices usually have more limited computing and communication resources than data center servers, however, in federated learning, local model parameter sharing and transmission requires a lot of communication bandwidth and resource consumption; Step 3: The edge server receives updated parameters from terminal devices dispersed in different subgroups through the edge network; The edge server aggregates the collected parameters and forms a global model in the current communication round after parameter alignment.

3. According to claim 1, the AIGC field customized framework based on edge collaborative fine-tuning is characterized by: The AIGC model operating system performs the following steps: Step 1: Model information published on a public distributed ledger increases the risk of user privacy leakage; therefore, the present invention further enhances privacy protection by adding Laplace noise Lap(1 / ϵ) to each dimension of the gradient, which is called ϵ-differential privacy (DP); the present invention ensures that it is difficult to infer private information from a shared model based on ϵ-differential privacy, which can prevent an adversary from inferring the information of the prover P' from the blockchain; Step 2: In order to verify the quality of fine-tuning, the present invention proposes a peer-to-peer sub-Gossip consensus verification scheme without the need for a centralized coordinator; the clients participating in the AIGC fine-tuning task are defined as two groups, namely, prover P and verifier V; first, P updates the model parameters and information as evidence of AIGC, which can be used to prove the contribution to collaborative fine-tuning. Then, the global model needs to be verified by consensus algorithms such as Proof of Work (PoW) and Proof of Stake (PoS) to verify the quality of AIGC. In this process, participants in P randomly select a sub-Gossip network S from V to prevent differential attacks against updates, and only broadcast their local models to participants in subset S, verifying the model quality through a public data set. As a consensus node in the blockchain, V can evaluate the performance of AIGC and broadcast it to the edge network; if successfully verified, some of the fine-tuned model parameters can also be used as part of the base model for further fine-tuning; Step 3: In the system model, the edge server recruits heterogeneous terminal clients to perform collaborative fine-tuning through federated learning; Model the interaction between edge servers and end clients.

4. According to claim 2, the AIGC field customized framework based on edge collaborative fine-tuning is characterized by: The edge-driven collaborative fine-tuning execution step three specifically includes: the aggregated parameters are only a small part of the local model parameters, most of which are frozen to reduce the communication budget in the edge network; after several aggregations, the global model achieves the expected inference accuracy and model iteration termination.

Images