Cloud-edge collaborative small model efficient customization method
Through the efficient customization method of small models in cloud-edge collaboration, LoRA parameters are generated using large language models on the cloud, which solves the problems of insufficient efficiency and generalization ability of language models in specific tasks and realizes rapid customization and efficient deployment of edge-side models.
Patent Information
- Application Number
- CN202510773647.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-11
- Publication Date
- 2025-09-16
AI Technical Summary
Existing technologies suffer from low efficiency, insufficient generalization ability, and training complexity when optimizing language models for specific tasks, making it difficult to meet real-time requirements and improve the performance of unseen tasks.
Through the efficient customization method of small models in cloud-edge collaboration, LoRA parameters are generated using large language models on the cloud. Combined with re-parameterization technology and routing modules, LoRA experts are dynamically selected to generate task-customized parameters and integrate them into the edge-side model, simplifying the deployment process.
It significantly improves the reasoning efficiency and generalization ability of edge models, simplifies the model deployment process, and achieves efficient customization for specific tasks and adaptability to unseen tasks.
Smart Images
Figure CN120654747A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to, and in particular to, a method for efficiently customizing small models for cloud-edge collaboration. Background Art
[0002] With the rapid development of natural language processing (NLP) in recent years, large language models (LLMs) have demonstrated strong performance across a wide range of tasks. Their scale and number of parameters have continued to grow, with models such as GPT-3 and LLaMA boasting billions or even hundreds of billions of parameters. These large language models, pre-trained on massive amounts of text data, acquire rich linguistic knowledge and patterns, enabling them to achieve excellent performance across a wide range of NLP tasks. However, due to their sheer size, they face the challenge of balancing efficiency and effectiveness when applied to domain-specific tasks. On the one hand, general-purpose large language models often struggle to achieve ideal performance when handling complex domain-specific tasks, necessitating further optimization and tuning. On the other hand, their large size makes deploying and running these models directly on edge devices computationally expensive and inefficient during inference, making them difficult to meet the demands of real-time applications. To overcome these challenges, researchers have begun exploring more specialized and compact language models tailored to specific tasks and deployment requirements on edge devices.
[0003] Existing efficient parameter fine-tuning techniques, such as LoRA (Low-Rank Adaptation), specialize models by training on specific datasets. Low-rank factorization layers are introduced to minimize the number of trainable parameters, enabling efficient model tuning without incurring significant additional computational cost. However, LoRA methods are prone to catastrophic forgetfulness when trained on unseen tasks, resulting in performance degradation. To alleviate this problem, some methods have attempted to enhance the generalization capabilities of models by introducing a mixture of experts (MoE) mechanism. Methods such as LoRAMoE introduce multiple LoRA experts into the model and dynamically select the appropriate expert for computation based on the input during inference, alleviating the knowledge forgetting problem to some extent. However, these methods introduce additional costs during inference, such as additional expert selection mechanisms and control units, resulting in reduced inference efficiency. Furthermore, due to model size limitations, the generalization and performance improvements of these methods on unseen tasks are also limited, making it difficult to fully leverage the knowledge advantages of large language models.
[0004] The main shortcomings of existing technologies are in several aspects: (1) Efficiency issues: When traditional methods optimize language models for specific tasks, they often need to fine-tune the entire model. This is not only computationally expensive, but also inefficient during inference, making it difficult to meet application scenarios with high real-time requirements. For example, for some tasks that require fast response on edge devices, such as intelligent voice assistants and real-time translation, traditional fine-tuning methods will lead to excessively high inference delays, affecting user experience. (2) Insufficient generalization ability: After specialized training of the model, some methods have weak generalization capabilities for unseen tasks, resulting in a significant drop in model performance when facing new tasks. For example, after the LoRA method fine-tunes the model for specific tasks, although the performance on the training task is improved, it is prone to catastrophic forgetting on unseen tasks, resulting in performance degradation. (3) Training complexity: Some existing methods require a complex training process for the model, including fine-tuning and optimization on multiple tasks, which increases the difficulty and time cost of model training. For example, some MoE-based methods need to be trained on multiple tasks and require the design of complex expert selection mechanisms and training strategies, which makes the model training process very complicated.
[0005] It should be noted that the information disclosed in the above background technology section is only used to understand the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention
[0006] The main purpose of the present invention is to overcome the defects existing in the above-mentioned background technology and provide an efficient customization method for small models in cloud-edge collaboration.
[0007] To achieve the above object, the present invention adopts the following technical solutions:
[0008] An efficient customization method for small models in cloud-edge collaboration, including:
[0009] The large language model on the cloud receives prompts from the task definition system and generates meta tags corresponding to each Transformer layer of the edge model;
[0010] Based on the meta-tag, K LoRA experts are dynamically selected from the LoRA expert pool through the routing module; the LoRA expert pool includes multiple independent experts, each expert includes a LoRA block corresponding to a gated linear layer, an upsampling linear layer, and a downsampling linear layer;
[0011] Combine the parameters of the selected experts to generate customized LoRA parameters;
[0012] The task-customized LoRA parameters are fused into the feedforward network layer of the edge-side model through reparameterization technology to form a specialized edge model.
[0013] Furthermore, the meta-tag generation is specifically as follows: the cloud model inputs the task definition system prompt and the meta-tag together, encodes the task knowledge in the meta-tag through causal masking, and autoregressively outputs the meta-tag sequence corresponding to each Transformer layer.
[0014] Furthermore, the routing module performs the following operations:
[0015] Perform two linear projections and batch normalization on the meta tags to generate routing weights;
[0016] Use the KeepTOP-K strategy to select the K experts with the highest routing weights;
[0017] The combined weight of each selected expert is calculated by a normalized exponential function.
[0018] Furthermore, the generation of task customized LoRA parameters is specifically: performing weighted summation on the selected K expert parameters, where the weight is determined by the normalized index value output by the routing module.
[0019] Furthermore, the reparameterization technology is specifically: linearly superimposing the generated LoRA parameters with the original parameters of the edge side model, so that the fused model does not introduce additional computing components during reasoning.
[0020] Furthermore, a load balancing auxiliary loss function is introduced during the training process: the square of the coefficient of variation of the probability of selection of each expert in the batch is calculated and added to the total loss function as a constraint term to promote the balanced utilization of experts.
[0021] Furthermore, the total loss function is composed of the linear superposition of language modeling cross entropy loss and load balancing auxiliary loss, and task knowledge learning and expert load balancing are achieved through joint optimization.
[0022] Furthermore, the task definition system prompts include task description, few-sample instances, role definition and input and output formats; this information is compressed to the edge model through LoRA parameters, reducing the context length during inference.
[0023] Furthermore, the routing module independently generates routing weights for each Transformer layer of the edge-side model;
[0024] Based on the routing weights, the following operations are performed on the LoRA expert pool:
[0025] A normalized exponential function is used to process routing weights and generate the combined weights of each expert;
[0026] Weight the gated linear layer, upsampling linear layer, and downsampling linear layer LoRA blocks of the selected expert according to the combined weight;
[0027] The weighted LoRA blocks are summed layer by layer to generate hierarchical customized parameters.
[0028] A computer program product includes a computer program, which, when executed by a processor, implements the efficient customization method of small models for cloud-edge collaboration.
[0029] The present invention has the following beneficial effects:
[0030] The core advantage of the efficient customization method of cloud-edge collaborative small models proposed in the present invention is that it generates LoRA parameters for edge-side models through large-scale language models on the cloud side, realizing knowledge transfer and rapid model customization. On the one hand, the powerful knowledge reserve of the cloud model is utilized to compress task-related system prompts into LoRA parameters, avoiding the edge-side model from calculating the entire large model. Combined with the re-parameterization technology, the additional computing cost during reasoning is eliminated, significantly improving the reasoning efficiency; on the other hand, through the LoRA expert pool and dynamic routing mechanism, the adaptive expert combination is selected based on meta-tags to generate task-specific parameters, enhancing the model's generalization ability for unseen tasks, while introducing load balancing auxiliary loss to optimize expert utilization and avoid the catastrophic forgetting problem of traditional methods. In addition, the framework does not require complex training on the edge side, simplifies the deployment process, and effectively balances the contradiction between specific task performance optimization and edge device resource limitations, with the dual advantages of improved efficiency and enhanced generalization ability.
[0031] The present invention solves the core problem of language models facing the difficulty in balancing efficiency and generalization capabilities in specific tasks through a cloud-edge collaborative architecture. Its main technical advantages are: utilizing the knowledge reserves of large language models on the cloud to generate task-customized LoRA parameters, significantly improving the reasoning efficiency and generalization capabilities of the edge-side model, and greatly simplifying the deployment process. Specifically, the cloud-side model dynamically compresses the task definition information into meta-tags, and efficiently generates hierarchical parameters from the LoRA expert pool through a routing mechanism, which are seamlessly integrated into the edge model feedforward network through re-parameterization technology, avoiding the full model computational overhead of traditional fine-tuning and eliminating the additional reasoning cost of the MoE method; through knowledge transfer, the resource-constrained edge model is adapted to unseen tasks, while eliminating the complex training process on the edge side, achieving plug-and-play lightweight deployment. Through the collaboration of cloud and edge-side models, the knowledge advantages of large language models can be fully utilized, so that the edge-side model can better complete tasks in specific fields.
[0032] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 This is the overall flow chart of the efficient customization method of small models for cloud-edge collaboration of the present invention.
[0034] Figure 2 This is a schematic diagram of the LoRA parameter generation framework for cloud-edge collaboration in an embodiment of the present invention. DETAILED DESCRIPTION
[0035] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the present invention and its application.
[0036] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, "plurality" means two or more, unless otherwise specifically defined.
[0037] This invention primarily addresses the difficulty in balancing efficiency and effectiveness in current language model applications for specific domain tasks, particularly in resource-constrained small edge models. While traditional fine-tuning methods can improve model performance on specific tasks, they often require extensive parameter updates for the entire model. This not only results in high training costs but also inefficiencies during inference due to the large model size, making it difficult to meet the demands of real-time applications. On the other hand, existing efficient parameter fine-tuning techniques, such as LoRa, while reducing the number of trainable parameters to a certain extent, are prone to catastrophic forgetting when faced with unseen tasks, leading to performance degradation. Furthermore, while some methods that attempt to mitigate the problem of knowledge forgetting through a mixture of experts (MoE) mechanism have made some progress in multi-task learning, they introduce additional costs during inference and, due to model size limitations, limit generalization and performance improvements on unseen tasks. Therefore, this invention aims to propose an efficient customization method for small edge models. By designing a framework that can efficiently generate task-specific LoRa parameters, this method enables rapid model customization, improves inference efficiency and generalization to unseen tasks, and simplifies model deployment.
[0038] See Figure 1 and Figure 2 , an embodiment of the present invention provides an efficient customization method for a small model in cloud-edge collaboration, comprising the following steps:
[0039] Step S1: The large language model on the cloud receives prompts from the task definition system and generates meta tags corresponding to each Transformer layer of the small model on the edge;
[0040] Step S2: Based on the meta-tag, dynamically select K LoRA experts from the LoRA expert pool through the routing module; the LoRA expert pool includes multiple independent experts, each expert includes a LoRA block corresponding to the gated linear layer, the upsampling linear layer, and the downsampling linear layer;
[0041] Step S3: combining the parameters of the selected experts to generate task-customized LoRA parameters;
[0042] Step S4: The task-customized LoRA parameters are integrated into the feedforward network layer of the edge side model through reparameterization technology to form a specialized edge model.
[0043] In some embodiments, the generating of meta-tags is specifically as follows: the cloud model inputs the task definition system prompt and the meta-tag together, encodes the task knowledge in the meta-tag through causal masking, and autoregressively outputs the meta-tag sequence corresponding to each Transformer layer.
[0044] In some embodiments, the task definition system prompt includes a task description, a few-sample instance, a role definition, and an input and output format; this information is compressed to the edge model through LoRA parameters to reduce the context length during reasoning.
[0045] See Figure 2 In some embodiments, the routing module performs the following operations: performing two linear projections and batch normalization on the meta-tags in sequence to generate routing weights; selecting the K experts with the highest routing weights using the KeepTOP-K strategy; and calculating the combined weight of each selected expert using a normalized exponential function.
[0046] In some embodiments, generating task-customized LoRA parameters specifically includes: performing weighted summation on the selected K expert parameters, where the weight is determined by the normalized index value output by the routing module.
[0047] In some embodiments, the reparameterization technology is specifically: linearly superimposing the generated LoRA parameters with the original parameters of the edge side model so that the fused model does not introduce additional computing components during reasoning.
[0048] In some embodiments, a load balancing auxiliary loss function is introduced during the training process: the square of the coefficient of variation of the probability of selection of each expert in the batch is calculated and added as a constraint term to the total loss function to promote balanced expert utilization.
[0049] In some embodiments, the total loss function is composed of a linear superposition of language modeling cross entropy loss and load balancing auxiliary loss, and task knowledge learning and expert load balancing are achieved through joint optimization.
[0050] See Figure 2In some embodiments, the routing module independently generates routing weights for each Transformer layer of the edge-side model; based on the routing weights, the following operations are performed on the LoRA expert pool: the routing weights are processed using a normalized exponential function to generate a combined weight of each expert; the gated linear layer, upsampling linear layer, and downsampling linear layer LoRA blocks of the selected expert are weighted according to the combined weights; the weighted LoRA blocks are summed by layer to generate layered customized parameters.
[0051] The following further describes specific embodiments of the present invention and algorithm examples.
[0052] This paper proposes an efficient customization method for small models in cloud-edge collaboration and designs a framework called LoRA-Gen. The core idea is to use a large language model on the cloud to generate LoRA parameters for the edge-side model, such as Figure 2 As shown, the specific implementation process of this method is as follows.
[0053] Cloud-based LoRA generator construction: A fine-tuned large language model and a set of LoRA experts are deployed on the cloud. This large language model autoregressively generates a set of meta-tokens based on input task definition system prompts (including task descriptions, a small number of samples, role definitions, and dialogue formats). Each meta-token corresponds to a Transformer layer in the edge-side language model. Specifically, after receiving the task definition system prompts, the cloud-based large language model inputs them along with the meta-tokens. Using causal masking, the meta-tokens are used to encode task-related knowledge. These meta-tokens thus become a carrier of task knowledge and provide a foundation for subsequent LoRA parameter generation. A LoRA expert pool and routing module: A pool of multiple LoRA experts is constructed. Each LoRA expert consists of three LoRA blocks, corresponding to the gated linear layer, upsampling linear layer, and downsampling linear layer of the Transformer layer in the edge model. The routing module uses the meta-tokens to control which experts from the LoRA expert pool are combined. The routing module consists of two linear projections and a batch normalization layer, and uses the KeepTOP-K strategy to determine the expert combination for each Transformer layer. Specifically, the routing module first processes the meta tag (T i meta ) performs two linear projections (f1,f2) and increases the diversity of the output through a batch normalization layer (BN).
[0054]
[0055] Then, the KeepTOP-K strategy is used to select the parameters of K experts from the expert pool for combination.
[0056]
[0057] This form ensures that each Transformer layer can obtain the most suitable LoRA parameter combination for the task. LoRA parameter generation and edge-side model fusion: According to the output of the routing module, the selected LoRA expert parameters are combined to generate the LoRA parameters (θ i ).
[0058]
[0059] These LoRA parameters are then merged into the feedforward network layer of the edge-side language model through reparameterization, resulting in a specialized edge-side model. Specifically, for each Transformer layer in the edge-side model, the generated LoRA parameters are merged with the original model parameters to form a specialized model. Thanks to the reparameterization technique, the merged model does not require additional computational cost during inference, thus ensuring inference efficiency.
[0060] Training objective and auxiliary loss: To ensure the load balance of the LoRA expert pool, an auxiliary loss function is introduced, which uses coefficient variation as a constraint to encourage more balanced use of each expert.
[0061]
[0062] Where σ and μ represent the variance and mean of the routing results, respectively. The total loss function consists of the cross entropy loss and auxiliary loss for language modeling.
[0063]
[0064] Specifically, the auxiliary loss function measures the degree of load imbalance by calculating the ratio of the standard deviation to the mean of each expert's probability of being selected in a batch of data. This ratio is used as part of the loss function to encourage the model to use experts more evenly. The total loss function is composed of the cross-entropy loss for language modeling and the auxiliary loss. By optimizing the total loss function, the model can not only learn task-related knowledge during training, but also maintain a balanced load across the expert pool.
[0065] Through the above technical solutions, the LoRA-Gen framework can achieve rapid specialization of edge-side models while avoiding the problems of low efficiency, insufficient generalization ability, and complex training existing in traditional methods. Specifically, the LoRA-Gen framework enables edge-side models to better adapt to specific domain tasks through knowledge transfer from large language models in the cloud, thereby improving the model's generalization ability. At the same time, by generating LoRA parameters in the cloud and incorporating them into the edge-side model, the problem of traditional methods requiring calculation of the entire model during inference is avoided, significantly improving the model's inference efficiency.
[0066] In summary, the present invention provides an efficient customization method for small models in cloud-edge collaboration. Its innovative work and important features include:
[0067] (1) Collaboration between cloud and edge models: Innovatively utilizing the powerful knowledge reserve of large-scale language models on the cloud to generate LoRA parameters for edge models, achieving knowledge transfer between cloud and edge models and giving full play to the advantages of both.
[0068] (2) Efficient LoRA parameter generation: By designing a reasonable LoRA expert pool and routing module, LoRA parameters for specific tasks can be quickly generated without increasing additional reasoning costs, significantly improving the reasoning efficiency of the model.
[0069] (3) Generalization capability for unseen tasks: The LoRA-Gen framework does not require special training for unseen tasks. It can obtain specialized model parameters through a single inference, demonstrating strong generalization capability for unseen tasks and simplifying the model deployment process.
[0070] (4) Context compression: Dynamically compressing task-related system prompts into LoRA weights effectively reduces the context length of the specialized model and further improves the model's reasoning efficiency.
[0071] (5) Application of reparameterization technology: Reparameterization technology is used to merge the generated LoRA parameters into the edge-side model, avoiding the introduction of additional components and computational costs during inference, so that the specialized model has higher inference efficiency while maintaining high performance.
[0072] The method of the present invention designs a framework that can efficiently generate LoRA parameters for specific tasks. By utilizing the knowledge of the large language model on the cloud, LoRA parameters are generated for the small model on the edge, thereby achieving rapid specialization of the model, while improving the reasoning efficiency and generalization ability for unseen tasks, and simplifying the model deployment process. Specifically, compared with traditional technologies, the cloud-edge collaborative small model efficient customization method of the present invention has the following significant advantages: (1) Improve reasoning efficiency: By generating LoRA parameters on the cloud and merging them into the edge model, the problem of needing to calculate the entire model during reasoning in traditional methods is avoided, which significantly improves the reasoning efficiency of the model. (2) Enhance generalization ability: By transferring the knowledge of the large language model on the cloud, the edge model can better adapt to unseen tasks, thereby improving the generalization ability of the model. (3) Simplify model deployment: There is no need to perform a complex training process on the edge model. The model can be specialized only through the LoRA parameters generated on the cloud, which simplifies the model deployment process. (4) Give full play to the knowledge advantages of the large language model: By collaborating with the cloud and edge models, the knowledge advantages of the large language model are fully utilized, so that the edge model can better complete specific domain tasks.
[0073] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.
[0074] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.
[0075] An embodiment of the present invention further provides a processor, which executes a computer program and at least performs the method described above.
[0076] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disc or a read-only optical disc (CD-ROM); the magnetic surface memory can be a magnetic disk memory or a magnetic tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.
[0077] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0078] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0079] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0080] Those skilled in the art will understand that all or part of the steps of the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc. Various media that can store program codes.
[0081] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.
[0082] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.
[0083] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.
[0084] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.
[0085] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. Those skilled in the art will recognize that, without departing from the scope of the present invention, several equivalent substitutions or obvious variations can be made, and the performance or use of the same should be considered to fall within the scope of protection of the present invention.
Claims
1. An efficient customization method for small models in cloud-edge collaboration, characterized by: include: The large language model on the cloud receives prompts from the task definition system and generates meta tags corresponding to each Transformer layer of the edge model; Based on the meta-tag, K LoRA experts are dynamically selected from the LoRA expert pool through the routing module; the LoRA expert pool includes multiple independent experts, each expert includes a LoRA block corresponding to a gated linear layer, an upsampling linear layer, and a downsampling linear layer; Combine the parameters of the selected experts to generate customized LoRA parameters; The task-customized LoRA parameters are fused into the feedforward network layer of the edge-side model through reparameterization technology to form a specialized edge model.
2. The method according to claim 1, wherein The meta-tag generation is specifically as follows: the cloud model inputs the task definition system prompt and the meta-tag together, encodes the task knowledge in the meta-tag through causal masking, and autoregressively outputs the meta-tag sequence corresponding to each Transformer layer.
3. The method according to claim 1 or 2, wherein: The routing module performs the following operations: Perform two linear projections and batch normalization on the meta tags to generate routing weights; Use the KeepTOP-K strategy to select the K experts with the highest routing weights; The combined weight of each selected expert is calculated by a normalized exponential function.
4. The method according to claim 3, wherein The generation of task-customized LoRA parameters is specifically as follows: performing weighted summation on the selected K expert parameters, where the weight is determined by the normalized index value output by the routing module.
5. The method according to any one of claims 1 to 4, characterized in that The reparameterization technology is specifically: linearly superimposing the generated LoRA parameters with the original parameters of the edge side model, so that the fused model does not introduce additional computing components during inference.
6. The method according to any one of claims 1 to 5, characterized in that During the training process, a load balancing auxiliary loss function is introduced: the square of the coefficient of variation of the probability of selection of each expert in the batch is calculated and added to the total loss function as a constraint term to promote balanced expert utilization.
7. The method according to claim 6, wherein The total loss function is composed of the linear superposition of language modeling cross entropy loss and load balancing auxiliary loss, and task knowledge learning and expert load balancing are achieved through joint optimization.
8. The method according to any one of claims 1 to 7, wherein: The task definition system prompts include task description, few-sample examples, role definition, and input and output formats; this information is compressed to the edge model through LoRA parameters, reducing the context length during inference.
9. The method according to any one of claims 1 to 8, wherein The routing module independently generates routing weights for each Transformer layer of the edge model; Based on the routing weights, the following operations are performed on the LoRA expert pool: A normalized exponential function is used to process routing weights and generate the combined weights of each expert; Weight the gated linear layer, upsampling linear layer, and downsampling linear layer LoRA blocks of the selected expert according to the combined weight; The weighted LoRA blocks are summed layer by layer to generate hierarchical customized parameters.
10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the efficient customization method of small models for cloud-edge collaboration as described in any one of claims 1 to 9.