Optimization method for deploying hybrid expert model in edge computing environment and related device

Abstract

The application discloses an optimization method for deploying a mixed expert model in an edge computing environment and related equipment. Through a pre-trained optimization model, the inference requests of each user are received, and the allocation scheme of these requests to each edge server and the expert mixed quantization bit width scheme of each server are output. The training process of the optimization model includes: collecting user inference requests, constructing pre-training data, the pre-training data including randomly generated inference request allocation and corresponding expert mixed quantization bit width; using the data to pre-train the policy network in reinforcement learning; inputting the user inference request into the initialized policy network to obtain the allocation scheme and the quantization bit width scheme; serving the user inference request according to the scheme to obtain the service quality, and taking the service quality as the reward of the policy network, iteratively updating the network parameters until the service quality reaches the preset requirement. The application aims to dynamically adjust the expert quantization bit width of the edge server, and combines the allocation optimization of the user inference request to realize high-precision and low-delay edge inference service.

Description

Technical Field

[0001] The present invention belongs to the field of edge computing deployment of large language models, and specifically relates to an optimization method and related equipment for deploying a hybrid expert model in an edge computing environment. Background Art

[0002] With the continuous advancement and widespread application of large language model (LLM) technology, it has demonstrated tremendous potential and value in a variety of fields, including natural language processing, artificial intelligence assistants, and intelligent customer service. Current mainstream deployment methods typically rely on hosting LLMs in cloud data centers, leveraging the cloud's powerful computing capabilities to process user inference requests. Users send requests over the network to the cloud, which completes the inference computation and returns the results to the user. While this cloud-based deployment model provides powerful computing services, it also introduces network latency issues. Furthermore, since user inference request data must be transmitted to the cloud for processing, this increases the risk of user privacy leakage and poses a potential threat to data security. To address these issues, the industry has recently begun exploring the possibility of deploying large-scale language models directly at the network edge. By offloading computing tasks from the cloud to the network edge, edge computing can significantly reduce response times and improve user experience. It also enhances user privacy protection to a certain extent, as user inference request data no longer needs to be transmitted to the cloud, reducing the risk of data leakage. However, the relatively limited hardware resources in edge computing environments contrast sharply with the massive model size of LLMs, posing significant challenges to the efficient deployment of LLMs on edge devices.

[0003] To address this challenge, researchers proposed an LLM architecture called "Mixture-of-Experts (MoE)". The MoE model significantly reduces the computational overhead during inference by splitting the large model into multiple smaller expert models and activating only some of the experts at each inference. This sparse activation feature makes the MoE model very suitable for resource-constrained edge environments. Therefore, how to optimize the deployment of the MoE model at the edge has become a current research hotspot. Currently, in the process of optimizing the edge deployment of the MoE model, the quantization technology of the expert model has gradually become a key technology. Quantization technology can reduce the storage requirements and computational complexity of the MoE model by compressing the weights of the MoE model, thereby enabling it to run on edge devices with limited hardware resources. However, the existing MoE model quantization methods still have many shortcomings.

[0004] On the one hand, existing MoE model quantization methods usually adopt a unified quantization strategy, such as converting all expert weights into Int8 format. However, the expert models in the MoE model have different activation frequencies and importance during the reasoning process, so adopting a unified quantization strategy may not fully utilize the performance advantages of each expert model. For example, for those expert models that are frequently activated, a higher precision (such as Bf16) should be retained to ensure the accuracy of reasoning; while for those expert models that are not frequently used, they can be quantized to a lower precision (such as Int4 or Int8) to further compress the model size. In other words, a unified quantization strategy cannot reduce the storage and computational overhead of the MoE model while maintaining high reasoning accuracy.

[0005] On the other hand, existing MoE model quantization methods do not fully consider how heterogeneously quantized MoE models affect the distribution of user requests across edge servers. In edge computing environments, different inference requests may activate different sets of expert models, and therefore they may benefit from different quantization schemes. However, existing MoE model quantization methods do not optimize for this characteristic, resulting in inefficient distribution of user requests across edge servers. Summary of the Invention

[0006] In response to the problems existing in the prior art, the present invention provides an optimization method and related equipment for deploying a hybrid expert model in an edge computing environment. The purpose is to achieve high-precision, low-latency edge inference services by dynamically adjusting the expert quantization bit width on the edge server and combining it with the distribution optimization of user inference requests.

[0007] In order to solve the above technical problems, the present invention is implemented through the following technical solutions:

[0008] According to a first aspect of the present invention, there is provided an optimization method for deploying a hybrid expert model in an edge computing environment, comprising:

[0009] Get the inference request of each current user;

[0010] Input the current inference requests of each user into a pre-trained optimization model, and output a distribution plan of each user's inference requests to each edge server, as well as an expert hybrid quantization bit width plan for each edge server;

[0011] The training method of the optimization model includes:

[0012] Obtain inference requests from each user;

[0013] The pre-training data is constructed by using reasoning requests of each user, and the pre-training data includes reasoning request assignments randomly generated by using the reasoning requests of each user, and a mixed quantization bit width of an expert determined according to an expert activation frequency of the reasoning request assignments when each edge server serves the randomly generated reasoning request assignments;

[0014] The policy network in reinforcement learning is pre-trained by using the pre-training data to obtain an initialized policy network;

[0015] The reasoning requests of each user are input into the initialized policy network to obtain an allocation scheme of the reasoning requests of each user to each edge server output by the current policy network and an expert mixed quantization bit width scheme of each edge server;

[0016] The allocation scheme of the reasoning requests of each user to each edge server output by the current policy network and the expert mixed quantization bit width scheme of each edge server are used to serve the reasoning requests of each user to obtain a service quality;

[0017] The service quality is used as a reward of the current policy network, and parameters of the current policy network are iteratively updated until the service quality reaches a preset requirement to obtain the optimization model.

[0018] In a possible implementation manner of the first aspect, the service quality includes a service accuracy, a service time delay, and a switching cost of the expert mixed quantization bit width scheme.

[0019] In a possible implementation manner of the first aspect, when each edge server serves the randomly generated reasoning request assignments, the mixed quantization bit width of the expert determined according to the expert activation frequency of the reasoning request assignments is specifically:

[0020] The reasoning requests randomly assigned to each edge server are input into a local original mixed expert model to obtain an activation frequency of each expert in the local original mixed expert model, and the local original mixed expert model includes a plurality of expert groups, and each expert group includes a plurality of experts.

[0021] The quantization bit width of each expert in each expert group is determined according to the activation frequency of each expert in the expert group.

[0022] The quantization bit widths of the experts in all expert groups form a mixed quantization decision of the local mixed expert model.

[0023] In a possible implementation manner of the first aspect, the quantization bit width of each expert in each expert group is determined according to the activation frequency of each expert in the expert group, and specifically:

[0024] Each expert in each expert group is sorted in descending order according to the activation frequency.

[0025] According to the preset mixed quantization bit width set, the last half of the experts in each layer expert group after descending order sorting are assigned the highest quantization bit width in the mixed quantization bit width set; wherein the quantization bit widths in the preset mixed quantization bit width set are arranged from large to small;

[0026] According to the capacity limit of each edge server, it is judged whether the total quantization bit width of each layer expert group after quantization exceeds the corresponding capacity limit;

[0027] If it exceeds, the quantization bit width of all experts in each layer expert group is reduced, and the quantization bit width of each expert is specifically reduced to the highest quantization bit width in the mixed quantization bit width set which is lower than the current bit width of the expert;

[0028] The judgment process is repeated until the total quantization bit width of each layer expert group after quantization does not exceed the corresponding capacity limit, and then the quantization bit width of each expert in the layer expert group is determined.

[0029] In a possible implementation manner of the first aspect, the pre-training of the policy network in reinforcement learning by using the pre-training data is specifically:

[0030] Each inference request randomly allocated to each edge server is used as the input of the policy network, the quantization bit width of each expert in each layer expert group determined according to the activation frequency of each expert in each layer expert group is used as the output label of the policy network, the policy network is pre-trained until convergence, and the initialized policy network is obtained.

[0031] In a possible implementation manner of the first aspect, the inference request of each user is input into the initialized policy network to obtain the allocation scheme of the inference request of each user to each edge server output by the current policy network, and the expert mixed quantization bit width scheme of each edge server, and the allocation scheme of the inference request of each user to each edge server output by the current policy network, and the expert mixed quantization bit width scheme of each edge server are specifically:

[0032]

[0033] In the formula, s t is the expert mixed quantization bit width on the current all edge servers and the current inference request of each user; is the expert mixed quantization bit width scheme of edge server i at time t, wherein represents the data type of expert j on edge server i at time t after quantization, and there are m experts; E is all edge servers; K t is the user inference request set at time t, k∈K t , wherein the user inference request k is allocated to the edge server with ID k s ; a tThe quantization bit width of each expert on all edge servers at the next moment and the reasoning request of each user are finally assigned to the user with ID k s edge servers.

[0034] In a possible implementation of the first aspect, the allocation scheme of each user's inference request output by the current policy network to each edge server, and the expert hybrid quantization bit width scheme of each edge server, are used to serve each user's inference request to obtain service quality, specifically:

[0035] r t =∑ i {p i -α1·l i -α2·C i}

[0036] Where r t is the reward of the current policy network; p i For service accuracy; i is the service delay; C i is the switching cost of the expert hybrid quantization bit width scheme; α1 and α2 are parameters used to weigh the service accuracy, service delay and the switching cost of the expert hybrid quantization bit width scheme.

[0037] According to a second aspect of the present invention, there is provided an optimization device for deploying a hybrid expert model in an edge computing environment, comprising:

[0038] The acquisition module is used to obtain the inference requests of each current user;

[0039] An optimization module, configured to input the inference requests of each current user into a pre-trained optimization model, and output a distribution scheme of the inference requests of each user to each edge server, as well as an expert hybrid quantization bit width scheme for each edge server;

[0040] The training method of the optimization model includes:

[0041] Obtain inference requests from each user;

[0042] Constructing pre-training data using each user's inference request, the pre-training data including an inference request allocation randomly generated using each user's inference request, and an expert hybrid quantization bit width determined based on the expert activation frequency of the inference request allocation when each edge server serves the randomly generated inference request allocation;

[0043] Pre-training a policy network in reinforcement learning using the pre-training data to obtain an initialized policy network;

[0044] input the inference requests of each user into the initialization strategy network to obtain an allocation scheme of the inference requests of each user to each edge server output by the current strategy network and an expert mixed quantization bit width scheme of each edge server;

[0045] serve the inference requests of each user by using the allocation scheme of the inference requests of each user to each edge server output by the current strategy network and the expert mixed quantization bit width scheme of each edge server to obtain a service quality;

[0046] take the service quality as a reward of the current strategy network, and iteratively update parameters of the current strategy network until the service quality reaches a preset requirement to obtain the optimization model.

[0047] According to a third aspect of the present application, an apparatus is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the optimization method for deploying a mixed expert model in an edge computing environment when executing the computer program.

[0048] According to a fourth aspect of the present application, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the optimization method for deploying a mixed expert model in an edge computing environment when executed by a processor.

[0049] Compared with the prior art, the present application has at least the following beneficial effects:

[0050] The optimization method for deploying a mixed expert model in an edge computing environment provided by the present application dynamically adjusts the quantization bit width of different experts on each edge server according to the frequency and task information of inference requests by using a mixed expert quantization strategy. High-frequency activated experts maintain high precision, and low-frequency activated experts use low-precision quantization. The method significantly reduces model storage and computing resources while maintaining high inference precision, and solves the problems of decreased inference precision and low storage resource utilization caused by uniform quantization. The expert model mixed quantization strategy proposed in the present application considers the influence of inference request distribution on the selection of mixed quantization schemes. By learning different quantization schemes under different requirements, the method realizes the joint optimization of expert quantization and inference request allocation, and ensures high precision and low delay of inference service.

[0051] In order to make the above-mentioned objects, features and advantages of the present application more apparent and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are referred to as follows. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the specific embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the specific embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 This is a flowchart of an optimization method for deploying a hybrid expert model in an edge computing environment according to an embodiment of the present invention;

[0054] Figure 2 Schematic diagram of the MoE model;

[0055] Figure 3 This is a typical scene diagram in the embodiment;

[0056] Figure 4 This is a comparison diagram of the reinforcement learning convergence process with and without pre-training design in the embodiment;

[0057] Figure 5 is the average reward value that changes over time in the embodiment. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0059] The hybrid expert model takes advantage of the sparse activation characteristics of large language models in single-task reasoning and uses only a small number of multi-layer perceptrons for efficient and accurate reasoning, that is, converting traditional dense layers into multiple expert models, such as Figure 2 As shown in Figure 2. Typical edge scenarios include multiple edge servers and multiple user requirements, such as Figure 3 shown.

[0060] Combine Figure 1 As shown, an embodiment of the present invention provides an optimization method for deploying a hybrid expert model in an edge computing environment. This method dynamically adjusts the expert quantization bit width on the edge server and optimizes the distribution of user inference requests to achieve high-precision, low-latency edge inference services. The specific steps of the optimization method are as follows:

[0061] Step 1: Get the inference request of each current user.

[0062] Specifically, in the edge computing environment, various users submit their inference requests to the system. These requests can include text generation, language understanding, question answering, picture recognition, and various types of processing tasks, and the system collects these inference requests as inputs for subsequent steps.

[0063] Step two, input the current inference requests of each user into the pre-trained optimization model, output the allocation scheme of the inference requests of each user to each edge server, and the expert mixed quantization bit width scheme of each edge server.

[0064] That is, input the current collected inference requests of each user into the pre-trained optimization model, which is a reinforcement learning-based policy network responsible for outputting the allocation scheme of the inference requests of each user to each edge server, and the expert mixed quantization bit width scheme of each edge server.

[0065] In the embodiment of the application, the training method of the optimization model specifically comprises the following steps:

[0066] a. Obtain the inference requests of each user.

[0067] Specifically, obtain a large number of inference requests of users from historical data as the basis for constructing pre-training data.

[0068] b. Construct pre-training data using the inference requests of each user, the pre-training data including randomly generated inference request allocations using the inference requests of each user, and expert mixed quantization bit widths determined according to the expert activation frequency of the inference request allocation when each edge server serves the randomly generated inference request allocation.

[0069] That is, for each inference request, a different inference request allocation scheme is randomly generated, i.e., it is decided which edge server should process each inference request. At the same time, according to each inference request allocation scheme, the expert activation frequency in the inference process is simulated, and the expert mixed quantization bit width scheme is determined based on these activation frequencies, which will be used as the label of the pre-training data.

[0070] c. Pre-train the policy network in reinforcement learning using the pre-training data to obtain an initialized policy network.

[0071] That is, pre-train the policy network in reinforcement learning using the pre-training data to obtain an initialized policy network. The initialized policy network will learn how to generate reasonable inference request allocation schemes and expert mixed quantization bit width schemes based on the characteristics of the inference requests.

[0072] Exemplarily, Figure 4This is a comparison of the reinforcement learning convergence process with and without pre-training design. The results show that pre-training design can effectively accelerate the convergence of reinforcement learning.

[0073] d. Input the reasoning request of each user into the initialized policy network to obtain the allocation scheme of the reasoning request of each user to each edge server output by the current policy network, as well as the expert hybrid quantization bit width scheme of each edge server.

[0074] e. Use the allocation scheme of each user's inference request to each edge server output by the current policy network and the expert hybrid quantization bit width scheme of each edge server to serve each user's inference request and obtain the service quality.

[0075] f. Using the service quality as a reward for the current policy network, iteratively updating the parameters of the current policy network until the service quality meets the preset requirements, thereby obtaining the optimization model.

[0076] Specifically, steps d, e, and f are the process of iteratively updating the policy network. New inference requests are input into the initialized policy network, resulting in the allocation and quantization bit width schemes for the current policy network output. Based on these schemes, each user's inference request is served, and quality of service metrics, such as response time and inference accuracy, are collected. Using service quality as a reward for the current policy network, a reinforcement learning algorithm (such as the policy gradient method) is used to iteratively update the parameters of the current policy network. This process is repeated until the service quality meets the preset requirements. The resulting policy network is then considered the optimized model.

[0077] It should be understood that after optimizing the model output, the system obtains a plan for allocating each user's inference requests to each edge server, as well as the expert hybrid quantization bit width scheme for each edge server. Based on these schemes, the system assigns inference requests to the corresponding edge server and quantizes the expert model on the edge server according to the specified quantization bit width scheme. Each edge server executes the inference service based on the assigned inference request and the quantized expert model. By utilizing a dynamically adjusted quantization bit width scheme, the system can maintain high inference accuracy while reducing storage and computational overhead, achieving high-precision, low-latency edge inference services.

[0078] This paper combines reinforcement learning and dynamic quantization techniques to propose an optimization method for deploying hybrid expert models in edge computing environments. This method dynamically adjusts the expert quantization bit width on edge servers and optimizes the allocation of user inference requests to achieve high-precision, low-latency edge inference services. By training a reinforcement learning-based policy network, the system learns how to generate appropriate allocation and quantization bit width schemes based on the characteristics of inference requests, achieving excellent results in practical applications.

[0079] In one implementation, the quality of service includes service accuracy, service latency, and the switching cost of the expert hybrid quantization bit width scheme. This method not only considers service accuracy and service latency but also incorporates the switching cost of the expert hybrid quantization bit width scheme (e.g., the computing resources and time required to switch from one quantization scheme to another) to comprehensively evaluate and optimize service quality.

[0080] It's important to note that in edge computing environments, each user's inference request may have different requirements for service accuracy and latency. The optimization model, a reinforcement learning-based policy network, comprehensively considers service accuracy, service latency, and the switching costs of the expert hybrid quantization bitwidth scheme. It then outputs a plan for allocating each user's inference request to each edge server, as well as the expert hybrid quantization bitwidth scheme for each edge server.

[0081] In detail, in this embodiment, the training method for the optimization model is as follows:

[0082] Obtain inference requests: Obtain a large number of users' inference requests from historical data. These requests should contain various task types and different requirements for service quality.

[0083] Constructing pre-training data: For each inference request, we randomly generate different inference request allocation schemes and simulate the frequency of expert activation during inference. We also generate multiple possible expert mixed quantization bit width schemes based on the activation frequency and the preset quantization strategy. For each scheme, we calculate the corresponding service accuracy, service latency, and switching cost.

[0084] Pre-trained policy network: Using the pre-trained data, the policy network in reinforcement learning is pre-trained. This policy network learns how to generate a reasonable inference request allocation plan and expert hybrid quantization bit width plan based on the characteristics of the inference request (such as task type and quality of service requirements) to maximize service quality.

[0085] Iteratively update the policy network: Input the new inference request into the initialized policy network to obtain the allocation scheme and quantization bit width scheme output by the current policy network. According to these schemes, serve the inference requests of each user and collect the actual service accuracy, service delay and switching cost data. Use these service quality data as rewards (or penalties) for the current policy network and use reinforcement learning algorithms (for example, deep deterministic policy gradient DDPG, proximal policy optimization PPO, etc.) to iteratively update the parameters of the current policy network. During the update process, it should be ensured that the policy network, while pursuing high accuracy and low latency, also considers reducing the switching cost of the quantization bit width scheme to achieve overall service quality optimization. Repeat the above process until the service quality (taking into account service accuracy, service delay and switching cost) meets the preset requirements or converges to a stable state. The policy network obtained at this time is the optimization model.

[0086] In one possible implementation, regarding "when each edge server services randomly generated inference requests, the expert hybrid quantization bit width is determined based on the expert activation frequency assigned to the inference requests", the specific implementation process is as follows:

[0087] a. Inputting the inference requests randomly assigned to each edge server into a local original hybrid expert model to obtain the activation frequency of each expert in the local original hybrid expert model; wherein the local original hybrid expert model includes multiple layers of expert groups, and each layer of expert group includes multiple experts.

[0088] Specifically, for each inference request randomly assigned to an edge server, the system inputs it into a local original hybrid expert model. The local original hybrid expert model consists of multiple layers of expert groups, each containing multiple experts. These experts are selectively activated to complete the inference task based on the input data.

[0089] The system records the activation frequency of each expert in the local original hybrid expert model for each inference request. It should be noted that the activation frequency refers to the ratio of the number of times a particular expert is called to handle that inference request to the total number of inference requests. This frequency reflects the importance and activity of the expert in handling a specific type of inference request.

[0090] b. According to the activation frequency of each expert in each layer of the expert group, the quantization bit width of each expert in the expert group at that layer is determined.

[0091] Specifically, the system determines the quantization bit width for each expert in each layer's expert group based on their activation frequency. It's important to note that quantization bit width refers to the number of bits used when converting the expert model's weights from a high-precision format to a low-precision format. The choice of quantization bit width directly impacts the model's storage requirements, computational complexity, and inference accuracy.

[0092] Specifically, the system can adopt a greedy strategy or heuristic algorithm to decide the quantization bit width. For example, for frequently activated experts, the system can retain a higher quantization bit width (such as 8 or 16 bits) to ensure inference accuracy; while for less frequently activated experts, the system can choose a lower quantization bit width (such as 4 bits or less) to save storage space and computing resources.

[0093] c. The quantization bit widths of each expert in all layers of the expert group form the mixed quantization decision of the local mixed expert model. In other words, the quantization bit widths of each expert in all layers of the expert group are combined to form the mixed quantization decision of the local mixed expert model. This decision represents the quantization bit width that each expert in the expert group at different layers should adopt.

[0094] In one possible implementation, regarding “determining the quantization bit width of each expert in each expert group according to the activation frequency of each expert in the expert group at that layer”, a specific implementation process may include:

[0095] a. Sort the experts in each level of the expert group in descending order of activation frequency. That is, the experts with the highest activation frequency are ranked first, and the experts with the lowest activation frequency are ranked last.

[0096] b. Based on the preset mixed quantization bit width set, the second half of the experts in each level of the expert group after descending sorting are assigned the highest quantization bit width in the mixed quantization bit width set. Assigning the highest quantization bit width is to ensure that the frequently activated experts have higher accuracy.

[0097] The quantization bit widths in the preset mixed quantization bit width set are arranged from large to small, that is, the preset mixed quantization bit width set is a set containing multiple quantization bit widths, and these quantization bit widths are arranged from large to small.

[0098] c. Based on the capacity limit of each edge server, determine whether the total quantized bit width of each layer of expert group after quantization exceeds the corresponding capacity limit.

[0099] It should be understood that the capacity limitations of each edge server may be storage space, computing power, etc.

[0100] d. If it exceeds, the quantization bit widths of all experts in each level of the expert group are reduced, and the quantization bit width of each expert is specifically reduced to the highest quantization bit width in the mixed quantization bit width set that is lower than the current bit width of the expert.

[0101] In other words, if the total quantization bit width of each expert group exceeds the corresponding capacity limit after quantization, the quantization bit width adjustment process begins. For each expert group, if its total quantization bit width exceeds the capacity limit, the quantization bit width of all experts in that expert group is reduced. Specifically, the quantization bit width of each expert is reduced to the highest quantization bit width in the mixed quantization bit width set that is lower than the expert's current bit width.

[0102] e. Repeat the judgment process until the total quantization bit width of each expert group does not exceed the corresponding capacity limit after quantization, and then decide the quantization bit width of each expert in the expert group at this level.

[0103] Specifically, the total quantized bit width of each expert group at each level after quantization is checked again to see if it exceeds the corresponding capacity limit. If it still does, the quantization bit width is further reduced until it meets the capacity limit. When the total quantized bit width of each expert group at each level after quantization no longer exceeds the corresponding capacity limit, the quantization bit width of each expert in that level is determined, and these quantized bit widths are used for subsequent edge inference services.

[0104] In one possible implementation, regarding “pre-training the policy network in reinforcement learning using the pre-training data to obtain an initialized policy network”, the specific implementation process can be as follows:

[0105] The inference requests randomly assigned to each edge server are used as the input to the policy network. The quantization bit widths of each expert in each layer of the expert group are used as the output labels of the policy network. These labels indicate the quantization bit width that each expert should use for a given inference request.

[0106] It should be understood that the policy network is a neural network that accepts inference requests as input and outputs the quantization bit width of each expert. The policy network is trained using pre-training data. During training, the system continuously adjusts the parameters of the policy network to minimize the difference between the output label (i.e., the decided quantization bit width) and the actual output of the policy network. The training process continues until the policy network converges. Once the policy network converges, it is initialized into a model that can decide the quantization bit width based on the inference request allocation scheme and the expert activation frequency.

[0107] Although the initialization policy network is able to determine the quantization bit width based on inference requests and expert activation frequency, it has not yet achieved optimal performance. Therefore, reinforcement learning algorithms (such as policy gradient methods and actor-critic algorithms) are subsequently used to further optimize the initialization policy network. During the optimization process, the system adjusts the policy network parameters based on feedback from service quality (such as accuracy, latency, and resource utilization).

[0108] In one implementation, the inference request of each user is input into the initialized policy network, and the allocation scheme of the inference request of each user to each edge server output by the current policy network and the expert hybrid quantization bit width scheme of each edge server are obtained. The specific implementation of this process is as follows:

[0109]

[0110] Where s t The mixed quantization bit width of experts on all current edge servers and the inference requests of each current user; is the expert hybrid quantization bit width scheme of edge server i at time t, in represents the data type (e.g., Int8 or Bf16) of the quantized data of expert j on edge server i at time t, with a total of m experts; E represents all edge servers; K t is the user reasoning request set at time t, k∈K t , where user inference request k is assigned to ID k s edge server; a t The quantization bit width of each expert on all edge servers at the next moment and the reasoning request of each user are finally assigned to the user with ID k s edge servers.

[0111] In one implementation, the current policy network outputs a method for allocating each user's inference request to each edge server, along with the expert hybrid quantization bit width scheme of each edge server, to serve each user's inference request and obtain the quality of service. The implementation process is as follows:

[0112] r t =∑ i {p i -α1·l i -α2·C i}

[0113] Where r t is the reward of the current policy network; p i For service accuracy; i is the service delay; C i is the switching cost of the expert hybrid quantization bit width scheme; α1 and α2 are parameters used to weigh the service accuracy, service delay and the switching cost of the expert hybrid quantization bit width scheme.

[0114] As a more preferred implementation, before continuing to execute the previously decided allocation scheme of each user's inference request to each edge server and the expert mixed quantization bit width scheme of each edge server, it also includes: monitoring the inference request of each current user for abnormalities; if the inference request of each current user undergoes abnormal changes, re-deciding the allocation scheme of each user's inference request to each edge server and the expert mixed quantization bit width scheme of each edge server based on the changed inference request.

[0115] Specifically, an adaptive deployment mechanism based on anomaly monitoring was designed, which can monitor the changes in the distribution of inference tasks on edge servers in real time, and automatically adjust the expert quantification strategy and task allocation method according to the changes in tasks. It can flexibly respond to dynamically changing task requirements and ensure high efficiency and low-latency services in long-term operation.

[0116] More specifically, developing the ability to monitor changes in the underlying environment is beneficial for the long-term operation of edge LLM deployments supported by reinforcement learning. It enables us to review feedback results to update deployment strategies to ensure low latency and high quality of edge LLM services. According to the Central Limit Theorem, no matter what the underlying true distribution is, as long as the sample is sufficiently "large" (i.e., size 30 or more), the sample mean can be considered to be approximately normally distributed. Therefore, assume that:

[0117]

[0118] For the most recent time interval Ω, a new average reward is obtained This will be used to determine whether the underlying deployment environment of the edge LLM service has changed. This embodiment uses Sigma event detection to achieve this goal. Specifically, if the newly observed average reward is within the 2-σ range, that is:

[0119] or

[0120] Assume that the observed event is rare (less than 5% probability) and identify it as an anomaly. In other words, continue to execute the current policy without retraining the reinforcement learning model. If it is observed that:

[0121]

[0122] Otherwise, the system believes that the underlying environment has changed and reinforcement learning should be retrained. It should be noted that the normal distribution of sample mean rewards can be accurately estimated using historical data.

[0123] This embodiment has carried out the operation experiment for a longer time, in order to simulate and observe Figure 5 The impact of the dynamic deployment strategy shown in Figure 2 is that the variance of the task distribution starts at 1 and is slowly increased by 0.1 after each batch of inference requests is processed. The system distributes inference requests at the beginning and deploys a mixed quantization model on the edge servers, then lets the system continue to process new inference requests. Figure 5 The following plot shows the rewards over time. It can be seen that increasing task diversity gradually weakens the effect of the original deployment. When the reward distribution falls into the 2-σ interval, the system detects a rare case (with a probability less than 5%). The system then makes new request allocation and quantification decisions based on the current task distribution. After appropriate redeployment, the average reward quickly returns to normal levels.

[0124] To evaluate the effectiveness of our method, we used user requests from samples of the following five datasets: Wiki, MNews, Samsum, PersonaChat, and Yelp. These samples were divided into training and testing parts. Five edge servers with varying size limits were set up. The training parts of the samples were used for pre-training and formal reinforcement learning training, and the testing parts were used for evaluation. Ten serving runs were conducted, each containing 50 user requests from the five datasets. These samples conformed to a normal distribution. The underlying model was Llama-MoE.

[0125] The present invention is compared with three different solutions:

[0126] ① Randomly distribute inference requests and quantize the model using a uniform bit width across all servers (RU): This solution only considers the size of each server for quantization; all experts are quantized to the same bit width.

[0127] ② Allocate inference requests according to task categories and use pruning (TP) to load MoE models: This solution configures a dedicated server for each task and uses pruning to load the model. Inference requests are allocated to the corresponding server based on the task type.

[0128] ③ Using a learning-based inference request allocation strategy similar to

[15] , all MoE experts use the same bit width for quantization (LU): Considering the size constraint, this solution first quantizes the models with the same bit width and uses a reinforcement learning-based solution to decide the allocation of requests.

[0129] Table 1 shows the average model performance results and total time cost after ten services. Ours in Table 1 refers to the solution of the present invention. It can be seen that the solution of the present invention has the best overall performance, balancing lower average perplexity and lower time cost. Although the TP solution is the least effective and most time-consuming, it performs the worst among the compared methods because the distribution of tasks is uneven. The task-based allocation strategy in TP causes tasks to pile up on some servers while other servers are idle. Compared with the quantization method, the direct expert pruning strategy reduces the quality of service. The LU solution saves some time compared to the random allocation strategy in RU, but they perform similarly in perplexity. Overall, the allocation-quantization collaborative design enables the edge LLM service of the present invention to improve the perplexity by about 8 while minimizing the time cost by about 20%.

[0130] Table 1 shows the model performance results and total time cost under different schemes

[0131]

[0132] In one embodiment of the present invention, a device for optimizing the deployment of a hybrid expert model in an edge computing environment is provided, which is used to implement the above-mentioned optimization method for optimizing the deployment of a hybrid expert model in an edge computing environment, including:

[0133] The acquisition module is configured to obtain the inference request of each current user.

[0134] The optimization module is configured to input the current inference requests of each user into a pre-trained optimization model, and output a distribution scheme of the inference requests of each user to each edge server, as well as an expert hybrid quantization bit width scheme of each edge server.

[0135] The training method of the optimization model includes:

[0136] Obtain inference requests from each user;

[0137] Constructing pre-training data using each user's inference request, the pre-training data including an inference request allocation randomly generated using each user's inference request, and an expert hybrid quantization bit width determined based on the expert activation frequency of the inference request allocation when each edge server serves the randomly generated inference request allocation;

[0138] Pre-training a policy network in reinforcement learning using the pre-training data to obtain an initialized policy network;

[0139] Inputting the inference request of each user into the initialized policy network, obtaining the allocation scheme of the inference request of each user to each edge server output by the current policy network, and the expert hybrid quantization bit width scheme of each edge server;

[0140] Using the allocation scheme of each user's inference request to each edge server output by the current policy network and the expert hybrid quantization bit width scheme of each edge server, each user's inference request is served to obtain the service quality;

[0141] The service quality is used as a reward for the current policy network, and the parameters of the current policy network are iteratively updated until the service quality meets the preset requirements, thereby obtaining the optimization model.

[0142] All relevant contents of each step involved in the embodiment of the aforementioned optimization method for deploying a hybrid expert model in an edge computing environment can be referred to the functional description of the functional module corresponding to the optimization device for deploying a hybrid expert model in an edge computing environment in the embodiment of the present invention, and will not be repeated here. The division of modules in the embodiment of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, the functional modules in various embodiments of the present invention can be integrated into one processor, or they can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0143] In another embodiment of the present invention, a computer device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of an optimization method for deploying a hybrid expert model in an edge computing environment.

[0144] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It can be understood that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the optimization method for deploying a hybrid expert model in an edge computing environment in the above embodiment.

[0145] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0147] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0149] In the present invention, the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0150] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An optimization method for deploying a hybrid expert model in an edge computing environment, characterized in that: include: Get the inference request of each current user; Input the current inference requests of each user into a pre-trained optimization model, and output a distribution plan of each user's inference requests to each edge server, as well as an expert hybrid quantization bit width plan for each edge server; The training method of the optimization model includes: Obtain inference requests from each user; Pre-training data is constructed using the inference requests of each user. The pre-training data includes the inference request allocation randomly generated using the inference request of each user, and the expert hybrid quantization bit width determined according to the expert activation frequency of the inference request allocation when each edge server serves the randomly generated inference request allocation. Specifically, it is: Inputting the inference request randomly assigned to each edge server into a local original hybrid expert model to obtain the activation frequency of each expert in the local original hybrid expert model; wherein the local original hybrid expert model includes a multi-layer expert group, and each layer of the expert group includes multiple experts; According to the activation frequency of each expert in each layer of expert group, the quantization bit width of each expert in the expert group at that layer is determined; The quantization bit widths of each expert in all layers of the expert group constitute the hybrid quantization decision of the local hybrid expert model; Pre-training a policy network in reinforcement learning using the pre-training data to obtain an initialized policy network; Inputting the inference request of each user into the initialized policy network, obtaining the allocation scheme of the inference request of each user to each edge server output by the current policy network, and the expert hybrid quantization bit width scheme of each edge server; Using the allocation scheme of each user's inference request to each edge server output by the current policy network and the expert hybrid quantization bit width scheme of each edge server, each user's inference request is served to obtain the service quality; The service quality is used as a reward for the current policy network, and the parameters of the current policy network are iteratively updated until the service quality meets the preset requirements, thereby obtaining the optimization model.

2. The optimization method for deploying a hybrid expert model in an edge computing environment according to claim 1, characterized in that: The service quality includes service accuracy, service delay and switching cost of the expert hybrid quantization bit width scheme.

3. The optimization method for deploying a hybrid expert model in an edge computing environment according to claim 2, characterized in that: The quantization bit width of each expert in each expert group is determined according to the activation frequency of each expert in the expert group at that level, specifically: Sort the experts in each level of expert group in descending order according to the activation frequency; According to a preset mixed quantization bit width set, assigning the highest quantization bit width in the mixed quantization bit width set to the second half of the experts in each layer of the expert group after descending sorting; wherein the quantization bit widths in the preset mixed quantization bit width set are arranged from largest to smallest; According to the capacity limit of each edge server, determine whether the total quantized bit width of each layer of expert group exceeds the corresponding capacity limit; If it exceeds, the quantization bit widths of all experts in each layer of the expert group will be reduced, and the quantization bit width of each expert will be reduced to the highest quantization bit width in the mixed quantization bit width set that is lower than the current bit width of the expert; The judgment process is repeated until the total quantization bit width of each expert group at each level does not exceed the corresponding capacity limit after quantization, and the quantization bit width of each expert in the expert group at this level is decided.

4. The optimization method for deploying a hybrid expert model in an edge computing environment according to claim 3, characterized in that: The method of using the pre-training data to pre-train the policy network in the reinforcement learning to obtain the initialized policy network is specifically as follows: The inference requests randomly assigned to each edge server are used as the input of the policy network. The quantization bit width of each expert in each layer of expert group, determined according to the activation frequency of each expert in each layer of expert group, is used as the output label of the policy network. The policy network is pre-trained until convergence to obtain the initialized policy network.

5. The optimization method for deploying a hybrid expert model in an edge computing environment according to claim 1, characterized in that: The inference request of each user is input into the initialized policy network to obtain the allocation scheme of the inference request of each user output by the current policy network to each edge server, and the expert hybrid quantization bit width scheme of each edge server, specifically: Where s t The mixed quantization bit width of experts on all current edge servers and the inference requests of each current user; is the expert hybrid quantization bit width scheme of edge server i at time t, in represents the data type quantified by expert j on edge server i at time t, with a total of m experts; E represents all edge servers; K t is the user reasoning request set at time t, k∈K t , where user inference request k is assigned to ID k s edge server; a t The quantization bit width of each expert on all edge servers at the next moment and the reasoning request of each user are finally assigned to the user with ID k s edge servers.

6. The optimization method for deploying a hybrid expert model in an edge computing environment according to claim 1, characterized in that: The method uses the allocation scheme of each user's inference request output by the current policy network to each edge server, and the expert hybrid quantization bit width scheme of each edge server to serve each user's inference request and obtain the service quality, which is specifically: r t =∑ i {p i -α1·l i -α2·C i } Where r t is the reward of the current policy network; p i For service accuracy; i is the service delay; C i is the switching cost of the expert hybrid quantization bit width scheme; α1 and α2 are parameters used to weigh the service accuracy, service delay and the switching cost of the expert hybrid quantization bit width scheme.

7. An optimization device for deploying a hybrid expert model in an edge computing environment, characterized in that: include: The acquisition module is used to obtain the inference requests of each current user; An optimization module, configured to input the inference requests of each current user into a pre-trained optimization model, and output a distribution scheme of the inference requests of each user to each edge server, as well as an expert hybrid quantization bit width scheme for each edge server; The training method of the optimization model includes: Obtain inference requests from each user; Pre-training data is constructed using the inference requests of each user. The pre-training data includes the inference request allocation randomly generated using the inference request of each user, and the expert hybrid quantization bit width determined according to the expert activation frequency of the inference request allocation when each edge server serves the randomly generated inference request allocation. Specifically, it is: Inputting the inference request randomly assigned to each edge server into a local original hybrid expert model to obtain the activation frequency of each expert in the local original hybrid expert model; wherein the local original hybrid expert model includes a multi-layer expert group, and each layer of the expert group includes multiple experts; According to the activation frequency of each expert in each layer of expert group, the quantization bit width of each expert in the expert group at that layer is determined; The quantization bit widths of each expert in all layers of the expert group constitute the hybrid quantization decision of the local hybrid expert model; Pre-training a policy network in reinforcement learning using the pre-training data to obtain an initialized policy network; Inputting the inference request of each user into the initialized policy network, obtaining the allocation scheme of the inference request of each user to each edge server output by the current policy network, and the expert hybrid quantization bit width scheme of each edge server; Using the allocation scheme of each user's inference request to each edge server output by the current policy network and the expert hybrid quantization bit width scheme of each edge server, each user's inference request is served to obtain the service quality; The service quality is used as a reward for the current policy network, and the parameters of the current policy network are iteratively updated until the service quality meets the preset requirements, thereby obtaining the optimization model.

8. A device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the optimization method for deploying a hybrid expert model in an edge computing environment as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements an optimization method for deploying a hybrid expert model in an edge computing environment as described in any one of claims 1 to 6.

Images