Optimization method for deploying hybrid expert model in edge computing environment and related equipment

Abstract

The invention discloses an optimization method for deploying a hybrid expert model in an edge computing environment and related equipment, and the method comprises the steps: receiving reasoning requests of all users through a pre-trained optimization model, and outputting a distribution scheme of the requests to all edge servers, and an expert hybrid quantization bit width scheme of all servers. The training process of the optimization model comprises the steps that user reasoning requests are collected, pre-training data are constructed, and the pre-training data comprise randomly-generated reasoning request distribution and corresponding expert mixed quantization bit width; pre-training a strategy network in reinforcement learning by using the data; inputting a user reasoning request into the initialization strategy network to obtain a distribution scheme and a quantization bit width scheme; according to the schemes, a user reasoning request is served to obtain service quality, the service quality is used as an award of a strategy network, and network parameters are iteratively updated until the service quality reaches a preset requirement. The invention aims to realize high-precision and low-delay edge reasoning service by dynamically adjusting the expert quantization bit width of the edge server and combining the distribution optimization of the user reasoning request.

Description

Technical Field

[0001] The present invention belongs to the field of edge computing deployment of large language models, and specifically relates to an optimization method and related equipment for deploying a hybrid expert model in an edge computing environment. Background Art

[0002] With the continuous advancement and widespread application of large language model (LLM) technology, it has shown great potential and value in many fields such as natural language processing, artificial intelligence assistants, and intelligent customer service. The current mainstream deployment method usually relies on hosting LLM in a cloud data center and processing the user's reasoning request through the powerful computing power of the cloud. The user sends the request to the cloud through the network, and the cloud completes the reasoning calculation and then returns the result to the user. Although this cloud-based deployment model can provide powerful computing services, it also brings about network latency problems. In addition, since the user's reasoning request data needs to be transmitted to the cloud for processing, this increases the risk of user privacy leakage to a certain extent and poses a potential threat to data security. In order to solve the above problems, the industry has begun to explore the possibility of directly deploying large-scale language models at the edge of the network in recent years. Edge computing can significantly shorten the response time and improve the user experience by sinking computing tasks from the cloud to the edge of the network. At the same time, edge computing can also enhance the protection of user privacy to a certain extent, because the user's reasoning request data no longer needs to be transmitted to the cloud, thereby reducing the risk of data leakage. However, the hardware resources in the edge computing environment are relatively limited, which is in sharp contrast to the huge model size of LLM, which brings great challenges to the efficient deployment of LLM on edge devices.

[0003] To meet this challenge, researchers proposed an LLM architecture called "Mixture-of-Experts (MoE)". The MoE model significantly reduces the computational overhead during inference by splitting the large model into multiple smaller expert models and activating only some of the experts at each time of inference. This sparse activation feature makes the MoE model very suitable for resource-constrained edge environments. Therefore, how to optimize the deployment of the MoE model at the edge has become a current research hotspot. Currently, in the process of optimizing the edge deployment of the MoE model, the quantization technology of the expert model has gradually become a key technology. Quantization technology can reduce the storage requirements and computational complexity of the MoE model by compressing the weights of the MoE model, so that it can run on edge devices with limited hardware resources. However, the existing MoE model quantization methods still have many shortcomings.

[0004] On the one hand, existing MoE model quantization methods usually adopt a unified quantization strategy, such as converting all expert weights into Int8 format. However, the expert models in the MoE model have different activation frequencies and importance during the reasoning process, so the use of a unified quantization strategy may not be able to give full play to the performance advantages of each expert model. For example, for those expert models that are frequently activated, a higher precision (such as Bf16) should be retained to ensure the accuracy of reasoning; while for those expert models that are not frequently used, they can be quantized to a lower precision (such as Int4 or Int8) to further compress the model size. In other words, a unified quantization strategy cannot reduce the storage and computational overhead of the MoE model while maintaining a high reasoning accuracy.

[0005] On the other hand, existing MoE model quantization methods do not fully consider how heterogeneously quantized MoE models affect the distribution of user requests among edge servers. In edge computing environments, different inference requests may activate different sets of expert models, so they may benefit from different quantization schemes. However, existing MoE model quantization methods are not optimized for this feature, resulting in inefficient distribution of user requests among edge servers. Summary of the invention

[0006] In response to the problems existing in the prior art, the present invention provides an optimization method and related equipment for deploying a hybrid expert model in an edge computing environment, which aims to achieve high-precision, low-latency edge reasoning services by dynamically adjusting the expert quantization bit width on the edge server and combining the allocation optimization of user reasoning requests.

[0007] In order to solve the above technical problems, the present invention is implemented by the following technical solutions:

[0008] According to a first aspect of the present invention, there is provided an optimization method for deploying a hybrid expert model in an edge computing environment, comprising:

[0009] Get the inference request of each current user;

[0010] Input the current reasoning requests of each user into a pre-trained optimization model, and output a distribution scheme of the reasoning requests of each user to each edge server, and an expert hybrid quantization bit width scheme of each edge server;

[0011] Wherein, the training method of the optimization model includes:

[0012] Obtain inference requests from each user;

[0013] Constructing pre-training data using the inference requests of each user, wherein the pre-training data includes an inference request allocation randomly generated using the inference requests of each user, and an expert hybrid quantization bit width determined according to an expert activation frequency of the inference request allocation when each edge server serves the randomly generated inference request allocation;

[0014] Pre-training a policy network in reinforcement learning using the pre-training data to obtain an initialized policy network;

[0015] Inputting the reasoning request of each user into the initialized policy network, obtaining the allocation scheme of the reasoning request of each user output by the current policy network to each edge server, and the expert hybrid quantization bit width scheme of each edge server;

[0016] Using the allocation scheme of each user's reasoning request to each edge server output by the current policy network and the expert mixed quantization bit width scheme of each edge server, each user's reasoning request is served to obtain the service quality;

[0017] The service quality is used as a reward for the current policy network, and the parameters of the current policy network are iteratively updated until the service quality reaches a preset requirement, thereby obtaining the optimization model.

[0018] In a possible implementation manner of the first aspect, the service quality includes service accuracy, service delay, and a switching cost of an expert hybrid quantization bit width solution.

[0019] In a possible implementation manner of the first aspect, when each edge server serves the randomly generated inference request allocation, the expert hybrid quantization bit width determined according to the expert activation frequency allocated to the inference request is specifically:

[0020] Inputting the inference request randomly assigned to each edge server into the local original hybrid expert model to obtain the activation frequency of each expert in the local original hybrid expert model; wherein the local original hybrid expert model includes a multi-layer expert group, and each layer of the expert group includes a plurality of experts;

[0021] According to the activation frequency of each expert in each layer of the expert group, the quantization bit width of each expert in the expert group of this layer is decided;

[0022] The quantization bit widths of each expert in all layers of the expert group constitute the mixed quantization decision of the local mixed expert model.

[0023] In a possible implementation manner of the first aspect, the quantization bit width of each expert in each layer of the expert group is determined according to the activation frequency of each expert in the expert group of that layer, specifically:

[0024] Sort the experts in each expert group in descending order according to the activation frequency;

[0025] According to a preset mixed quantization bit width set, assigning the highest quantization bit width in the mixed quantization bit width set to the second half of the experts in each layer of the expert group after being sorted in descending order; wherein the quantization bit widths in the preset mixed quantization bit width set are arranged from large to small;

[0026] According to the capacity limit of each edge server, determine whether the total quantized bit width of each layer of expert group after quantization exceeds the corresponding capacity limit;

[0027] If it exceeds, the quantization bit widths of all experts in each layer of the expert group are reduced, and the quantization bit width of each expert is specifically reduced to the highest quantization bit width in the mixed quantization bit width set that is lower than the current bit width of the expert;

[0028] The judgment process is repeated until the total quantization bit width of each layer of expert group after quantization does not exceed the corresponding capacity limit, and then the quantization bit width of each expert in the expert group of this layer is decided.

[0029] In a possible implementation manner of the first aspect, using the pre-training data to pre-train a policy network in reinforcement learning to obtain an initialized policy network is specifically:

[0030] The inference requests randomly assigned to each edge server are used as the input of the policy network. The quantization bit width of each expert in each expert group in each layer, which is determined according to the activation frequency of each expert in each layer of the expert group, is used as the output label of the policy network. The policy network is pre-trained until convergence to obtain an initialized policy network.

[0031] In a possible implementation manner of the first aspect, the inputting the inference request of each user into the initialized policy network to obtain the allocation scheme of the inference request of each user output by the current policy network to each edge server, and the expert hybrid quantization bit width scheme of each edge server, are specifically:

[0032]

[0033] In the formula, s t It contains the expert mixed quantization bit width on all current edge servers and the inference requests of each current user; is the expert hybrid quantization bit width scheme of edge server i at time t, in represents the data type quantified by expert j on edge server i at time t, with a total of m experts; E represents all edge servers; K t is the user reasoning request set at time t, k∈K t , where user reasoning request k is assigned to ID k s edge server; a tThe quantized bit width of each expert on all edge servers at the next moment and the reasoning request of each user are finally assigned to the user with ID k s edge server.

[0034] In a possible implementation manner of the first aspect, the allocation scheme of the inference request of each user output by the current policy network to each edge server, and the expert hybrid quantization bit width scheme of each edge server, serve the inference request of each user to obtain the service quality, specifically:

[0035] r t =∑ i {p i -α 1 ·l i -α 2 ·C i}

[0036] In the formula, r t is the reward of the current policy network; p i To serve accuracy; i is the service delay; C i is the switching cost of the expert hybrid quantization bit width scheme; α1 and α2 are parameters used to weigh the service accuracy, service delay and the switching cost of the expert hybrid quantization bit width scheme.

[0037] According to a second aspect of the present invention, there is provided an optimization device for deploying a hybrid expert model in an edge computing environment, comprising:

[0038] The acquisition module is used to obtain the inference requests of each current user;

[0039] An optimization module, used to input the current reasoning requests of each user into a pre-trained optimization model, and output a distribution scheme of the reasoning requests of each user to each edge server, and an expert hybrid quantization bit width scheme of each edge server;

[0040] Wherein, the training method of the optimization model includes:

[0041] Obtain inference requests from each user;

[0042] Constructing pre-training data using the inference requests of each user, wherein the pre-training data includes an inference request allocation randomly generated using the inference requests of each user, and an expert hybrid quantization bit width determined according to an expert activation frequency of the inference request allocation when each edge server serves the randomly generated inference request allocation;

[0043] Pre-training a policy network in reinforcement learning using the pre-training data to obtain an initialized policy network;

[0044] Inputting the reasoning request of each user into the initialized policy network, obtaining the allocation scheme of the reasoning request of each user output by the current policy network to each edge server, and the expert hybrid quantization bit width scheme of each edge server;

[0045] Using the allocation scheme of each user's reasoning request to each edge server output by the current policy network and the expert mixed quantization bit width scheme of each edge server, each user's reasoning request is served to obtain the service quality;

[0046] The service quality is used as a reward for the current policy network, and the parameters of the current policy network are iteratively updated until the service quality reaches a preset requirement, thereby obtaining the optimization model.

[0047] According to a third aspect of the present invention, a device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the optimization method for deploying a hybrid expert model in an edge computing environment is implemented.

[0048] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the optimization method for deploying a hybrid expert model in an edge computing environment is implemented.

[0049] Compared with the prior art, the present invention has at least the following beneficial effects:

[0050] The present invention provides an optimization method for deploying a hybrid expert model in an edge computing environment. By adopting a hybrid expert quantization strategy, the quantization bit width of different experts on each edge server is dynamically adjusted according to the frequency of reasoning requests and task information. Experts activated at high frequencies maintain higher accuracy, while experts activated at low frequencies use low-precision quantization. While significantly reducing model storage and computing resources, high reasoning accuracy is maintained, solving the problem of reduced reasoning accuracy and low storage resource utilization caused by unified quantization. The hybrid quantization strategy of expert models combined with task allocation proposed in the present invention takes into account the impact of the distribution of reasoning requests on the selection of hybrid quantization schemes. By learning different quantization schemes under different requirements, the joint optimization of expert quantization and reasoning request allocation is achieved, ensuring high accuracy and low latency of reasoning services.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the specific implementation modes of the present invention, the drawings required for use in the description of the specific implementation modes will be briefly introduced below. Obviously, the drawings described below are some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 A flowchart of an optimization method for deploying a hybrid expert model in an edge computing environment according to an embodiment of the present invention;

[0054] Figure 2 is a schematic diagram of the MoE model;

[0055] Figure 3 A typical scene diagram in the embodiment;

[0056] Figure 4 A comparison diagram of the reinforcement learning convergence process with and without pre-training design in the embodiment;

[0057] Figure 5 is the average reward value that changes over time in the embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] The hybrid expert model takes advantage of the sparse activation characteristics of the large language model during single-task reasoning and uses only a small number of multi-layer perceptrons for efficient and accurate reasoning, that is, converting the traditional dense layers into multiple expert models, such as Figure 2 A typical edge scenario includes multiple edge servers and multiple user requirements, such as Figure 3 shown.

[0060] Combination Figure 1 As shown, an embodiment of the present invention provides an optimization method for deploying a hybrid expert model in an edge computing environment. The method dynamically adjusts the expert quantization bit width on the edge server and combines the allocation optimization of user reasoning requests to achieve high-precision, low-latency edge reasoning services. The specific steps of the optimization method are as follows:

[0061] Step 1: Get the inference request of each current user.

[0062] Specifically, in an edge computing environment, each user submits their reasoning requests to the system. These requests can include various types of processing tasks such as text generation, language understanding, question answering, and image recognition. The system collects these reasoning requests as input for subsequent steps.

[0063] Step 2: Input the current reasoning requests of each user into a pre-trained optimization model, and output the allocation scheme of each user's reasoning requests to each edge server, as well as the expert hybrid quantization bit width scheme of each edge server.

[0064] That is to say, the currently collected inference requests of each user are input into the pre-trained optimization model, which is a policy network based on reinforcement learning, responsible for outputting the allocation plan of each user's inference request to each edge server, as well as the expert mixed quantization bit width plan of each edge server.

[0065] In an embodiment of the present invention, the training method of the optimization model specifically includes the following steps:

[0066] a. Obtain the inference request of each user.

[0067] Specifically, a large number of users' inference requests are obtained from historical data as the basis for building pre-training data.

[0068] b. construct pre-training data using the inference requests of each user, wherein the pre-training data includes an inference request allocation randomly generated using the inference requests of each user, and an expert mixed quantization bit width determined according to the expert activation frequency of the inference request allocation when each edge server serves the randomly generated inference request allocation.

[0069] That is to say, for each inference request, a different inference request allocation scheme is randomly generated, that is, which edge server should handle each inference request. At the same time, according to each inference request allocation scheme, the expert activation frequency in the inference process is simulated, and based on these activation frequencies, the expert mixed quantization bit width scheme is decided, which will be used as the label of the pre-training data.

[0070] c. Pre-training the policy network in reinforcement learning using the pre-training data to obtain an initialized policy network.

[0071] That is, using the above pre-training data, the policy network in the reinforcement learning is pre-trained to obtain an initialized policy network. The initialized policy network will learn how to generate a reasonable inference request allocation scheme and an expert hybrid quantization bit width scheme according to the characteristics of the inference request.

[0072] For example, Figure 4This is a comparison chart of the reinforcement learning convergence process with and without pre-training design. The results show that pre-training design can effectively accelerate the convergence of reinforcement learning.

[0073] d. Input the reasoning request of each user into the initialized policy network to obtain the allocation scheme of the reasoning request of each user to each edge server output by the current policy network, and the expert mixed quantization bit width scheme of each edge server.

[0074] e. Use the allocation scheme of each user's inference request to each edge server output by the current policy network, and the expert mixed quantization bit width scheme of each edge server to serve the inference request of each user and obtain the service quality.

[0075] f. Using the service quality as a reward for the current policy network, iteratively updating the parameters of the current policy network until the service quality reaches a preset requirement, thereby obtaining the optimization model.

[0076] Specifically, steps d, e, and f are the process of iteratively updating the policy network. New inference requests are input into the initialized policy network to obtain the allocation scheme and quantization bit width scheme output by the current policy network. According to these schemes, the inference requests of each user are served, and service quality indicators such as response time and inference accuracy are collected. The service quality is used as the reward of the current policy network, and the parameters of the current policy network are iteratively updated using a reinforcement learning algorithm (such as the policy gradient method). The above process is repeated until the service quality meets the preset requirements. At this time, the obtained policy network is the optimization model.

[0077] It should be understood that after the optimization model output, the system obtains the allocation scheme of each user's reasoning request to each edge server, as well as the expert mixed quantization bit width scheme of each edge server. According to these schemes, the system allocates the reasoning request to the corresponding edge server, and quantizes the expert model on the edge server according to the specified quantization bit width scheme. Each edge server performs the reasoning service according to the allocated reasoning request and the quantized expert model. Due to the use of a dynamically adjusted quantization bit width scheme, the system can reduce storage and computing overhead while maintaining high reasoning accuracy, and realize high-precision, low-latency edge reasoning services.

[0078] This paper proposes an optimization method for deploying a hybrid expert model in an edge computing environment by combining reinforcement learning and dynamic quantization technology. This method can dynamically adjust the expert quantization bit width on the edge server, and combine the allocation optimization of user reasoning requests to achieve high-precision, low-latency edge reasoning services. By training a policy network based on reinforcement learning, the system can learn how to generate reasonable allocation schemes and quantization bit width schemes based on the characteristics of the reasoning request, thereby achieving good results in practical applications.

[0079] In an implementable manner, the service quality includes service accuracy, service delay, and switching cost of the expert hybrid quantization bit width scheme. That is, the method not only considers service accuracy and service delay, but also introduces the switching cost of the expert hybrid quantization bit width scheme (such as the computing resources and time required to switch from one quantization scheme to another) to comprehensively evaluate and optimize the service quality.

[0080] It should be noted that in the edge computing environment, the requirements of service accuracy and latency for the inference requests of each user may be different. The optimization model is a policy network based on reinforcement learning, which is responsible for comprehensively considering the service accuracy, service latency and the switching cost of the expert hybrid quantization bit width scheme, and outputting the allocation scheme of each user's inference request to each edge server, as well as the expert hybrid quantization bit width scheme of each edge server.

[0081] In detail, in this embodiment, the training method for the optimization model is as follows:

[0082] Obtain inference requests: Obtain a large number of users’ inference requests from historical data. These requests should contain various task types and different requirements for service quality.

[0083] Construct pre-training data: For each inference request, randomly generate different inference request allocation schemes and simulate the expert activation frequency during the inference process. At the same time, based on the activation frequency and the preset quantization strategy, generate multiple possible expert mixed quantization bit width schemes. For each scheme, calculate its corresponding service accuracy, service latency, and switching cost.

[0084] Pre-trained policy network: Use the above pre-training data to pre-train the policy network in reinforcement learning. The policy network will learn how to generate a reasonable inference request allocation scheme and expert mixed quantization bit width scheme based on the characteristics of the inference request (such as task type, service quality requirements, etc.) to maximize the service quality.

[0085] Iteratively update the policy network: Input the new reasoning request into the initialized policy network to obtain the allocation scheme and quantization bit width scheme output by the current policy network. According to these schemes, serve the reasoning requests of each user, and collect the actual service accuracy, service delay and switching cost data. Use these service quality data as rewards (or penalties) for the current policy network, and use reinforcement learning algorithms (exemplarily, deep deterministic policy gradient DDPG, proximal policy optimization PPO, etc.) to iteratively update the parameters of the current policy network. During the update process, it should be ensured that the policy network, while pursuing high accuracy and low latency, also considers reducing the switching cost of the quantization bit width scheme to achieve the optimization of the overall service quality. Repeat the above process until the service quality (comprehensively considering service accuracy, service delay and switching cost) meets the preset requirements or converges to a stable state. At this time, the obtained policy network is the optimization model.

[0086] In one possible implementation, regarding “when each edge server serves a randomly generated inference request allocation, the expert mixed quantization bit width is determined according to the expert activation frequency allocated to the inference request”, the specific implementation process is as follows:

[0087] a. Inputting the inference request randomly assigned to each edge server into the local original hybrid expert model to obtain the activation frequency of each expert in the local original hybrid expert model; wherein the local original hybrid expert model includes a multi-layer expert group, and each layer of the expert group includes multiple experts.

[0088] Specifically, for each inference request randomly assigned to an edge server, the system inputs it into the local original hybrid expert model. The local original hybrid expert model includes multiple layers of expert groups, each layer of expert groups contains multiple experts, and these experts are selectively activated to complete the inference task according to different input data.

[0089] The system records the activation frequency of each expert in the local original hybrid expert model for each reasoning request. It should be noted that the activation frequency refers to the ratio of the number of times a certain expert is called when processing the reasoning request to the total number of reasoning requests. This frequency reflects the importance and activity of the expert in processing a specific type of reasoning request.

[0090] b. According to the activation frequency of each expert in each layer of the expert group, the quantization bit width of each expert in the expert group of this layer is determined.

[0091] Specifically, according to the activation frequency of each expert in each layer of the expert group, the system determines the quantization bit width of each expert in the expert group at that layer. It should be noted that the quantization bit width refers to the number of bits used when converting the weights of the expert model from a high-precision format to a low-precision format. The choice of quantization bit width directly affects the storage requirements, computational complexity, and inference accuracy of the model.

[0092] Specifically, the system can use a greedy strategy or a heuristic algorithm to decide the quantization bit width. For example, for frequently activated experts, the system can retain a higher quantization bit width (such as 8 bits or 16 bits) to ensure reasoning accuracy; while for less activated experts, the system can choose a lower quantization bit width (such as 4 bits or less) to save storage space and computing resources.

[0093] c. The quantization bit widths of each expert in all layers of the expert group form the mixed quantization decision of the local mixed expert model. In other words, the quantization bit widths of each expert in all layers of the expert group are combined to form the mixed quantization decision of the local mixed expert model, which represents the quantization bit width that each expert should adopt in the expert group of different layers.

[0094] In one achievable manner, regarding “determining the quantization bit width of each expert in each layer of the expert group according to the activation frequency of each expert in the expert group at that layer”, the specific implementation process may include:

[0095] a. Sort the experts in each expert group in descending order according to their activation frequency. That is, the experts with the highest activation frequency are ranked first, and the experts with the lowest activation frequency are ranked last.

[0096] b. According to the preset mixed quantization bit width set, the second half of the experts in each level of the expert group after descending sorting are assigned the highest quantization bit width in the mixed quantization bit width set. The purpose of assigning the highest quantization bit width is to ensure that the frequently activated experts have higher accuracy.

[0097] The quantization bit widths in the preset mixed quantization bit width set are arranged from large to small, that is, the preset mixed quantization bit width set is a set containing multiple quantization bit widths, and these quantization bit widths are arranged from large to small.

[0098] c. According to the capacity limit of each edge server, determine whether the total quantized bit width of each layer of expert group after quantization exceeds the corresponding capacity limit.

[0099] It should be understood that the capacity limitation of each edge server may be storage space, computing power, etc.

[0100] d. If it exceeds, the quantization bit widths of all experts in each layer of the expert group are reduced, and the quantization bit width of each expert is specifically reduced to the highest quantization bit width in the mixed quantization bit width set that is lower than the current bit width of the expert.

[0101] That is to say, if the total quantization bit width of each layer of expert groups exceeds the corresponding capacity limit after quantization, the process of adjusting the quantization bit width will be entered. For each layer of expert groups, if its total quantization bit width exceeds the capacity limit, the quantization bit width of all experts in the expert group of this layer will be reduced. Specifically, the quantization bit width of each expert is reduced to the highest quantization bit width in the mixed quantization bit width set that is lower than the current bit width of the expert.

[0102] e. Repeat the judgment process until the total quantization bit width of each layer of expert group after quantization does not exceed the corresponding capacity limit, and then decide the quantization bit width of each expert in the expert group of this layer.

[0103] That is to say, check again whether the total quantization bit width of each layer of expert groups after quantization exceeds the corresponding capacity limit. If it still exceeds, continue to reduce the quantization bit width until the capacity limit is met. When the total quantization bit width of each layer of expert groups after quantization no longer exceeds the corresponding capacity limit, the quantization bit width of each expert in the expert group at this layer is determined, and these quantization bit widths will be used for subsequent edge inference services.

[0104] In one achievable manner, regarding “pre-training a policy network in reinforcement learning using the pre-training data to obtain an initialized policy network”, the specific implementation process may be as follows:

[0105] The inference request randomly assigned to each edge server is used as the input of the policy network. The quantization bit width of each expert in each layer of the expert group is used as the output label of the policy network. These labels indicate the quantization bit width that each expert should use under a given inference request.

[0106] It should be understood that the policy network is a neural network that accepts inference requests as input and outputs the quantization bit width of each expert. The policy network is trained using pre-trained data. During training, the system continuously adjusts the parameters of the policy network to minimize the difference between the output label (i.e., the decided quantization bit width) and the actual output of the policy network. The training process continues until the policy network converges. When the policy network converges, it is initialized as a model that can decide the quantization bit width based on the inference request allocation scheme and the expert activation frequency.

[0107] Although the initialization policy network is able to decide the quantization bit width based on the inference request and the expert activation frequency, it has not yet achieved the optimal performance. Therefore, reinforcement learning algorithms (such as policy gradient method, actor-critic algorithm, etc.) are used to further optimize the initialization policy network. During the optimization process, the system adjusts the parameters of the policy network based on the feedback of service quality (such as accuracy, latency, and resource utilization).

[0108] In one achievable manner, the reasoning request of each user is input into the initialized policy network, and the allocation scheme of the reasoning request of each user output by the current policy network to each edge server and the expert hybrid quantization bit width scheme of each edge server are obtained. The specific implementation of the process is as follows:

[0109]

[0110] In the formula, s t It contains the expert mixed quantization bit width on all current edge servers and the inference requests of each current user; is the expert hybrid quantization bit width scheme of edge server i at time t, in represents the data type (e.g., Int8 or Bf16) of the expert j on the edge server i after quantization at time t, with a total of m experts; E represents all edge servers; K t is the user reasoning request set at time t, k∈K t , where user reasoning request k is assigned to ID k s edge server; a t The quantized bit width of each expert on all edge servers at the next moment and the reasoning request of each user are finally assigned to the user with ID k s edge server.

[0111] In one possible implementation, the inference request of each user output by the current policy network is used to allocate the inference request to each edge server, and the expert mixed quantization bit width scheme of each edge server is used to serve the inference request of each user and obtain the service quality. The specific implementation of the process is as follows:

[0112] r t =∑ i {p i -α1·l i -α2·C i}

[0113] In the formula, r t is the reward of the current policy network; p i To serve accuracy; i is the service delay; C i is the switching cost of the expert hybrid quantization bit width scheme; α1 and α2 are parameters used to weigh the service accuracy, service delay and the switching cost of the expert hybrid quantization bit width scheme.

[0114] As a more preferred implementation, before continuing to execute the previously decided allocation scheme of each user's inference request to each edge server and the expert mixed quantization bit width scheme of each edge server, it also includes: performing abnormal monitoring on the current inference request of each user; if the current inference request of each user changes abnormally, re-deciding the allocation scheme of each user's inference request to each edge server and the expert mixed quantization bit width scheme of each edge server according to the changed inference request.

[0115] Specifically, an adaptive deployment mechanism based on anomaly monitoring was designed, which can monitor the changes in the distribution of inference tasks of edge servers in real time, and automatically adjust the expert quantization strategy and task allocation method according to the changes in tasks. It can flexibly respond to dynamically changing task requirements and ensure high efficiency and low-latency services in long-term operation.

[0116] More specifically, developing the ability to monitor changes in the underlying environment is beneficial for the long-term operation of edge LLM deployments supported by reinforcement learning. It enables us to revisit feedback results to update deployment strategies to ensure low latency and high quality of edge LLM services. The average reward at a given time interval Ω is set to According to the Central Limit Theorem, no matter what the underlying true distribution is, as long as the sample is "large" enough (i.e., size 30 or more), the sample mean can be considered to be approximately normally distributed. Therefore, assume that:

[0117]

[0118] For the most recent time interval Ω, a new average reward is obtained This will be used to determine whether the underlying deployment environment of the edge LLM service has changed. This embodiment uses Sigma event detection to achieve this goal. Specifically, if the newly observed average reward is within the 2-σ range, that is:

[0119] or

[0120] Assume that the observed event is rare (less than 5% probability) and identify it as an anomaly. In other words, continue to execute the current policy without retraining the reinforcement learning model. If it is observed that:

[0121]

[0122] Otherwise, the system believes that the underlying environment has changed and reinforcement learning should be retrained. It should be noted that the normal distribution of sample mean rewards can be accurately estimated by historical data.

[0123] This embodiment has carried out the operation experiment for a longer time, in order to simulate and observe such Figure 5 The impact of the dynamic deployment strategy shown. The variance of the task distribution is set to start from 1 and only slowly increase by 0.1 after each batch of inference requests are processed. The system distributes inference requests at the beginning and deploys the hybrid quantization mode on the edge servers, and then lets the system continue to process new inference requests. Figure 5 The rewards over time are shown. It can be seen that the increase in task diversity gradually weakens the effect of the original deployment. When the reward distribution falls into the 2-σ interval, the system detects a rare case (with a probability of less than 5%). Subsequently, the system makes new request allocation and quantization decisions based on the current task distribution. After appropriate redeployment, the average reward quickly returns to normal levels.

[0124] To evaluate the effectiveness of the present invention, samples from the following five datasets are used as user requests: Wiki, MNews, Samsum, PersonaChat, Yelp. These samples are divided into training and testing parts. Five edge servers with different size limits are set up. The training part of the samples is used for pre-training and formal reinforcement learning training, and then the test part is used for evaluation. There are ten services in total, each containing 50 user requests from five datasets. These samples conform to the normal distribution. The basic model is Llama-MoE.

[0125] The present invention is compared with three different solutions:

[0126] ① Randomly assign inference requests and quantize the model using a uniform bit width for all servers (RU): This solution only considers the size of each server for quantization; all experts are quantized to the same bit width.

[0127] ② Allocate inference requests according to task categories and use pruning (TP) to load the MoE model: This solution configures a dedicated server for each task and uses pruning to load the model. Inference requests are allocated to the corresponding server according to the task type.

[0128] ③ Using a learning-based inference request allocation strategy similar to that in [5], all MoE experts use the same bit width for quantization (LU): Considering the size constraint, this solution first quantizes the models with the same bit width and uses a reinforcement learning-based solution to decide the allocation of requests.

[0129] Table 1 shows the average model performance results and total time cost after ten services, and Ours in Table 1 refers to the solution of the present invention. It can be seen that the solution of the present invention has the best overall performance, balancing lower average perplexity and lower time cost. Although the TP solution is the most ineffective and time-consuming, it performs the worst among the compared methods because of the uneven distribution of tasks. The task-based allocation strategy in TP causes tasks to pile up on some servers while other servers are idle. Compared with the quantization method, the direct expert pruning strategy reduces the quality of service. The LU solution saves some time compared to the random allocation strategy in RU, but they perform similarly in perplexity. Overall, the allocation-quantization collaborative design enables the edge LLM service of the present invention to improve perplexity by about 8 while minimizing the time cost by about 20%.

[0130] Table 1 shows the model performance results and total time cost under different schemes

[0131]

[0132] In one embodiment of the present invention, a device for optimizing the deployment of a hybrid expert model in an edge computing environment is provided, which is used to implement the above-mentioned method for optimizing the deployment of a hybrid expert model in an edge computing environment, including:

[0133] The acquisition module is configured to acquire the current inference requests of each user.

[0134] The optimization module is configured to input the current reasoning requests of each user into a pre-trained optimization model, and output a distribution scheme of the reasoning requests of each user to each edge server, and an expert hybrid quantization bit width scheme of each edge server.

[0135] Wherein, the training method of the optimization model includes:

[0136] Obtain inference requests from each user;

[0137] Constructing pre-training data using the inference requests of each user, wherein the pre-training data includes an inference request allocation randomly generated using the inference requests of each user, and an expert hybrid quantization bit width determined according to an expert activation frequency of the inference request allocation when each edge server serves the randomly generated inference request allocation;

[0138] Pre-training a policy network in reinforcement learning using the pre-training data to obtain an initialized policy network;

[0139] Inputting the reasoning request of each user into the initialized policy network, obtaining the allocation scheme of the reasoning request of each user output by the current policy network to each edge server, and the expert hybrid quantization bit width scheme of each edge server;

[0140] Using the allocation scheme of each user's reasoning request to each edge server output by the current policy network and the expert mixed quantization bit width scheme of each edge server, each user's reasoning request is served to obtain the service quality;

[0141] The service quality is used as a reward for the current policy network, and the parameters of the current policy network are iteratively updated until the service quality reaches a preset requirement, thereby obtaining the optimization model.

[0142] All relevant contents of each step involved in the embodiment of the aforementioned optimization method for deploying a hybrid expert model in an edge computing environment can be referred to the functional description of the functional module corresponding to the optimization device for deploying a hybrid expert model in an edge computing environment in an embodiment of the present invention, and will not be repeated here. The division of modules in the embodiment of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, the functional modules in the various embodiments of the present invention can be integrated into a processor, or they can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0143] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in a computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of an optimization method for deploying a hybrid expert model in an edge computing environment.

[0144] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the optimization method for deploying a hybrid expert model in an edge computing environment in the above embodiment.

[0145] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0147] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0149] In the present invention, the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0150] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An optimization method for deploying a hybrid expert model in an edge computing environment, characterized in that: include: Get the inference request of each current user; Input the current reasoning requests of each user into a pre-trained optimization model, and output a distribution scheme of the reasoning requests of each user to each edge server, and an expert hybrid quantization bit width scheme of each edge server; Wherein, the training method of the optimization model includes: Obtain inference requests from each user; Constructing pre-training data using the inference requests of each user, wherein the pre-training data includes an inference request allocation randomly generated using the inference requests of each user, and an expert hybrid quantization bit width determined according to an expert activation frequency of the inference request allocation when each edge server serves the randomly generated inference request allocation; Pre-training a policy network in reinforcement learning using the pre-training data to obtain an initialized policy network; Inputting the reasoning request of each user into the initialized policy network, obtaining the allocation scheme of the reasoning request of each user output by the current policy network to each edge server, and the expert hybrid quantization bit width scheme of each edge server; Using the allocation scheme of each user's reasoning request to each edge server output by the current policy network and the expert mixed quantization bit width scheme of each edge server, each user's reasoning request is served to obtain the service quality; The service quality is used as a reward for the current policy network, and the parameters of the current policy network are iteratively updated until the service quality reaches a preset requirement, thereby obtaining the optimization model.

2. According to the optimization method for deploying a hybrid expert model in an edge computing environment according to claim 1, it is characterized in that: The service quality includes service accuracy, service delay and switching cost of the expert hybrid quantization bit width solution.

3. According to the optimization method for deploying a hybrid expert model in an edge computing environment according to claim 1, it is characterized in that: When each edge server serves the randomly generated inference request allocation, the expert mixed quantization bit width determined according to the expert activation frequency allocated to the inference request is specifically: Inputting the inference request randomly assigned to each edge server into the local original hybrid expert model to obtain the activation frequency of each expert in the local original hybrid expert model; wherein the local original hybrid expert model includes a multi-layer expert group, and each layer of the expert group includes a plurality of experts; According to the activation frequency of each expert in each layer of the expert group, the quantization bit width of each expert in the expert group of this layer is decided; The quantization bit widths of each expert in all layers of the expert group constitute the mixed quantization decision of the local mixed expert model.

4. According to the optimization method for deploying a hybrid expert model in an edge computing environment according to claim 3, it is characterized in that: According to the activation frequency of each expert in each layer of the expert group, the quantization bit width of each expert in the expert group of this layer is decided, specifically: Sort the experts in each expert group in descending order according to the activation frequency; According to a preset mixed quantization bit width set, assigning the highest quantization bit width in the mixed quantization bit width set to the second half of the experts in each layer of the expert group after being sorted in descending order; wherein the quantization bit widths in the preset mixed quantization bit width set are arranged from large to small; According to the capacity limit of each edge server, determine whether the total quantized bit width of each layer of expert group after quantization exceeds the corresponding capacity limit; If it exceeds, the quantization bit widths of all experts in each layer of the expert group are reduced, and the quantization bit width of each expert is specifically reduced to the highest quantization bit width in the mixed quantization bit width set that is lower than the current bit width of the expert; The judgment process is repeated until the total quantization bit width of each layer of expert group after quantization does not exceed the corresponding capacity limit, and then the quantization bit width of each expert in the expert group of this layer is decided.

5. According to the optimization method for deploying a hybrid expert model in an edge computing environment according to claim 4, it is characterized in that: The method of using the pre-training data to pre-train the policy network in the reinforcement learning to obtain the initialized policy network is specifically as follows: The inference requests randomly assigned to each edge server are used as the input of the policy network. The quantization bit width of each expert in each expert group in each layer, which is determined according to the activation frequency of each expert in each layer of the expert group, is used as the output label of the policy network. The policy network is pre-trained until convergence to obtain an initialized policy network.

6. The optimization method for deploying a hybrid expert model in an edge computing environment according to claim 1, characterized in that: The inference request of each user is input into the initialized policy network to obtain the allocation scheme of the inference request of each user output by the current policy network to each edge server, and the expert mixed quantization bit width scheme of each edge server, which is specifically: In the formula, s t The mixed quantization bit width of experts on all current edge servers and the inference requests of each current user; is the expert hybrid quantization bit width scheme of edge server i at time t, in represents the data type quantified by expert j on edge server i at time t, with a total of m experts; E represents all edge servers; K t is the user reasoning request set at time t, k∈K t , where user reasoning request k is assigned to ID k s edge server; a t The quantized bit width of each expert on all edge servers at the next moment and the reasoning request of each user are finally assigned to the user with ID k s edge server.

7. The optimization method for deploying a hybrid expert model in an edge computing environment according to claim 1, characterized in that: The method uses the allocation scheme of the inference requests of each user output by the current policy network to each edge server, and the expert mixed quantization bit width scheme of each edge server to serve the inference requests of each user and obtain the service quality, which is specifically: r t =∑ i {p i -α1·l i -α2·C i } In the formula, r t is the reward of the current policy network; p i To serve accuracy; i is the service delay; C i is the switching cost of the expert hybrid quantization bit width scheme; α1 and α2 are parameters used to weigh the service accuracy, service delay and the switching cost of the expert hybrid quantization bit width scheme.

8. An optimization device for deploying a hybrid expert model in an edge computing environment, characterized in that: include: The acquisition module is used to obtain the inference requests of each current user; An optimization module, used to input the current reasoning requests of each user into a pre-trained optimization model, and output a distribution scheme of the reasoning requests of each user to each edge server, and an expert hybrid quantization bit width scheme of each edge server; Wherein, the training method of the optimization model includes: Obtain inference requests from each user; Constructing pre-training data using the inference requests of each user, wherein the pre-training data includes an inference request allocation randomly generated using the inference requests of each user, and an expert hybrid quantization bit width determined according to an expert activation frequency of the inference request allocation when each edge server serves the randomly generated inference request allocation; Pre-training a policy network in reinforcement learning using the pre-training data to obtain an initialized policy network; Inputting the reasoning request of each user into the initialized policy network, obtaining the allocation scheme of the reasoning request of each user output by the current policy network to each edge server, and the expert hybrid quantization bit width scheme of each edge server; Using the allocation scheme of each user's reasoning request to each edge server output by the current policy network and the expert mixed quantization bit width scheme of each edge server, each user's reasoning request is served to obtain the service quality; The service quality is used as a reward for the current policy network, and the parameters of the current policy network are iteratively updated until the service quality reaches a preset requirement, thereby obtaining the optimization model.

9. A device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements an optimization method for deploying a hybrid expert model in an edge computing environment as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements an optimization method for deploying a hybrid expert model in an edge computing environment as described in any one of claims 1 to 7.

Images