A method for low-cost deployment of hybrid expert large models in edge computing networks

Abstract

This invention belongs to the field of next-generation information technology. Specifically, it is a method for low-cost deployment of large hybrid expert models in edge computing networks. The method comprises the following steps: 1. Problem definition and initialization; 2. Initial expert model deployment method, including cost calculation and initial expert model deployment operations; 3. Subsequent expert model deployment method, including updating the deployed node set, cost update, subsequent expert model deployment, and routing operations; 4. Repetition and termination; and 5. Output. By reducing the average cost of a single expert model deployment, the present invention achieves the goal of reducing costs and increasing efficiency. This provides large model service providers with a resource-saving and more efficient deployment solution for hybrid expert models in edge computing networks.

Description

Technical Field

[0001] The present invention relates to the field of new generation information technology, and more specifically, to a method for deploying a hybrid expert large model at low cost in an edge computing network. Background Art

[0002] Currently, there is an urgent need to deploy hybrid expert models (MoE) in edge computing networks at low cost. Traditional deployment solutions mainly rely on the product of the integration of network function virtualization (NFV) and cloud computing technology, namely virtual network embedding (VNE) technology.

[0003] VNE technology aims to map predefined virtual networks (with fixed virtual topologies) onto shared physical networks through the effective deployment and utilization of physical resources. VNE technology assumes that the topological logic of the virtual network is fixed and unchanging, and that this topological logic will strictly limit the mapping process of the virtual network. Specifically, this fixed topological logic will require that the physical nodes that carry the virtual nodes need to establish corresponding physical paths on the physical network one by one according to the connection method of the virtual topology. In the technical framework of VNE, each physical node can only carry one virtual node, even if there are still spare computing and storage resources. Although traditional VNE technology is relatively simple and clear at the implementation level, it is unable to cope with complex scenarios with dynamic connection requirements such as the Mixed of Experts (MoE) model.

[0004] Due to the dynamic activation mechanism of its internal expert model, the MoE model's virtual connections exhibit a high degree of uncertainty before deployment. This uncertainty contrasts sharply with the fixed nature of predefined virtual topologies in VNE technology. Furthermore, existing VNE technology limits each physical node to hosting only one virtual node. This limitation can lead to underutilization of physical network capacity during resource allocation, highlighting the limitations of using VNE technology in MoE deployments.

[0005] The current existing approach is to use the XT (Crosstalk) avoidance model for cross-core communication and the distributed algorithm based on dynamic routing and random topology (DRART).

[0006] The XT avoidance model is an optimization technology for optical networks. Its core goal is to effectively reduce cross-interference between different cores through a rational resource allocation strategy, thereby improving resource utilization across the entire network. The advantage of this model is that it ensures efficient use of network resources and avoids unnecessary resource waste. In this solution, the connections of the virtual topology must also be predetermined. This mapping relationship simplifies the deployment and management of virtual networks, but it also brings certain limitations, limiting the flexibility and scalability of service deployment.

[0007] The DRART (Dynamic Routing and Random Topology) model is a method for implementing dynamic routing and resource allocation at the link level. It analyzes the blocking probability of virtual links and develops a resource allocation method based on a distributed genetic algorithm to achieve efficient resource utilization by jointly allocating node and link resources. Specifically, the DRART model adopts a design approach of randomly generating virtual topologies, which enables the network to show higher flexibility and adaptability when dealing with different traffic patterns and demands. However, although the DRART model has demonstrated good performance in dynamic resource allocation, it still assumes that each physical node can only host one virtual node in the VNE technical architecture. In actual applications, as the network scale expands and traffic increases, a single physical node needs to host multiple virtual nodes to meet business needs, which also limits the performance of the DRART model in deploying MoE.

[0008] Therefore, the prior art has the following disadvantages:

[0009] A. Fixed topology and mapping relationships: Existing VNE technologies are mostly based on fixed topologies. This design not only limits network flexibility but also makes it difficult for the network to adapt to dynamically changing network environments and diverse resource requirements.

[0010] B. Physical Node Load Limitation: Existing VNE technologies typically stipulate that a physical node can only host one virtual node. This limitation further weakens the network's resource optimization capabilities and flexibility, especially on physical nodes with abundant resources. This limitation hinders full resource utilization and efficient integration.

[0011] C. Rigid resource allocation strategies: Constrained by fixed topologies and mapping relationships, existing VNE technology's resource allocation strategies often fail to efficiently utilize physical network resources. Physical nodes with low resource utilization cannot reallocate their idle resources to virtual nodes with urgent resource needs. This not only wastes resources but also reduces the overall network's operational efficiency.

[0012] D. Lack of Flexibility: Due to the aforementioned limitations, existing VNE technologies often lack sufficient flexibility to adapt to diverse deployment requirements and complex network environments. This is particularly true when dealing with complex issues such as expert model (EM) deployment, often failing to provide efficient and flexible solutions. Summary of the Invention

[0013] The present invention provides a method for low-cost deployment of hybrid expert large models in edge computing networks, which can overcome certain or some defects of the existing technology.

[0014] According to the present invention, a method for low-cost deployment of a hybrid expert large model in an edge computing network includes the following steps:

[0015] 1. Problem definition and initialization;

[0016] Problem definition: Given a physical network G = (N, L) and a set of expert model deployment requests R = {EM, BW}, the goal is to determine how to deploy these expert models on the physical network to meet resource constraints and minimize the overall deployment cost;

[0017] Initialization: Define the physical network as an undirected graph G = (N, L), where N is the set of physical nodes and L is the set of physical links. At the same time, initialize the resource usage of all nodes and the bandwidth usage of all links to 0.

[0018] 2. The first expert model deployment method;

[0019] Cost calculation: For each physical node n and each expert model v, calculate the cost of deploying v on n, including the computational cost of running the expert model and storage costs of parameters associated with the expert model

[0020] First expert model deployment operation: using the minimum cost metric To measure the total cost of identification The smallest pair of physical nodes n and expert models v, and deploy the corresponding expert models on the corresponding nodes;

[0021] 3. Subsequent expert model deployment method;

[0022] Deployed Node Set Update: is the set of nodes that have been determined to support EM; after the first expert model deployment operation is completed, Contains at least one physical node;

[0023] Cost update: For each undeployed expert model v and each physical node n, calculate the cost of deploying v on n and taking into account the cost of Routing cost of midpoints Total cost after The routing cost is n to The minimum routing cost of each node in;

[0024] Subsequent expert model deployment and routing: Identify a physical node n, an expert model v, and a path connecting n and the set of deployed nodes. The shortest physical connection path Make Minimize; deploy the next expert model to the identified node and update the node and routing information;

[0025] 4. Repetition and termination;

[0026] 5. Output.

[0027] Preferably, the overall deployment cost includes computing cost, storage cost, and bandwidth cost.

[0028] Preferably, in step 4, specifically:

[0029] Repeat: Repeat the above subsequent expert model deployment steps until all expert models are deployed;

[0030] Termination: Terminate when all expert models are successfully deployed on the physical network.

[0031] As a preference, in step five, specifically:

[0032] Output the final expert model deployment plan, including which physical node each expert model is deployed on and the routing path between them.

[0033] The beneficial effects of the present invention are as follows:

[0034] This invention aims to address the shortcomings and problems faced by existing Virtual Network Embedding (VNE) technology when deploying Mixed of Experts (MoE) models in edge computing networks:

[0035] A) Breaking through the limitations of fixed topology and mapping relationships: As most existing VNE technologies are based on fixed topology structures, the present invention aims to utilize a more flexible and variable deployment strategy to adapt to dynamically changing network environments and diverse resource requirements.

[0036] B) Breaking the physical node load limitation: Existing technologies typically stipulate that a physical node can only host one virtual node. This limitation weakens the network's resource optimization capabilities and flexibility. This invention aims to leverage the ability of a physical node to host multiple expert models to improve resource utilization efficiency and network flexibility.

[0037] C) Optimizing Resource Allocation Strategies: Constrained by fixed topologies and mapping relationships, existing VNE resource allocation strategies often fail to efficiently utilize physical network resources. This paper aims to propose a more efficient resource allocation and deployment cost reduction strategy that dynamically adjusts resource allocation based on actual needs, thereby improving resource utilization and reducing the deployment cost of the hybrid expert model.

[0038] D) Enhanced Deployment Flexibility and Adaptability: Existing VNE technologies often lack sufficient flexibility to handle complex requirements, such as deploying expert models (EMs). This invention aims to improve deployment flexibility and adaptability by introducing new deployment algorithms and mechanisms, enabling it to better adapt to diverse deployment requirements and complex network environments.

[0039] The present invention effectively optimizes the computing, storage and bandwidth costs by distributing the large-scale language model (LLM) of the hybrid expert (MoE) model in the edge computing network. Specifically, the present invention proposes a cost minimization algorithm (LCG-EMD) for deploying hybrid expert models in edge networks based on the minimum cost index (LCG), which can accurately identify suitable physical nodes and deploy expert models at the lowest cost. Compared with existing benchmark algorithms, the present invention shows significant advantages in deployment costs. Simulation experiment results show that compared with the lowest price first matching (LPFM) and topology first resource matching (TFRM) algorithms, the LCG-EMD algorithm saves an average of 17.31% and 36.98% of deployment costs, respectively. In addition, the algorithm of the present invention also has a competitive advantage in running time, with an average running time of 15.46 milliseconds.

[0040] This invention technically achieves cost optimization for large-scale model deployment in edge environments, with significant social and economic benefits. Specifically:

[0041] 1. Reduce resource consumption for a single service: This algorithm significantly reduces the total resource requirements when deploying a single service by accurately selecting computing, storage, and bandwidth resources, thereby reducing the cost of a single deployment.

[0042] 2. Increase the number of business deployments: Under the same resource capacity conditions, since each business occupies less resources, this algorithm can support the deployment of more businesses and achieve efficient resource utilization.

[0043] By reducing the cost per transaction and increasing the number of deployed transactions, the LCG-EMD algorithm achieves the goal of "reducing costs and increasing efficiency," providing users with a resource-efficient and more cost-effective edge computing deployment solution. This algorithm can be widely applied in high-frequency machine call scenarios such as smart cities and smart factories, helping to promote the application and popularization of edge computing technology in the field of artificial intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flowchart of a method for low-cost deployment of a hybrid expert large model in an edge computing network in Example 1. DETAILED DESCRIPTION

[0045] In order to further understand the content of the present invention, the present invention is described in detail with reference to the accompanying drawings and embodiments. It should be understood that the embodiments are merely for explanation of the present invention and are not intended to limit the present invention.

[0046] Example 1

[0047] like Figure 1 As shown, this embodiment provides a method for low-cost deployment of a hybrid expert large model in an edge computing network, which includes the following steps:

[0048] 1. Problem definition and initialization;

[0049] Problem definition: Given a physical network G = (N, L) and a set of expert model deployment requests R = {EM, BW}, the goal is to determine how to deploy these expert models on the physical network to meet resource constraints and minimize the overall deployment cost; the overall deployment cost includes computational cost, storage cost, and bandwidth cost.

[0050] Initialization: Define the physical network as an undirected graph G = (N, L), where N is the set of physical nodes and L is the set of physical links. Also, initialize the resource usage (computing, storage) of all nodes and the bandwidth usage of all links to 0.

[0051] 2. The first expert model deployment method;

[0052] Cost calculation: For each physical node n and each expert model v, calculate the cost of deploying v on n, including the computational cost of running the expert model and storage costs of parameters associated with the expert model

[0053] First expert model deployment operation: using the minimum cost metric To measure the total cost of identification The smallest pair of physical nodes n and expert models v, and deploy the corresponding expert models to the corresponding nodes.

[0054] 3. Subsequent expert model deployment method;

[0055] Deployed Node Set Update: is the set of nodes that have been determined to support EM (computing, storage); after completing the first expert model deployment operation, Contains at least one physical node.

[0056] Cost update: For each undeployed expert model v and each physical node n, calculate the cost of deploying v on n and taking into account the cost of Routing cost of midpoints Total cost after The routing cost is n to The minimum routing cost of each node in .

[0057] Subsequent expert model deployment and routing: Identify a physical node n, an expert model v, and a path connecting n and the set of deployed nodes. The shortest physical connection path Make Minimize; deploy the next expert model to the identified node and update the node and routing information.

[0058] 4. Repetition and termination;

[0059] Repeat: Repeat the above subsequent expert model deployment steps until all expert models are deployed.

[0060] Termination: Terminate when all expert models are successfully deployed on the physical network.

[0061] 5. Output;

[0062] Output the final expert model deployment plan, including which physical node each expert model is deployed on and the routing path between them.

[0063] This invention enables cost-optimized deployment of expert models in edge networks, thereby meeting the growing demand for machine learning model deployment. By considering computational, storage, and bandwidth costs, as well as connectivity constraints between nodes and links, the invention ensures the effectiveness and feasibility of the deployment solution.

[0064] Example 2

[0065] This embodiment is a cost optimization method for distributed deployment of a hybrid expert model (MoE) in an edge computing network. The specific implementation process is as follows:

[0066] 1. System architecture:

[0067] This example is based on the DBRX model, which uses a Mixture of Experts (MoE) architecture. The MoE consists of the following key components:

[0068] Router: Responsible for dynamically selecting the appropriate expert model to complete reasoning.

[0069] Aggregator: Aggregates the output of each expert model to generate the final result.

[0070] Expert Models: Dynamically activated based on input content and perform reasoning tasks.

[0071] 2. Method and steps:

[0072] Step 1: Parameter setting and resource evaluation

[0073] The computing resources (number of CUDA cores) and storage capacity (TB) of each edge node are set based on the hardware resource configuration of the experimental platform. Specifically, each node has a computing power of 18,000 to 36,000 CUDA cores and a storage capacity of 12 to 24 TB. The bandwidth requirement for each network link is set at 100 Gbps, and the unit price of computing and storage is based on the standard pricing publicly available on the Azure platform.

[0074] Step 2: Deployment of the first expert model

[0075] The cost minimization algorithm for deploying hybrid expert models in edge networks based on the minimum cost metric (LCG) (LCG-EMD) is used to select appropriate initial expert models for deployment on physical nodes with sufficient resources. It is used to evaluate the computing and storage costs of each node and prioritize the expert model to the node with the lowest cost.

[0076] Step 3: Dynamic deployment of subsequent expert models

[0077] After the initial deployment is completed, continue to select appropriate nodes in the physical network to deploy the remaining expert models. The selection of each new model is based on the minimum cost gain algorithm, while updating the resource utilization of the physical nodes and dynamically updating the node set. and link set Ensure that resources are allocated appropriately.

[0078] 3. Experimental process and data sources:

[0079] a. Experimental Topology: The experiment uses the NY-20 edge network topology to simulate a highly connected edge environment. The compute and storage resource parameters of each node in this network are configured based on existing Azure platform data sources.

[0080] b. Comparison Algorithms: To verify the superiority of the method of this embodiment, two comparison algorithms were designed:

[0081] Lowest Price First Matching (LPFM): By sorting the computing and storage prices of nodes, the nodes with the lowest prices are prioritized for expert model deployment.

[0082] Topology-First Resource Matching (TFRM): Generates a virtual topology and then matches nodes based on their resource availability, reducing duplicate bandwidth reservations.

[0083] c. Experimental data and results:

[0084] A simulation request is used, each request includes 1 router (m0), 16 expert models (m1 to m2), and 16 ), and 1 aggregator (m 17 Each expert model requires 6,000 CUDA cores and 330 GB of storage resources, while the router and aggregator require 500 CUDA cores, 2 GB, and 1,000 CUDA cores, 5 GB of storage resources, respectively.

[0085] In multiple deployment process experiments, the average deployment cost of the LCG-EMD algorithm was 17.31% and 36.98% lower than that of LPFM and TFRM, respectively. The highest cost savings rate was 51.95% higher than that of the LPFM algorithm and 69.54% higher than that of the TFRM algorithm, significantly reducing the overall cost of deploying expert models in edge computing networks. The average running time was 15.46 milliseconds, demonstrating the dual advantages of cost and efficiency.

[0086] Through detailed parameter settings and experimental data, this embodiment demonstrates the effectiveness and cost optimization of large-scale language model deployment of the MoE architecture in edge networks, and has broad practical application value.

[0087] The above is a schematic description of the present invention and its embodiments, which is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs a structure and embodiment similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. A method for low-cost deployment of a hybrid expert large model in an edge computing network, characterized by: The following steps are involved:

1. Problem definition and initialization; Problem definition: Given a physical network G = (N, L) and a set of expert model deployment requests R = {EM, BW}, the goal is to determine how to deploy these expert models on the physical network to meet resource constraints and minimize the overall deployment cost; Initialization: Define the physical network as an undirected graph G = (N, L), where N is the set of physical nodes and L is the set of physical links. At the same time, initialize the resource usage of all nodes and the bandwidth usage of all links to 0.

2. The first expert model deployment method; Cost calculation: For each physical node n and each expert model v, calculate the cost of deploying v on n, including the computational cost of running the expert model and storage costs of parameters associated with the expert model First expert model deployment operation: using the minimum cost metric To measure the total cost of identification The smallest pair of physical nodes n and expert models v, and deploy the corresponding expert models on the corresponding nodes; 3. Subsequent expert model deployment method; Deployed Node Set Update: is the set of nodes that have been determined to support EM; after the first expert model deployment operation is completed, Contains at least one physical node; Cost update: For each undeployed expert model v and each physical node n, calculate the cost of deploying v on n and taking into account the cost of Routing cost of midpoints Total cost after The routing cost is n to The minimum routing cost of each node in; Subsequent expert model deployment and routing: Identify a physical node n, an expert model v, and a path connecting n and the set of deployed nodes. The shortest physical connection path Make Minimize; deploy the next expert model to the identified node and update the node and routing information; 4. Repetition and termination; 5. Output.

2. The method for low-cost deployment of a hybrid expert large model in an edge computing network according to claim 1, characterized in that: The total deployment cost includes computing cost, storage cost, and bandwidth cost.

3. The method for low-cost deployment of a hybrid expert large model in an edge computing network according to claim 2, characterized in that: In step 4, specifically: Repeat: Repeat the above subsequent expert model deployment steps until all expert models are deployed; Termination: Terminate when all expert models are successfully deployed on the physical network.

4. The method for low-cost deployment of a hybrid expert large model in an edge computing network according to claim 3, characterized in that: In step five, specifically: Output the final expert model deployment plan, including which physical node each expert model is deployed on and the routing path between them.

Images