Method for deploying large hybrid expert model in edge computing power network at low cost

Abstract

The invention belongs to the technical field of new-generation information, and particularly relates to a method for deploying a hybrid expert large model in an edge computing power network at low cost, which comprises the following steps of: 1, defining and initializing a problem; 2, a first expert model deployment method; comprising cost calculation and first expert model deployment operation; 3, a subsequent expert model deployment method; comprising the steps of deployed node set updating, cost updating, subsequent expert model deployment and routing operation. 4, repeating and terminating; and 5, outputting. According to the invention, by reducing the average deployment cost of the single expert model, the purposes of reducing cost and increasing efficiency are achieved, and a deployment scheme of the hybrid expert model in the edge computing network, which saves resources and has higher benefits, is provided for large model service providers.

Description

Technical Field

[0001] The present invention relates to the field of new generation information technology, and in particular to a method for deploying a hybrid expert large model at low cost in an edge computing network. Background Art

[0002] Currently, there is an urgent need to deploy hybrid expert models (MoE) in edge computing networks at low cost. Traditional deployment solutions mainly rely on the product of the integration of network function virtualization (NFV) and cloud computing technology, namely virtual network embedding (VNE) technology.

[0003] VNE technology aims to map predefined virtual networks (with fixed virtual topology) to shared physical networks through the effective deployment and utilization of physical resources. VNE technology assumes that the topological logic of the virtual network is fixed and that the topological logic will strictly limit the mapping process of the virtual network. Specifically, the fixed topological logic will require that the physical nodes that carry the virtual nodes need to establish corresponding physical paths on the physical network one by one according to the connection method of the virtual topology. In the technical framework of VNE, each physical node can only carry one virtual node, even if there are still spare computing and storage resources. Although traditional VNE technology is relatively simple and clear at the implementation level, it is unable to cope with complex scenarios with dynamic connection requirements such as the Mixed Experts (MoE) model.

[0004] The dynamic activation mechanism of the MoE model's internal expert model makes its virtual connection highly uncertain before deployment. This uncertainty is in sharp contrast to the fixedness of the predefined virtual topology in VNE technology. In addition, the existing VNE technology sets a limit that each physical node can only carry one virtual node. This limitation may lead to insufficient utilization of physical network capacity during resource allocation, which in turn highlights the limitations of using VNE technology in deploying the MoE model.

[0005] The current existing approach is to use the XT (Crosstalk) avoidance model for cross-core communication and a distributed algorithm based on dynamic routing and random topology (DRART).

[0006] The XT avoidance model is an optimization technology for optical networks. Its core is to effectively reduce cross-interference between different cores through a reasonable resource allocation strategy, thereby improving the resource utilization of the entire network. The advantage of this model is that it can ensure the efficient use of network resources and avoid unnecessary waste of resources. In this solution, the connection of the virtual topology must also be predetermined. This mapping relationship simplifies the deployment and management of virtual networks, but it also brings certain limitations, which to a certain extent limits the flexibility and scalability of service deployment.

[0007] The DRART (Dynamic Routing and Random Topology) model is a method for implementing dynamic routing and resource allocation at the link level (reference: [VNE3]). It analyzes the blocking probability of virtual links and develops a resource allocation method based on a distributed genetic algorithm to achieve efficient resource utilization by jointly allocating node and link resources. Specifically, the DRART model adopts a design idea of ​​randomly generating virtual topology, which enables the network to show higher flexibility and adaptability when dealing with different traffic patterns and demands. However, although the DRART model has shown good performance in dynamic resource allocation, it still assumes that each physical node can only carry one virtual node in the VNE technical architecture. In practical applications, as the network scale expands and traffic increases, a single physical node needs to carry multiple virtual nodes to meet business needs, which also limits the performance of the DRART model in deploying MoE.

[0008] Therefore, the prior art has the following disadvantages:

[0009] A. Fixed topology and mapping relationship: Most existing VNE technologies are based on fixed topology. This design not only limits the flexibility of the network, but also makes it difficult for the network to adapt to dynamically changing network environments and diverse resource requirements.

[0010] B. Physical node load limitation: Existing VNE technologies usually stipulate that a physical node can only carry one virtual node. This limitation further weakens the network's resource optimization capabilities and flexibility, especially on physical nodes with sufficient resources. This limitation hinders the full utilization and efficient integration of resources.

[0011] C. Rigid resource allocation strategy: Constrained by fixed topology and mapping relationships, the resource allocation strategy of existing VNE technologies often cannot efficiently utilize physical network resources. Physical nodes with low resource utilization cannot reallocate their idle resources to virtual nodes with urgent resource needs, which not only wastes resources but also reduces the operating efficiency of the entire network.

[0012] D. Lack of flexibility: Due to the above limitations, existing VNE technologies often lack sufficient flexibility to adapt to different deployment requirements and complex network environments. Especially when dealing with complex problems such as expert model (EMs) deployment, they are often unable to provide efficient and flexible solutions. Summary of the invention

[0013] The content of the present invention is to provide a method for low-cost deployment of a hybrid expert large model in an edge computing network, which can overcome certain or some defects of the prior art.

[0014] According to the present invention, a method for low-cost deployment of a hybrid expert large model in an edge computing network includes the following steps:

[0015] 1. Problem definition and initialization;

[0016] Problem definition: Given a physical network G = (N, L) and a set of expert model deployment requests R = {EM, BW}, the goal is to determine how to deploy these expert models on the physical network to meet resource constraints and minimize the overall deployment cost;

[0017] Initialization: Define the physical network as an undirected graph G = (N, L), where N is the set of physical nodes and L is the set of physical links; at the same time, initialize the resource usage of all nodes and the bandwidth usage of the links to 0;

[0018] 2. The first expert model deployment method;

[0019] Cost calculation: For each physical node n and each expert model v, calculate the cost of deploying v on n, including the computational cost of running the expert model and storage cost of parameters associated with storing expert models

[0020] First expert model deployment operation: using the minimum cost metric To measure the total cost of identification The smallest pair of physical nodes n and expert models v, and deploy the corresponding expert models to the corresponding nodes;

[0021] 3. Subsequent expert model deployment method;

[0022] Deployed Node Set Update: is the set of nodes that have been determined to support EM; after the first expert model deployment operation is completed, Contains at least one physical node;

[0023] Cost update: For each undeployed expert model v and each physical node n, calculate the cost of deploying v on n and taking into account Routing cost of midpoints Total cost after The routing cost is n to The minimum routing cost of each node in;

[0024] Subsequent expert model deployment and routing: Identify a physical node n, an expert model v, and a path connecting n and the set of deployed nodes The shortest physical connection path Make Minimize; deploy the next expert model to the identified node and update the node and routing information;

[0025] 4. Repetition and termination;

[0026] 5. Output.

[0027] Preferably, the overall deployment cost includes computing cost, storage cost and bandwidth cost.

[0028] Preferably, in step 4, specifically:

[0029] Repeat: Repeat the above subsequent expert model deployment steps until all expert models are deployed;

[0030] Termination: Terminate when all expert models are successfully deployed on the physical network.

[0031] As a preference, in step five, specifically:

[0032] Output the final expert model deployment plan, including which physical node each expert model is deployed on and the routing path between them.

[0033] The beneficial effects of the present invention are as follows:

[0034] The present invention aims to solve the defects and problems faced by existing virtual network embedding (VNE) technology when deploying a hybrid of experts (MoE) model in edge computing networks:

[0035] A) Breaking through the limitations of fixed topology and mapping relationships: As most existing VNE technologies are based on fixed topology, the present invention aims to utilize a more flexible and variable deployment strategy to adapt to dynamically changing network environments and diverse resource requirements.

[0036] B) Breaking the physical node carrying limit: The existing technology usually stipulates that a physical node can only carry one virtual node. This limitation weakens the resource optimization capability and flexibility of the network. The present invention aims to improve resource utilization efficiency and network flexibility by utilizing the characteristic that a physical node can carry multiple expert models.

[0037] C) Optimizing resource allocation strategy: Constrained by fixed topology and mapping relationships, the resource allocation strategy of existing VNE technology often cannot efficiently utilize physical network resources. The present invention aims to propose a more efficient resource allocation and deployment cost reduction strategy, which can dynamically adjust resource allocation according to actual needs to improve resource utilization and reduce the deployment cost of the hybrid expert model.

[0038] D) Enhanced deployment flexibility and adaptability: Existing VNE technologies often lack sufficient flexibility when dealing with complex requirements such as expert model (EMs) deployment. This invention aims to improve the flexibility and adaptability of deployment by introducing new deployment algorithms and mechanisms, so that it can better adapt to different deployment requirements and complex network environments.

[0039] The present invention effectively optimizes the computing, storage and bandwidth costs by distributing the large-scale language model (LLM) of the hybrid expert (MoE) model in the edge computing network. Specifically, the present invention proposes a cost minimization algorithm (LCG-EMD) for deploying hybrid expert models in edge networks based on the minimum cost metric (LCG), which can accurately identify suitable physical nodes and deploy expert models at the lowest cost. Compared with existing benchmark algorithms, the present invention shows significant advantages in deployment costs. Simulation experiment results show that compared with the lowest price first matching (LPFM) and topology first resource matching (TFRM) algorithms, the LCG-EMD algorithm saves an average of 17.31% and 36.98% of deployment costs, respectively. In addition, the algorithm of the present invention also has a competitive advantage in running time, with an average running time of 15.46 milliseconds.

[0040] The present invention technically achieves cost optimization of large model deployment in edge environments, which has significant social and economic benefits. Specifically:

[0041] 1. Reduce resource consumption of a single business: The algorithm significantly reduces the total resource requirements when deploying a single business by accurately selecting computing, storage, and bandwidth resources, thereby reducing the cost of a single deployment.

[0042] 2. Increase the number of business deployments: Under the same resource capacity conditions, since each business occupies less resources, this algorithm can support the deployment of more businesses and achieve efficient use of resources.

[0043] By reducing the cost of a single service and increasing the number of service deployments, the LCG-EMD algorithm achieves the goal of "reducing costs and increasing efficiency", providing users with a resource-saving and more efficient edge computing deployment solution. The algorithm can be widely used in high-frequency machine call scenarios such as smart cities and smart factories, helping to promote the application and popularization of edge computing technology in the field of artificial intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flowchart of a method for low-cost deployment of a hybrid expert large model in an edge computing network in Example 1. DETAILED DESCRIPTION

[0045] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments are only for explaining the present invention and are not intended to limit it.

[0046] Example 1

[0047] like Figure 1 As shown, this embodiment provides a method for deploying a hybrid expert large model in an edge computing network at low cost, which includes the following steps:

[0048] 1. Problem definition and initialization;

[0049] Problem definition: Given a physical network G = (N, L) and a set of expert model deployment requests P = {EM, BW}, the goal is to determine how to deploy these expert models on the physical network to meet resource constraints and minimize the overall deployment cost; the overall deployment cost includes computing cost, storage cost and bandwidth cost.

[0050] Initialization: Define the physical network as an undirected graph G = (N, L), where N is the set of physical nodes and L is the set of physical links. At the same time, initialize the resource usage (computing, storage) of all nodes and the bandwidth usage of the links to 0.

[0051] 2. The first expert model deployment method;

[0052] Cost calculation: For each physical node n and each expert model v, calculate the cost of deploying v on n, including the computational cost of running the expert model and storage cost of parameters associated with storing expert models

[0053] First expert model deployment operation: using the minimum cost metric To measure the total cost of identification The smallest pair of physical nodes n and expert models v, and deploy the corresponding expert models to the corresponding nodes.

[0054] 3. Subsequent expert model deployment method;

[0055] Deployed Node Set Update: is the set of nodes that have been determined to support EM (computing, storage); after completing the first expert model deployment operation, Contains at least one physical node.

[0056] Cost update: For each undeployed expert model v and each physical node n, calculate the cost of deploying v on n and taking into account Routing cost of midpoints Total cost after The routing cost is n to The minimum routing cost of each node in .

[0057] Subsequent expert model deployment and routing: Identify a physical node n, an expert model v, and a path connecting n and the set of deployed nodes The shortest physical connection path Make Minimize; deploy the next expert model to the identified node and update the node and routing information.

[0058] 4. Repetition and termination;

[0059] Repeat: Repeat the above subsequent expert model deployment steps until all expert models are deployed.

[0060] Termination: Terminate when all expert models are successfully deployed on the physical network.

[0061] 5. Output;

[0062] Output the final expert model deployment plan, including which physical node each expert model is deployed on and the routing path between them.

[0063] The present invention can deploy expert models in edge networks in a cost-optimized manner, thereby meeting the growing demand for machine learning model deployment. The present invention ensures the effectiveness and feasibility of the deployment scheme by considering computing, storage and bandwidth costs, as well as connectivity constraints between nodes and links.

[0064] Example 2

[0065] This embodiment is a cost optimization method for distributed deployment of a hybrid expert model (MoE) in an edge computing network. The specific implementation process is as follows:

[0066] 1. System architecture:

[0067] This example is based on the DBRX model, which uses a Mixed of Experts (MoE) architecture. The MoE consists of the following key components:

[0068] Router: Responsible for dynamically selecting the appropriate expert model to complete reasoning.

[0069] Aggregator: Aggregates the outputs of each expert model to generate the final result.

[0070] Expert Models: Dynamically activated based on input content and perform reasoning tasks.

[0071] 2. Methods and steps:

[0072] Step 1: Parameter setting and resource evaluation

[0073] The computing resources (number of CUDA cores) and storage capacity (TB) of each edge node are set according to the hardware resource configuration of the experimental platform. Specifically, each node has a computing power of 18,000 to 36,000 CUDA cores and a storage capacity of 12 to 24 TB. The bandwidth requirement of each network link is set to 100 Gbps, and the unit price of computing and storage refers to the standard pricing publicly available on the Azure platform.

[0074] Step 2: Deployment of the first expert model

[0075] The LCG-EMD algorithm for deploying hybrid expert models in edge networks based on the minimum cost metric (LCG) is used to select appropriate initial expert models for deployment on physical nodes with sufficient resources. It is used to evaluate the computing and storage costs of each node and prioritize the expert model to the node with the lowest cost.

[0076] Step 3: Dynamic deployment of subsequent expert models

[0077] After the initial deployment is completed, continue to select appropriate nodes in the physical network to deploy the remaining expert models. The selection of each new model is based on the minimum cost gain algorithm, while updating the resource utilization of the physical nodes and dynamically updating the node set and link set Ensure that resources are allocated appropriately.

[0078] 3. Experimental process and data source:

[0079] a. Experimental topology: The experiment uses the NY-20 edge network topology to simulate a highly connected edge environment. In this network, the computing and storage resource parameters of each node are set according to the data source of the existing Azure platform.

[0080] b. Comparison algorithm: In order to verify the superiority of the method of this embodiment, two comparison algorithms are designed:

[0081] Lowest Price First Matching (LPFM): By sorting the computing and storage prices of the nodes, the nodes with the lowest prices are prioritized for expert model deployment.

[0082] Topology-first resource matching (TFRM): Generates a virtual topology first, and then matches nodes according to their resource conditions to reduce duplicate bandwidth reservations.

[0083] c. Experimental data and results:

[0084] A simulated request is used, each request includes 1 router (m0 ), 16 expert models (m 1 To m 16 ), and 1 aggregator (m 17 Each expert model requires 6000 CUDA cores and 330 GB of storage resources, and the resource requirements of the router and aggregator are 500 CUDA cores, 2 GB and 1000 CUDA cores, 5 GB respectively.

[0085] In multiple deployment process experiments, the average deployment cost of the LCG-EMD algorithm was 17.31% and 36.98% lower than that of LPFM and TFRM respectively; the highest cost saving rate was 51.95% higher than that of the LPFM algorithm and 69.54% higher than that of the TFRM algorithm, significantly reducing the overall cost of deploying expert models in edge computing networks; and the average running time was 15.46 milliseconds, showing the dual advantages of cost and efficiency.

[0086] Through detailed parameter settings and experimental data, this embodiment demonstrates the effectiveness and cost optimization effect of large-scale language model deployment of the MoE architecture in edge networks, and has broad practical application value.

[0087] The present invention and its embodiments are described schematically above, and the description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs a structural method and an embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they shall all fall within the protection scope of the present invention.

Claims

1. A method for low-cost deployment of a hybrid expert large model in an edge computing network, characterized by: The following steps are involved:

1. Problem definition and initialization; Problem definition: Given a physical network G = (N, L) and a set of expert model deployment requests R = {EM, BW}, the goal is to determine how to deploy these expert models on the physical network to meet resource constraints and minimize the overall deployment cost; Initialization: Define the physical network as an undirected graph G = (N, L), where N is the set of physical nodes and L is the set of physical links; at the same time, initialize the resource usage of all nodes and the bandwidth usage of the links to 0; 2. The first expert model deployment method; Cost calculation: For each physical node n and each expert model v, calculate the cost of deploying v on n, including the computational cost of running the expert model and storage cost of parameters associated with storing expert models First expert model deployment operation: using the minimum cost metric To measure the total cost of identification The smallest pair of physical nodes n and expert models v, and deploy the corresponding expert models to the corresponding nodes; 3. Subsequent expert model deployment method; Deployed Node Set Update: is the set of nodes that have been determined to support EM; after the first expert model deployment operation is completed, Contains at least one physical node; Cost update: For each undeployed expert model v and each physical node n, calculate the cost of deploying v on n and taking into account Routing cost of midpoints Total cost after The routing cost is n to The minimum routing cost of each node in; Subsequent expert model deployment and routing: Identify a physical node n, an expert model v, and a path connecting n and the set of deployed nodes The shortest physical connection path Make Minimize; deploy the next expert model to the identified node and update the node and routing information; 4. Repetition and termination; 5. Output.

2. According to claim 1, a method for low-cost deployment of a hybrid expert large model in an edge computing network is characterized in that: The overall deployment cost includes computing cost, storage cost, and bandwidth cost.

3. The method for low-cost deployment of a hybrid expert large model in an edge computing network according to claim 2, characterized in that: In step 4, specifically: Repeat: Repeat the above subsequent expert model deployment steps until all expert models are deployed; Termination: Terminate when all expert models are successfully deployed on the physical network.

4. The method for low-cost deployment of a hybrid expert large model in an edge computing network according to claim 3, characterized in that: In step five, specifically: Output the final expert model deployment plan, including which physical node each expert model is deployed on and the routing path between them.

Images