Large language model optimization method based on edge calculation

By segmenting the Transformer model to user equipment and cloud servers in an edge computing environment, and using MCTS to combine decision-making networks and value networks to dynamically adjust segmentation points and resource allocation, the computing resources of large language models in edge computing are solved. The problems of insufficient utilization and high latency have achieved more efficient resource utilization and practicality.

CN119026640AInactive Publication Date: 2024-11-26NANJING UNIV OF INFORMATION SCI & TECH

Patent Information

Application Number
CN202411525003.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-30
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Large language models are difficult to operate directly in edge computing environments, resulting in insufficient utilization of computing resources and high latency. The existing algorithms are slow to converge and are easily trapped in local optimal decisions, and it is impossible to achieve effective model segmentation point selection and resource allocation strategies.

Method used

Using the Transformer model segmentation deployment method based on edge computing, the model is divided into two parts: user equipment and cloud server, and through Monte Carlo Tree Search (MCTS), the decision-making network and value network are combined, the segmentation points and resource allocation are dynamically adjusted to realize the calculation. Resource optimization and latency reduction.

Benefits of technology

The system's computing power and resource utilization rate are improved, the delay is reduced, the optimal segmentation of the model and resource balance utilization are achieved, and the practicality and robustness of the system are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026640A_ABST
    Figure CN119026640A_ABST
Patent Text Reader

Abstract

The invention discloses a large language model optimization method based on edge calculation, which belongs to the field of machine learning, and comprises the following steps of: in an edge calculation environment and under a large model structure framework based on Transform, segmenting a Transform model into a bottom layer part and a high layer part by searching an optimal segmentation point reasoned by the model, and then, carrying out edge calculation on the bottom layer part and the high layer part; the method comprises the following steps: deploying a lightweight bottom-layer model on user equipment with limited computing power, and deploying a computing-intensive high-layer model on a cloud server; the optimal segmentation point is found, and the Transform model is reasonably segmented and deployed on the user equipment and the cloud server, so that the optimization of computing resource utilization and the reduction of response delay are realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of machine learning, and specifically relates to a large language model optimization method based on edge computing. Background Art

[0002] Large Language Models (LLMs) refer to deep learning models with a large number of parameters and complex structures. However, these models usually require a lot of computing resources and storage space, and are mainly deployed on high-performance cloud computing platforms, leading to problems such as computing resource limitations, latency, and bandwidth.

[0003] Edge computing can reduce latency and improve real-time performance by moving some computing tasks to edge devices. However, due to the limited computing power of edge devices, it is difficult to directly run large-scale language models.

[0004] Currently, most algorithms use DQN, Q-learning or PSO. During the training process, they have the disadvantages of slow algorithm convergence and low training efficiency. At the same time, they are prone to fall into local optimal decisions and cannot obtain the global optimal model segmentation point selection and resource allocation strategy, making it impossible to achieve the purpose of large language model optimization. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a large language model optimization method based on edge computing in response to the shortcomings of the background technology. By finding the optimal segmentation point, the Transformer model is reasonably segmented and deployed on the user device and the cloud server, thereby optimizing the utilization of computing resources and reducing the response delay.

[0006] The present invention adopts the following technical solution to solve the above technical problems: a large language model optimization method based on edge computing, specifically comprising the following steps; Step 1, model building: the model includes a Transformer model deployed on a user device and a cloud server; in an edge computing environment, the Transformer model is split and deployed on the user device and the cloud; Among them, the user devices in the environment represent the set , the cloud server represents the set ; In the time period t, the computing resources of the user device and the cloud server are represented as and ; Step 2, segmentation model, specifically includes the following steps: Step 2.1: split the Transformer model and record the split points as Q ; Step 2.2, let the total number of layers of the model beX , in the model Q Layer segmentation, front Q The layer deployment on the user device is represented as ,the remaining X - Q The layer is deployed on the cloud server as , where the split point Q It is adjustable; Step 2.3, for an input , through UE x Convert to intermediate tensor : in, represents the parameters deployed at the user device layer; the intermediate tensor y is transmitted to the cloud server through the wireless channel, and is affected by the Rayleigh fading channel gain and Gaussian white noise during the transmission, and is obtained : Where, h represents the Rayleigh fading channel gain, and n represents Gaussian white noise; Step 2.4, processed by the cloud server : in, Indicates the parameters deployed in the cloud server layer; Step 3, Constraint Model: Since the computing resources of user devices and cloud servers are limited in the edge computing environment, the following constraints are given: in, Indicates the maximum computing resources of the user device, Indicates the maximum computing resources of the cloud server; Step 4: Optimize the model: Select appropriate split points Q When computing, it is necessary to consider the dynamic balance between computing resources and computing time, that is, while ensuring the completion of the task, it is also necessary to minimize the computing delay and transmission delay; the optimization model can be expressed as: in, Indicates that the user device calculates Q The time required for the layer, Indicates that the cloud server calculates the remaining X - Q The time required for the layer, Indicates the time required for data to be transmitted from the user device to the cloud server.

[0007] As a further preferred embodiment of a large language model optimization method based on edge computing of the present invention, the edge computing environment includes the selection of segmentation points, the association decision between user devices and cloud servers, and the dynamic allocation of resources.

[0008] As a further preferred solution of the large language model optimization method based on edge computing of the present invention, the segmentation point selection and specific implementation strategy are as follows: 1) Initialization: Initialize the MCTS tree, the root node represents the initial state, and the split point at this time is Q ; Among them, MCTS is Monte Carlo Tree Search; 2) Selection: In the MCTS tree, starting from the root node, the child nodes are selected through the decision network, the tree is expanded along the optimal path, and a node that is not fully expanded is selected; 3) Expansion: If the selected node is not fully expanded, expand the node according to the current state, generate new child nodes, and calculate the previous Q Layer and back X - Q The data transmission volume and computational requirements of the layer, the model can be expressed as: in, and Respectively indicate the front Q Layer and back X - Q The computational complexity of the layer, The amount of data transferred, B is bandwidth; 4) Simulation: Use the value network to simulate the newly expanded nodes, estimate the value of the current nodes, and obtain a reward value; 5) Backtracking: trace the reward value obtained from the simulation back to the root node, update the node value on the path, and adjust the parameters of the decision network and value network; Learning to update the node value, the formula is as follows: in: is in state s Next select action a The state-action value of is the learning rate, which determines the importance of new information in the update process; is the reward obtained at the current step, is the discount factor; is in the next state The maximum of all possible moves value; 6) Iteration: Repeat the above process of selection, expansion, simulation, and backtracking until the termination condition is met, thereby achieving the purpose of determining the optimal segmentation point.

[0009] As a further preferred solution of the large language model optimization method based on edge computing of the present invention, the user device is associated with the cloud server, specifically as follows: In each time period tAccording to the result of segmentation point selection, the association strategy between the user end and the cloud server is optimized; the association strategy is iteratively adjusted according to the feedback value to achieve the purpose of minimizing the calculation delay and transmission delay.

[0010] As a further preferred solution of the large language model optimization method based on edge computing of the present invention, the dynamic allocation of resources is as follows: After determining the optimal split point and associated strategy, dynamically adjust resource allocation according to real-time load and task requirements; set the following dynamic resource allocation strategy to maximize resource utilization while avoiding overload: First, the real-time computing load of user devices and cloud servers is monitored, then resource allocation is predicted based on the load, and finally resource reservation is implemented to cope with sudden computing demands; Dynamic resource allocation can be defined as: in, Indicates the relationship between the time required for the user device and the cloud server to calculate the corresponding number of layers and the corresponding current system load; is the current load.

[0011] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects: The present invention adopts a segmentation optimization method based on the Transformer model in an edge computing environment; by deploying the bottom layer of the Transformer model on the user device and the high layer on the cloud server, the hierarchical collaborative reasoning of the model is realized; at the same time, a decision network and a value network MCTS method are combined to establish an adaptive dynamic adjustment mechanism to find the best segmentation point, so as to optimally segment the model; the computing power of the system is improved, the device delay is reduced, and the balanced utilization of resources is achieved; The present invention adopts the MCTS method that combines the decision network and the value network. By learning a large amount of historical data and complex features, the decision network and the value network can give more accurate estimates, thereby guiding MCTS to reduce the search for unnecessary branches during the search process, greatly improving the search efficiency and resource utilization; the end-to-end training of the method through the deep learning model can adapt to different environments more quickly, thereby achieving a faster convergence speed; The present invention alleviates computing pressure through an efficient edge computing environment in which the user end and the cloud server work together, and combines a segmented deployed Transformer model to achieve accurate and fast interconnection and achieve the purpose of distributed collaborative computing; in addition, the present invention focuses on the adaptive selection of LLMs segmentation points to achieve the best research effect, ultimately improving the practicality and robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a Transformer model architecture diagram of the present invention segmented in an edge computing environment; Figure 2 This is the working principle of the Monte Carlo Tree Search MCTS of the present invention; Figure 3 It is a flow chart of a large language model optimization method based on edge computing of the present invention. DETAILED DESCRIPTION

[0013] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings: The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The present invention is described in detail below based on the drawings and preferred embodiments, and the purpose and effect of the present invention will become clearer. It should be understood that the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.

[0014] In the edge computing environment, the present invention is based on the Transformer large model structure framework, and by finding the optimal split point of model reasoning, the Transformer model is split into a low-level part and a high-level part, and a lightweight low-level model is deployed on user devices with limited computing power, and the computing-intensive high-level model is deployed on a cloud server.

[0015] The edge computing environment includes a user device and a cloud server. The bottom layer of the Transformer is deployed on the user device, and the high layer is deployed on the cloud server. Through the wireless communication link, the user device and the cloud server interact with each other, thereby achieving collaborative computing based on the model layering. The present invention adopts the Monte Carlo Tree Search (MCTS) method combined with the decision network and the value network to establish an adaptive dynamic segmentation point adjustment mechanism.

[0016] The user devices include smart phones, computers, smart wearable devices, etc.; the cloud server acts as a remote computing node to perform complex model reasoning and optimization decisions.

[0017] The optimization decision environment includes model segmentation point selection, user device computing load optimization, and cloud server computing resource allocation, using MCTS combined with a neural network method. In this environment, an optimization method combining a decision network and a value network is proposed to transform the performance optimization problem into a sequential decision problem, and find the optimal segmentation point and computing resource allocation strategy.

[0018] The hierarchical optimization method based on the Transformer model in the edge computing environment specifically includes the following steps: (1) The user device receives the input text sequence, converts it into an intermediate tensor through preliminary processing, and then sends the intermediate tensor to the cloud server through a wireless communication link for further processing.

[0019] (2) In order to optimize the reasoning performance of the system, a MCTS method combining decision network and value network is proposed to maximize the efficiency and accuracy of model reasoning.

[0020] (3) At each decision time, the optimal strategy is determined by analyzing the model inference results and system performance indicators, combined with the current segmentation point selection, user device computing load, and cloud server computing resource allocation. The user device and cloud server work together according to the determined strategy.

[0021] (4) After the cloud server processes the intermediate tensor, it returns the result to the user device through the wireless communication link. The user device performs the final processing and outputs the result.

[0022] Preferably, the environmental system in (1) includes the following features: all user devices are randomly distributed in the target area, and the wireless communication link between the user devices and the cloud server is stable and efficient; each user device can independently perform preliminary reasoning and send the intermediate tensor to the cloud server via the wireless link for further processing.

[0023] Preferably, in the Monte Carlo Tree Search (MCTS) method in (2), the agent is composed of a decision network and a value network. The purpose of the decision network is to generate strategies. The input of the decision network is the current system state, and the output is the probability distribution of each possible split point selection and computing resource allocation strategy. The purpose of the value network is to evaluate the value of the current system state. The input of the value network is the current system state, and the output is the value estimate of the state.

[0024] Preferably, the optimal strategy in (3) includes: processing the model reasoning results and system performance indicators in each decision time slot, determining the current optimal segmentation point selection and computing resource allocation strategy, so that the computing resources of the user device and the cloud server are optimally utilized, thereby improving the reasoning efficiency and accuracy of the system.

[0025] Preferably, the result return process in (4) adopts an efficient wireless communication link to ensure the timely return of the reasoning result; if the reasoning task is completed, the system enters a standby state and waits for new input.

[0026] A large language model optimization method based on edge computing specifically includes the following steps; Step 1, model establishment: the details are as follows; 1) System model: The system model of the present invention includes a Transformer model deployed on a user device and a cloud server. In order to optimize the performance of the model, the present invention splits and deploys the Transformer model on the user device and the cloud in an edge computing environment. For ease of representation, the user devices in the environment are represented by a set , the cloud server represents the set, In time period t, the computing resources of the user device and the cloud server are represented as and .

[0027] 2) Segmentation model: Figure 1 As shown, the Transformer model architecture diagram of the present invention segmented in the edge computing environment, the overall Transformer model is divided into X layers, where the Qth layer is the designated segmentation point; The method adopted by the present invention is to split the Transformer model, and the split points are recorded as Q . Assume that the total number of layers in the model is X , in the model Q Layer segmentation, front Q The layer deployment on the user device is represented as ,the remaining X - Q The layer is deployed on the cloud server as , where the split point Q is adjustable; for an input , through UE x Convert to intermediate tensor : in, represents the parameters deployed at the user device layer; the intermediate tensor y is transmitted to the cloud server through the wireless channel, and is affected by the Rayleigh fading channel gain and Gaussian white noise during the transmission, and is obtained : Among them, h represents the Rayleigh fading channel gain, and n represents Gaussian white noise; finally, it is processed by the cloud server to obtain : in, Indicates the parameters deployed in the cloud server layer.

[0028] 3) Constraint model: Since the computing resources of user devices and cloud servers are limited in the edge computing environment, the following constraints are given: in, Indicates the maximum computing resources of the user device, Indicates the maximum computing resources of the cloud server.

[0029] 4) Optimize the model: Select the appropriate split point Q When computing, it is necessary to consider the dynamic balance between computing resources and computing time, that is, while ensuring the completion of the task, it is also necessary to minimize the computing delay and transmission delay; the optimization model can be expressed as: in, Indicates that the user device calculates Q The time required for the layer, Indicates that the cloud server calculates the remaining X - Q The time required for the layer, Indicates the time required for data to be transmitted from the user device to the cloud server. The purpose of the present invention is to find the best split point, optimize the association strategy between user devices and cloud servers, and reasonably allocate computing resources to maximize the application capability of the Transformer model in the edge computing environment. The computing decision environment includes the selection of split points, the association decision between user devices and cloud servers, and the dynamic allocation of resources.

[0030] 1. Split point selection: Figure 2 In order to optimize the selection of segmentation points, the present invention proposes a MCTS method that combines decision networks and value networks. The specific implementation strategy is as follows: 1) Initialization: Initialize the MCTS tree, the root node represents the initial state, and the split point at this time is Q ; Among them, MCTS is Monte Carlo Tree Search; 2) Selection: In the MCTS tree, starting from the root node, the child nodes are selected through the decision network, the tree is expanded along the optimal path, and a node that is not fully expanded is selected; 3) Expansion: If the selected node is not fully expanded, expand the node according to the current state, generate new child nodes, and calculate the previous Q Layer and back X - Q The data transmission volume and computational requirements of the layer, the model can be expressed as: in, and Respectively indicate the front Q Layer and back X - QThe computational complexity of the layer, The amount of data transferred, B is bandwidth; 4) Simulation: Use the value network to simulate the newly expanded nodes, estimate the value of the current nodes, and obtain a reward value; 5) Backtracking: trace the reward value obtained from the simulation back to the root node, update the node value on the path, and adjust the parameters of the decision network and value network; Learning to update the node value, the formula is as follows: in: is in state s Next select action a The state-action value of is the learning rate, which determines the importance of new information in the update process; is the reward obtained at the current step, is the discount factor; is in the next state The maximum of all possible moves value; 6) Iteration: Repeat the above process of selection, expansion, simulation, and backtracking until the termination condition is met, thereby achieving the purpose of determining the optimal segmentation point.

[0031] 2. User devices are associated with cloud servers: in each time period t According to the result of the segmentation point selection, the association strategy between the user end and the cloud server is optimized. Similarly, the association strategy is iteratively adjusted according to the feedback value to achieve the purpose of minimizing the computation delay and transmission delay.

[0032] 3. Dynamic allocation of resources: After determining the optimal split point and associated strategy, dynamically adjust the allocation of resources according to the real-time load and task requirements. Set the following dynamic resource allocation strategy to maximize resource utilization while avoiding overload: first monitor the real-time computing load of user devices and cloud servers, then predict resource allocation based on the load, and finally implement resource reservation to cope with sudden computing needs. Dynamic resource allocation can be defined as: in, Indicates the relationship between the time required for the user device and the cloud server to calculate the corresponding number of layers and the corresponding current system load; is the current load.

[0033] According to the strategy determined by the above method, the user device and the cloud server dynamically allocate corresponding computing resources, and the cloud server returns the optimized computing results to the user device to complete the computing task. The above steps are repeated in the next computing cycle, and the segmentation point optimization and computing resource allocation are continuously performed to ensure the optimal system performance. Throughout the process, the collaborative work of the user device and the cloud server and the segmented deployment of the Transformer model are considered to be more in line with the actual application scenarios, so as to achieve efficient resource utilization, reduce latency, and improve the actual application capabilities of the Transformer model.

[0034] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the above examples, those skilled in the art can still modify the technical solutions recorded in the above examples or replace some of the technical features with equivalents. Any modification, equivalent replacement, etc. made within the spirit and principle of the invention should be included in the protection scope of the invention. All technical features in this embodiment can be freely combined according to actual needs.

[0035] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A large language model optimization method based on edge computing, characterized by: The specific steps include: Step 1, model building: the model includes a Transformer model deployed on a user device and a cloud server; in an edge computing environment, the Transformer model is split and deployed on the user device and the cloud; Among them, the user devices in the environment represent the set , the cloud server represents the set ; In the time period t, the computing resources of the user device and the cloud server are represented as and ; Step 2, segmentation model, specifically includes the following steps: Step 2.1: split the Transformer model and record the split points as Q ; Step 2.2, let the total number of layers of the model be X , in the model Q Layer segmentation, front Q The layer deployment on the user device is represented as ,the remaining X - Q The layer is deployed on the cloud server as , where the split point Q It is adjustable; Step 2.3, for an input , through UE x Convert to intermediate tensor : in, represents the parameters deployed at the user device layer; the intermediate tensor y is transmitted to the cloud server through the wireless channel, and is affected by the Rayleigh fading channel gain and Gaussian white noise during the transmission, and is obtained : Where, h represents the Rayleigh fading channel gain, and n represents Gaussian white noise; Step 2.4, processed by the cloud server : in, Indicates the parameters deployed in the cloud server layer; Step 3, Constraint Model: Since the computing resources of user devices and cloud servers are limited in the edge computing environment, the following constraints are given: in, Indicates the maximum computing resources of the user device, Indicates the maximum computing resources of the cloud server; Step 4: Optimize the model: Select appropriate split points Q When computing, it is necessary to consider the dynamic balance between computing resources and computing time, that is, while ensuring the completion of the task, it is also necessary to minimize the computing delay and transmission delay; the optimization model can be expressed as: in, Indicates that the user device calculates Q The time required for the layer, Indicates that the cloud server calculates the remaining X - Q The time required for the layer, Indicates the time required for data to be transmitted from the user device to the cloud server.

2. According to the large language model optimization method based on edge computing according to claim 1, it is characterized in that: The edge computing environment includes the selection of segmentation points, the association between user devices and cloud servers, and the dynamic allocation of resources.

3. According to a large language model optimization method based on edge computing according to claim 2, it is characterized in that: The specific implementation strategy for the segmentation point selection is as follows: 1) Initialization: Initialize the MCTS tree, the root node represents the initial state, and the split point at this time is Q ; Among them, MCTS is Monte Carlo Tree Search; 2) Selection: In the MCTS tree, starting from the root node, the child nodes are selected through the decision network, the tree is expanded along the optimal path, and a node that is not fully expanded is selected; 3) Expansion: If the selected node is not fully expanded, expand the node according to the current state, generate new child nodes, and calculate the previous Q Layer and back X - Q The data transmission volume and computational requirements of the layer, the model can be expressed as: in, and Respectively indicate the front Q Layer and back X - Q The computational complexity of the layer, The amount of data transferred, B is bandwidth; 4) Simulation: Use the value network to simulate the newly expanded nodes, estimate the value of the current nodes, and obtain a reward value; 5) Backtracking: trace the reward value obtained from the simulation back to the root node, update the node value on the path, and adjust the parameters of the decision network and value network; Learning to update the node value, the formula is as follows: in: is in state s Next select action a The state-action value of is the learning rate, which determines the importance of new information in the update process; is the reward obtained at the current step, is the discount factor; is in the next state The maximum of all possible moves value; 6) Iteration: Repeat the above process of selection, expansion, simulation, and backtracking until the termination condition is met, thereby achieving the purpose of determining the optimal segmentation point.

4. According to a large language model optimization method based on edge computing according to claim 2, it is characterized in that: The user device is associated with the cloud server as follows: In each time period t According to the result of segmentation point selection, the association strategy between the user end and the cloud server is optimized; the association strategy is iteratively adjusted according to the feedback value to achieve the purpose of minimizing the calculation delay and transmission delay.

5. The large language model optimization method based on edge computing according to claim 2 is characterized in that: The dynamic allocation of resources is as follows: After determining the optimal split point and associated strategy, dynamically adjust resource allocation according to real-time load and task requirements; set the following dynamic resource allocation strategy to maximize resource utilization while avoiding overload: First, the real-time computing load of user devices and cloud servers is monitored, then resource allocation is predicted based on the load, and finally resource reservation is implemented to cope with sudden computing demands; Dynamic resource allocation can be defined as: in, Indicates the relationship between the time required for the user device and the cloud server to calculate the corresponding number of layers and the corresponding current system load; is the current load.

Citation Information

Patent Citations

  • Edge computing task allocation method based on deep Monte Carlo tree search

    CN110427261A

  • Radar segmentation method fusing space attention and self-attention transformation network

    CN114119615A

  • Deep learning model division method and device for streamline distributed end-cloud collaborative reasoning

    CN115130649A

  • Neural network model segmentation and resource allocation method in edge computing system

    CN115913955A

  • Charging pile power distribution control method, system and equipment

    CN118636728A

Cited By

  • AI algorithm process and service all-in-one machine based on edge computing

    CN119690673A

  • Efficient large language model adaptation method based on server-free edge computing

    CN121072646A