Efficient large language model reasoning method based on distributed edge cloud
The distributed edge cloud approach optimizes large language model reasoning by classifying devices and partitioning models, addressing latency and resource constraints to enhance efficiency and practicality in edge computing.
Patent Information
- Application Number
- CN202411436438.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2025-01-03
AI Technical Summary
Centralized cloud computing architectures for large language models (LLMs) face challenges such as privacy issues, usage restrictions, and latency, hindering widespread acceptance and real-time data processing needs.
A distributed edge cloud approach is implemented, where user equipment and edge nodes are classified into GPU, memory, and hybrid groups using a depth-first search tree, and the LLM is partitioned using a Markov decision process to optimize model deployment and resource usage.
This method accelerates reasoning speed, optimizes resource usage, and improves performance by making full use of heterogeneous devices, ensuring efficient and practical large language model reasoning in edge computing environments.
Smart Images
Figure CN119248502A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of Internet of Things, and in particular to an efficient large language model reasoning method, device and storage medium based on distributed edge cloud. Background Art
[0002] With the advent of the Internet of Everything era and the popularization of 5G communication networks, the scale of Internet data is growing exponentially. The ultra-low latency, massive data, edge intelligence and other characteristics of edge cloud have prompted more companies to choose edge cloud technology solutions, making edge cloud computing an important component between the central cloud and the terminal in the market. The emergence of large language models (LLMs) has attracted widespread attention from the public, industry, and academia, representing a major breakthrough in artificial intelligence (AI). The outstanding capabilities of LLMs make them valuable and perform well in a wide range of applications, from chatbots and content generation (e.g., text summarization and code generation) to tools to assist education and research. In this context, the centralized architecture of traditional cloud computing can no longer meet the end-users' needs for timeliness, capacity, and computing power.
[0003] With the rapid development of artificial intelligence technology, more and more application scenarios require real-time and efficient data processing and decision-making on the edge side, which has driven the rapid growth of demand for large model edge deployment. The current market size has not yet reached a huge level, but with the continuous improvement of the performance of edge computing devices and breakthroughs in algorithm optimization technology. The impressive generation potential of LLM stems from its complex structure with billions of neurons, which leads to resource-intensive training and reasoning. Therefore, LLM is mainly hosted in the cloud. However, this centralized approach brings challenges such as privacy issues, usage restrictions, and latency, which hinder the wider acceptance of GAI. Summary of the invention
[0004] The purpose of the present invention is to solve the shortcomings of the prior art and provide an efficient large language model reasoning method based on a distributed edge cloud, comprising the following steps: S1: Assign user equipment UE and edge node EN to classification groups including GPU priority group, memory priority group and hybrid group according to the depth-first search tree; S2: The Markov decision process obtains a split point according to the training set, divides the inference model according to the split point to obtain a partition model, and deploys the partition model to each group in the classification group to obtain an optimized model; S3: Input the test set into the optimization model for reasoning to obtain reasoning results.
[0005] Preferably, in step S1, the step of allocating the user equipment UE and the edge node EN to classification groups including a GPU priority group, a memory priority group and a hybrid group according to a depth-first search tree comprises the following steps: S11: The user device and the edge node construct the depth-first search tree according to GPU performance and / or memory capacity; S12: Selecting a number of the user devices and / or the edge nodes with good GPU performance in the depth-first search tree to add to a GPU priority group, and selecting a number of the user devices and / or the edge nodes with large memory capacity to add to a memory priority group; S13: Select the user device and / or the edge node from the GPU priority group and the memory priority group to join the hybrid group.
[0006] Preferably, in step S2, the Markov decision process obtains a split point according to the training set, and divides the inference model according to the split point to obtain a partition model, further comprising: The policy gradient enhanced PPO obtains the training set, and obtains the expected discounted return according to the policy network and the value function network; The Markov decision process obtains a split point based on the expected discounted return.
[0007] Preferably, the policy gradient enhanced PPO obtains the training set, and obtains the expected discounted return according to the policy network and the value function network, further comprising: The policy gradient enhanced PPO obtains the training set, and the policy network calculates the current policy parameters according to the training set. and the last strategy parameters , according to the current policy parameters and the last policy parameters Calculate strategy ratios; Parameters are calculated based on the value function network , according to the parameters calculating an advantage estimate by generalized estimated advantage, and limiting the magnitude of policy updates according to the advantage estimate; The truncated surrogate objective function obtains the expected discounted return based on the strategy ratio and the advantage estimate.
[0008] Preferably, the parameters Calculating an advantage estimate by generalized estimated advantage, limiting the magnitude of policy updates according to the advantage estimate, further comprising: The advantage estimate is calculated by generalized estimated advantage, and the calculation formula of the advantage estimate is as follows: in, represents the time difference error, is the discount factor, is the GAE parameter that controls the bias-variance tradeoff, is the current state, is the index of the time step.
[0009] Preferably, the truncated proxy objective function obtains the expected discounted return according to the advantage estimate and the strategy estimate, further comprising: The proxy objective is obtained according to the truncated proxy objective function, and the expected discounted return is obtained according to the proxy objective. The calculation formula of the truncated proxy objective function is as follows: in, represents the strategy parameters of the training set, is the action at the current moment, is the current state, is a hyperparameter, is the strategy ratio, through Make sure the policy ratio does not deviate significantly from 1 to prevent large, destabilizing updates; Among them, the truncated proxy objective updates the strategy parameters according to the stage proxy objective function , the expected discounted return is calculated according to the update strategy parameters, and the calculation formula of the update strategy parameters is as follows: in, is the learning rate.
[0010] Preferably, the Markov decision process obtains the split point according to the expected discounted return, further comprising: The expected discounted return approximates the long-term overall goal, and the future state is calculated based on the long-term overall goal; A reward in the UE network is calculated according to the future state, and a split point is obtained according to the reward, wherein the calculation formula of the reward is: The Markov decision process is composed of a tuple Composition, where the state space Include noise intensity , Rayleigh fading scale and the current split point The key factors include , action space including the split point There are three possible actions: move up or move down, or keep the current split point 0. Mathematically, , for the time step , given the state The environment state will transition to the next state according to the transition probability P, that is, the split point P. , depending on the selected action and real-time channel conditions.
[0011] Preferably, the decoding layer network obtains the predicted workload according to the workload sequence and the encoding parameters output by the most effective static feature encoder layer, further comprising: The calculation formula for the long-term overall goal is as follows: in, is the parameter, discount factor Involving the importance of future rewards, and the discount factor Learning by Parameterized strategy parameters To obtain the maximum value of the calculation formula for the long-term overall goal.
[0012] Based on the same concept, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of an efficient large language model reasoning method based on a distributed edge cloud as described in any one of the embodiments.
[0013] Based on the same concept, the present invention also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the efficient large language model reasoning method based on distributed edge cloud as described in any one of the embodiments.
[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention allocates user equipment UE and edge node EN to classification groups including GPU priority group, memory priority group and hybrid group according to a depth-first search tree, thereby accelerating the reasoning speed while making full use of heterogeneous devices, and improving the performance of large language model reasoning in an edge computing environment through the depth-first search tree, and also ensuring that resource usage is optimized while meeting complex user needs, demonstrating high efficiency and practicality in actual deployment.
[0015] The present invention obtains splitting points according to a training set through a Markov decision process, divides the inference model according to the splitting points to obtain a partitioning model, and deploys the partitioning model to each group in the classification group, thereby reducing the load of the user equipment.
[0016] The present invention achieves a significant improvement in computational efficiency by approximating the expected discounted return to the long-term overall goal, allowing the RL agent to flexibly adapt to the changing deployment environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.
[0018] Figure 1 This is a flow chart of an efficient large language model reasoning method based on distributed edge cloud in the present invention. DETAILED DESCRIPTION
[0019] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.
[0020] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.
[0021] In order to achieve efficient reasoning of LLM on resource-constrained edge devices while ensuring that the user's requirements for reasoning response time (latency) and result accuracy are met, this embodiment aims to maximize the processing throughput of reasoning tasks while achieving efficient resource management and task scheduling under the constraints of strict resource constraints and the diversity of user requirements.
[0022] The optimization objectives focus on two main aspects: increasing throughput and meeting quality of service (QoS) standards. Specifically: Throughput maximization: The main goal is to maximize the number of user requests successfully processed per unit time under given resource constraints, and to improve the efficiency of edge devices by intelligently scheduling batch processing and resource allocation; Latency and accuracy guarantee: During the inference process, it is necessary to ensure that each user request can be responded to within its specific latency tolerance threshold, and the inference results must meet the predetermined accuracy standards. This requires the optimization model to be not only fast but also accurate, especially when dealing with application scenarios such as real-time interaction or critical decision support.
[0023] This embodiment first sets computing resource constraints, memory resource constraints, network bandwidth constraints, and quality of service (QoS) constraints.
[0024] Computing resource constraints include the processing power of the CPU and GPU of edge devices, which directly affects the number of tasks that can be processed in parallel and the processing speed of a single task.
[0025] Memory resource constraints: This involves the RAM and storage space of the edge device, which determines the size and number of models that can be loaded into memory for inference at the same time.
[0026] Network bandwidth constraints: This includes the uplink and downlink network bandwidth of edge devices, which affects the speed and latency of data transmission, especially when the model needs to frequently exchange data with the cloud or other edge nodes.
[0027] Quality of Service (QoS) constraints: requirements specific to each user or task, such as maximum tolerated latency, data privacy requirements, and accuracy of inference results.
[0028] See also Figure 1 As shown, this embodiment provides an efficient large language model reasoning method based on a distributed edge cloud, comprising the following steps: S1: Assign user equipment UE and edge node EN to classification groups including GPU priority group, memory priority group and hybrid group according to the depth-first search tree; This embodiment aims to optimize the reasoning efficiency and performance of large language models (LLM) in edge computing environments. It mainly consists of three parts: user equipment (UE), edge node (EN), and cloud server. This three-layer architecture design allows fast data processing on devices close to users, while using the powerful computing power of cloud servers for in-depth data analysis and storage, thereby balancing latency, cost, and computing power.
[0029] Generally, in order to maximize the utilization efficiency of hardware resources, the system classifies UE and EN according to the indicators of memory capacity and GPU performance, and divides them into two main groups: G group and M group, namely GPU priority group and memory priority group. The devices in the GPU priority group are equipped with high-performance GPUs, which are suitable for processing computing-intensive tasks. The devices in the memory priority group have large memory capacity, which can realize fast reading and storage of data.
[0030] Preferably, in step S1, the user equipment UE and the edge node EN are assigned to classification groups including a GPU priority group, a memory priority group and a hybrid group according to a depth-first search tree, including the following steps: S11: The user device and the edge node construct a depth-first search tree based on GPU performance and / or memory capacity; S12: Selecting several user devices and / or edge nodes with good GPU performance in the depth-first search tree to join the GPU priority group, and selecting several user devices and / or edge nodes with large memory capacity to join the memory priority group. Specifically, in this embodiment, two independent device groups, namely the GPU priority group G and the memory priority group M, are used for prompt application and token processing; S13: Select user devices and / or edge nodes from the GPU priority group and the memory priority group to join the hybrid group. Specifically, in this embodiment, a new virtual group level, namely hybrid group F, is established. The group is composed of some devices in groups G and M, and can be expanded and shrunk according to workload needs. There are a certain number of edge nodes and user devices in each group, and each user device is linked to an edge node.
[0031] Since the performance of user devices is limited and heterogeneous in distributed scenarios, some devices with weaker performance cannot generate prompts and tokens, so the optimization model will be split in each group.
[0032] To characterize the LLM configuration with model partitioning, we mainly consider the model of a general system. Without loss of generality, we assume that for an L-layer LLM, the first p layers LU(·) are deployed at the UE, and the remaining L−p layers LC(·) are located at the edge. The split point p is adjustable, and the application of RL is studied to dynamically optimize the split point of the LLM across user devices and edge computing resources.
[0033] S2: The Markov decision process obtains the splitting point according to the training set, divides the inference model according to the splitting point to obtain the partitioning model, and deploys the partitioning model to each group in the classification group to obtain the optimized model.
[0034] Preferably, in step S2, the Markov decision process obtains a split point according to the training set, and divides the inference model according to the split point to obtain a partition model, further comprising: Policy gradient reinforcement PPO obtains the training set and obtains the expected discounted return based on the policy network and value function network; The Markov decision process obtains splitting points based on expected discounted returns.
[0035] Preferably, the policy gradient enhanced PPO obtains a training set, obtains an expected discounted return according to a policy network and a value function network, and further includes: Policy gradient reinforcement PPO obtains the training set, and the policy network calculates the current policy parameters based on the training set and the last strategy parameters , according to the current policy parameters and the last strategy parameters Calculate strategy ratios; Calculate parameters based on the value function network , according to the parameters Calculate the advantage estimate through the generalized estimated advantage, and constrain the policy update amplitude according to the advantage estimate; The truncated surrogate objective function obtains the expected discounted return based on the strategy ratio and advantage estimate.
[0036] Preferably, according to the parameters Calculating an advantage estimate by generalized estimated advantage, limiting the policy update amplitude according to the advantage estimate, further comprising: The advantage estimate is calculated by generalized estimated advantage. The advantage estimate is calculated as follows: in, represents the time difference error, is the discount factor, is the GAE parameter that controls the bias-variance tradeoff, is the current state, is the index of the time step.
[0037] Preferably, the truncated proxy objective function obtains the expected discounted return based on the advantage estimate and the strategy estimate, further comprising: The proxy objective is obtained according to the truncated proxy objective function, and the expected discounted return is obtained according to the proxy objective. The calculation formula of the truncated proxy objective function is as follows: in, represents the strategy parameters of the training set, is the action at the current moment, is the current state, is a hyperparameter, is the strategy ratio, through Make sure the policy ratio does not deviate significantly from 1 to prevent large, destabilizing updates; Among them, the truncated proxy objective updates the strategy parameters according to the stage proxy objective function , the expected discounted return is calculated based on the updated strategy parameters. The calculation formula for the updated strategy parameters is as follows: in, is the learning rate. Specifically, in this embodiment, the strategy parameter The update is to use the truncated proxy objective function This gradient ascent ensures that the policy is iteratively updated to maximize the expected reward while maintaining stability through the clipping mechanism.
[0038] Preferably, the Markov decision process obtains the split point according to the expected discounted return, further comprising: The expected discounted return approximates the long-term overall goal, and future states are calculated based on the long-term overall goal; The reward in the UE network is calculated based on the future state, and the split point is obtained based on the reward. The calculation formula of the reward is: A Markov decision process consists of a tuple Composition, where the state space Include noise intensity , Rayleigh fading scale and the current split point The key factors include , action space Include the split point There are three possible actions: move up or move down, or keep the current split point 0. Mathematically, , for the time step , given the state The environment state will transition to the next state according to the transition probability P, that is, the split point P. , depending on the selected action and real-time channel conditions.
[0039] Better yet, integrating MBRL into the process of splitting the LLM model to obtain the optimized model significantly improves the efficiency and effectiveness of the RL method. However, the slow reasoning ability of LLM makes the learning process slow. Therefore, inspired by the use of predictive models to simulate the environment in classic MBRL, a proxy model is used to approximate the reward function to improve learning efficiency. Specifically, the approximation requires running the PPL calculated by LLM. , and calculated by Parameterized DNN-based proxy model gPPL , to minimize the mean square error (MSE), as follows: It is worth noting that cross-validation is used to prevent overfitting and ensure the generalizability of the proxy model. Accordingly, during the RL training process, the advantage estimate in the calculation formula is can be rewritten as: On this basis, we can calculate ), so as to facilitate By adopting this model-based approach, we achieve a substantial improvement in computational efficiency, enabling RL agents to flexibly adapt to changing deployment environments.
[0040] Preferably, the decoding layer network obtains the predicted workload according to the workload sequence and the encoding parameters output by the most effective static feature encoder layer, further comprising: The calculation formula for the long-term overall goal is as follows: in, is the parameter, discount factor Involving the importance of future rewards, and the discount factor Learning by Parameterized strategy parameters To obtain the maximum value of the calculation formula for the long-term overall goal.
[0041] S3: Input the test set into the optimization model for inference to obtain the inference result.
[0042] All devices are pre-installed with optimized models. When a new inference request arrives in the test set, the scheduler assigns the inference request to a pair of devices (i.e., one device belongs to group G and one device belongs to group M). The device g used for prompt generation will be responsible for processing all input prompt tokens and generating kv-buffers in the prompt phase, and sending the kv-buffers to the token processing device m to generate the first token. The token processing device then continues to generate tokens until the response is completed. Continuous batching is used on the token processing device to maximize the utilization of the device.
[0043] This embodiment uses a hierarchical two-level scheduling, with the cluster-level scheduler (CLS) 1 responsible for device pool management and routing incoming inference requests, and the device-level scheduler (MLS) 2 maintaining the pending queue and managing request batches on each device.
[0044] Hint generation groups, token handling groups, and hybrid device groups, namely GPU priority groups, memory priority groups, and hybrid groups, are maintained by the cluster-level scheduler CLS. Devices are initially assigned to the hint generation group or token handling group based on the expected request load and input / output token distribution, and then devices from the hint generation group or token handling group are dynamically moved into or out of the hybrid device group to reduce fragmentation and meet SLOs at higher loads. Devices in the hybrid device group retain their identity as hint generation devices or token handling devices and return to their original group when there are no tasks of the opposite type in the waiting queue. Switching groups does not incur any noticeable latency. If the load distribution deviates significantly from the initial assumptions, a coarse-grained device reshuffling is employed and devices are moved between the hint generation group and token handling group. Device reshuffling is done rarely, usually only when they stay in the hybrid device group for a considerable amount of time.
[0045] The cluster-level scheduler CLS assigns hint generation devices and token handling devices to each request by scheduling in a join-shortest-queue (JSQ) schedule, where the queue length is defined by the number of pending tokens. Each device periodically communicates changes in its memory capacity or pending queue to the cluster-level scheduler CLS. Note that this does not happen at every iteration boundary. Hint generation devices and token handling devices are simultaneously scheduled when scheduling requests, because kv-buffer transfers can be overlapped with hint calculations to reduce transfer overhead. When routing a request, if the pending queue is larger than a certain threshold, the cluster-level scheduler CLS looks for the target device in the mixed device group. If the mixed device group is also full, it will continue to look for the opposite group and move the device to the mixed device group. Devices in the mixed device group operate in a way that once the mixed request queue is exhausted, the cluster-level scheduler CLS will transition the device back to its original group. For example, when the queue is too long, a hint generation device can be moved to the mixed device group to run tokens; once the device has finished running tokens, we transition the device back to the hint generation group.
[0046] The device-level scheduler MLS runs on each device and is responsible for tracking GPU memory utilization, maintaining the pending queue, deciding the batch for each iteration, and reporting relevant status to CLS.
[0047] The device-level scheduler MLS uses only First Come First Served (FCFS) to schedule hints. In testing, throughput drops after 2048 hint tokens, so MLS limits batching of multiple hints to a total of 2048 tokens. This is a configurable value that can be changed for different models or hardware.
[0048] The device-level scheduler MLS uses FCFS to schedule tokens and batches as best it can. The token generation throughput scales with the batch size until the device runs out of memory. Therefore, MLS keeps track of memory and starts queuing tokens when the device is about to run out of memory.
[0049] To meet the TTFT SLO, the device-level scheduler MLS must prioritize running hints and immediately schedule any new hints in the pending queue. If the device is running a token phase and does not have additional capacity to run a hint phase, the MLS will preempt a token. To avoid token phase starvation due to preemption, tokens are prioritized as they age and the number of preemptions each request can have is limited.
[0050] The depth-first search tree may also be used during the use of the cluster-level scheduler (CLS) 1 and the device-level scheduler (MLS) 2.
[0051] More preferably, depth-first search (DFS) is an algorithm for traversing or searching a tree or graph. In the context of edge computing inference optimization, DFS is used to explore different combinations of resource allocation and request scheduling to find the optimal solution that meets all user requirements and optimizes system performance.
[0052] The implementation details of the depth-first search tree DFTSP of this embodiment include the construction of the search tree, the exploration of nodes and the pruning logic, each of which is the key to optimizing the edge computing reasoning process.
[0053] The search tree is constructed dynamically according to the current system status and all pending user requests. Each node contains decision information for processing one or more requests, such as resource allocation and the execution order of requests, for example, deciding whether to execute a user's request and its corresponding resource configuration. Branch generation mechanism: The branches of the tree are generated based on priority rules, such as the urgency of the request, resource demand, and contribution to throughput.
[0054] The exploration of nodes adopts a depth-first strategy. By evaluating the feasibility and potential benefits of each decision path, the paths that are expected to bring the highest benefits are explored first. First, the search starts from the root node of the search tree, which represents the initial state without any resource allocation and request scheduling. For the current node, the algorithm considers all possible operations, such as allocating specific resources to a request or scheduling a request for processing. For each possible operation, a child node is generated and explored in depth for node expansion; a child node is selected to continue in-depth exploration. This process is recursive until a leaf node is reached, that is, a complete resource allocation and request scheduling solution is explored in depth; if a leaf node is reached or it is determined that the current path cannot be the optimal solution, the algorithm will backtrack to the previous decision point and explore other possible child nodes for backtracking.
[0055] The pruning logic is that if the current path can no longer meet the latency requirements of some users or exceeds the resource limit, or its potential benefit is lower than the minimum benefit of the explored path, it will be pruned immediately. In addition, if the resource usage of a path has reached or exceeded the limit of the edge device, it will also be pruned. Online pruning is a key technology for optimizing the depth-first search process. It avoids invalid calculations and resource waste by eliminating those paths that are unlikely to produce optimal results in real time.
[0056] Pruning can be performed based on the following criteria: Resource constraints: If the resource allocation under a certain path exceeds the capacity of the edge device or violates the preset constraints (such as memory limit, processing power, etc.), the path is pruned.
[0057] Latency requirements: If the current path cannot meet the latency requirements of certain requests, that is, it cannot complete the inference within the time specified by the user, the path will be pruned.
[0058] Cost-effectiveness analysis: By estimating the cost-effectiveness of each path, that is, the ratio of the throughput gain it can bring to the resource consumption, the low-efficiency paths are pruned.
[0059] Historical record comparison: The performance records of the explored paths are compared, and the paths that are lower than the optimal records are pruned.
[0060] By combining depth-first search with online pruning, the DFTSP algorithm can significantly improve search efficiency and reduce computing time. Specifically: Reduce invalid exploration: The pruning operation effectively reduces the exploration of hopeless paths, saving a lot of computing resources and time.
[0061] Accelerate the decision-making process: By quickly eliminating unfeasible or low-efficiency options, the process of reaching the optimal solution is accelerated.
[0062] Dynamically adjust strategies: Dynamically adjust search strategies during the exploration process to respond to real-time changes in network conditions and device performance, ensuring the real-time and accuracy of the solution.
[0063] Through this highly optimized search and pruning strategy, the DFTSP algorithm not only improves the performance of large language model reasoning in edge computing environments, but also ensures optimized resource usage while meeting complex user needs, demonstrating its efficiency and practicality in actual deployment.
[0064] In some embodiments of the present application, a computer device is also provided, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the efficient large language model reasoning method based on a distributed edge cloud as described in any one of Example 1.
[0065] The present invention also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, enables the one or more processors to perform the steps of the efficient large language model reasoning method based on distributed edge cloud as described in any one of Example 1.
[0066] It is understandable that for the aforementioned efficient large language model reasoning method based on distributed edge cloud, if it is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0067] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.
[0068] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
Claims
1. An efficient large language model reasoning method based on distributed edge cloud, characterized in that: The following steps are involved: S1: Assign user equipment UE and edge node EN to classification groups including GPU priority group, memory priority group and hybrid group according to the depth-first search tree; S2: The Markov decision process obtains a split point according to the training set, divides the inference model according to the split point to obtain a partition model, and deploys the partition model to each group in the classification group to obtain an optimized model; S3: Input the test set into the optimization model for reasoning to obtain reasoning results.
2. The efficient large language model reasoning method based on distributed edge cloud according to claim 1 is characterized in that: In step S1, the user equipment UE and the edge node EN are assigned to classification groups including a GPU priority group, a memory priority group and a hybrid group according to a depth-first search tree, including the following steps: S11: The user device and the edge node construct the depth-first search tree according to GPU performance and / or memory capacity; S12: Selecting a number of the user devices and / or the edge nodes with good GPU performance in the depth-first search tree to add to a GPU priority group, and selecting a number of the user devices and / or the edge nodes with large memory capacity to add to a memory priority group; S13: Select the user device and / or the edge node from the GPU priority group and the memory priority group to join the hybrid group.
3. The efficient large language model reasoning method based on distributed edge cloud according to claim 2 is characterized in that: In step S2, the Markov decision process obtains a split point according to the training set, and divides the inference model according to the split point to obtain a partition model, further comprising: The policy gradient enhanced PPO obtains the training set, and obtains the expected discounted return according to the policy network and the value function network; The Markov decision process obtains a split point based on the expected discounted return.
4. The efficient large language model reasoning method based on distributed edge cloud according to claim 3 is characterized in that: The policy gradient enhanced PPO obtains the training set, and obtains the expected discounted return according to the policy network and the value function network, further comprising: The policy gradient enhanced PPO obtains the training set, and the policy network calculates the current policy parameters according to the training set. and the last strategy parameters , according to the current policy parameters and the last policy parameters Calculate strategy ratios; Parameters are calculated based on the value function network , according to the parameters calculating an advantage estimate by generalized estimated advantage, and limiting the magnitude of policy updates according to the advantage estimate; The truncated surrogate objective function obtains the expected discounted return based on the strategy ratio and the advantage estimate.
5. The efficient large language model reasoning method based on distributed edge cloud according to 4 is characterized in that: According to the parameters Calculating an advantage estimate by generalized estimated advantage, limiting the policy update amplitude according to the advantage estimate, further comprising: The advantage estimate is calculated by generalized estimated advantage, and the calculation formula of the advantage estimate is as follows: in, represents the time difference error, is the discount factor, is the GAE parameter that controls the bias-variance tradeoff, is the current state, is the index of the time step.
6. The efficient large language model reasoning method based on distributed edge cloud according to claim 5 is characterized in that: The truncated proxy objective function obtains an expected discounted return based on the advantage estimate and the policy estimate, further comprising: The proxy objective is obtained according to the truncated proxy objective function, and the expected discounted return is obtained according to the proxy objective. The calculation formula of the truncated proxy objective function is as follows: in, represents the strategy parameters of the training set, is the action at the current moment, is the current state, is a hyperparameter, is the strategy ratio, through Make sure the policy ratio does not deviate significantly from 1 to prevent large, destabilizing updates; Among them, the truncated proxy objective updates the strategy parameters according to the stage proxy objective function , the expected discounted return is calculated according to the update strategy parameters, and the calculation formula of the update strategy parameters is as follows: in, is the learning rate.
7. The efficient large language model reasoning method based on distributed edge cloud according to claim 6 is characterized in that: The Markov decision process obtains a split point according to the expected discounted return, further comprising: The expected discounted return approximates the long-term overall goal, and the future state is calculated based on the long-term overall goal; A reward in the UE network is calculated according to the future state, and a split point is obtained according to the reward, wherein the calculation formula of the reward is: The Markov decision process is composed of a tuple Composition, where the state space Include noise intensity , Rayleigh fading scale and the current split point The key factors include , action space including the split point There are three possible actions: move up or move down, or keep the current split point 0. Mathematically, , for the time step , given the state The environment state will transition to the next state according to the transition probability P, that is, the split point P. , depending on the selected action and real-time channel conditions.
8. The efficient large language model reasoning method based on distributed edge cloud according to claim 7 is characterized in that: The decoding layer network obtains a predicted workload according to the workload sequence and the encoding parameters output by the most effective static feature encoder layer, and further include: The calculation formula for the long-term overall goal is as follows: in, is the parameter, discount factor Involving the importance of future rewards, and the discount factor Learning by Parameterized strategy parameters To obtain the maximum value of the calculation formula for the long-term overall goal.
9. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the efficient large language model reasoning method based on a distributed edge cloud as described in any one of claims 1 to 8.
10. A storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the efficient large language model reasoning method based on a distributed edge cloud as described in any one of claims 1 to 8.