A task allocation method, device and equipment based on a large language model and a medium

Abstract

This application belongs to the field of task allocation technology, and relates to a task allocation method, apparatus, device, and medium based on a large language model. The method includes: acquiring tasks to be allocated; dividing the large language model into various edge servers, ensuring each edge server meets partitioning constraints, and obtaining an initial pipeline configuration result; establishing a global configuration problem based on partitioning constraints, energy consumption constraints, and effective throughput; performing an equivalent transformation based on the effective capabilities of the edge servers to convert the global configuration problem into a spatial subproblem; solving the spatial subproblem based on the initial pipeline configuration result to obtain a feasible pipeline configuration result; and obtaining the final pipeline configuration result based on the feasible pipeline configuration result, enabling the distributed edge server cluster to match renewable energy conditions and complete task allocation. This application enables pipeline configuration to adapt to renewable energy conditions that change over time.

Description

Technical Field

[0001] This application relates to the field of task allocation technology, and in particular to a task allocation method, apparatus, device and medium based on a large language model. Background Technology

[0002] With the advancement of science and technology, large language models are profoundly changing computing paradigms. To reduce response latency, conserve wide-area bandwidth, and enhance data privacy, an increasing number of systems are attempting to migrate Large Language Model (LLM) inference from cloud data centers to edge environments. However, edge-side LLM services are inherently energy-intensive; even a single LLM inference request can consume significant amounts of energy, making sustainable deployment on resource-constrained edge environments difficult. This is driving a shift from grid-dominated operations to self-powered edge systems. By directly utilizing intermittent but green renewable energy sources, the edge promises to achieve more environmentally friendly distributed inference.

[0003] However, renewable energy-driven edge LLM inference is not simply about "adding energy constraints to the inference task." Its fundamental difficulty lies in the fact that energy fluctuations occur simultaneously across multiple time scales: on the one hand, macro-level fluctuations at the minute and hour levels alter the sustainable service capacity of nodes within the current time period; on the other hand, short-term fluctuations at the second and millisecond levels further perturb runtime execution in a finer-grained manner. The former can usually be tracked at the scheduling level using recent generation trends or short-term forecasts, and is therefore suitable for handling through coarse-grained pipeline reconfiguration; the latter often occurs within a single scheduling epoch, faster than the repartitioning operation itself, yet sufficient to disrupt what initially seemed like a reasonable configuration. In other words, static layer partitioning or periodic rescheduling alone cannot completely solve the problem.

[0004] In existing technologies, task allocation methods based on large language models can be roughly divided into three categories.

[0005] The first category comprises distributed inference systems targeting deep neural networks (DNNs) and LLMs. These works typically improve system throughput through mechanisms such as model parallelism, pipelined execution, and runtime scheduling. Representative works include GPipe, DAPPLE, PipeDream, Alpa, and more recently, Galaxy, Galaxy+, DeTransformer, Asteroid, and Jupiter for edge computing. Meanwhile, service systems such as vLLM, Orca, Llumnix, and ServerlessLLM further enhance resource utilization through batch processing, memory optimization, migration, and fast loading. However, these systems generally prioritize performance optimization, typically assuming stable power supply or statically unchanged node capabilities, and have not yet dynamically adjusted pipeline structures and execution behaviors to accommodate fluctuations in renewable energy.

[0006] The second category is research on green computing and renewable energy-aware scheduling. Extensive work in this area has focused on cloud and edge environments, such as data center resource allocation, green edge scheduling and task offloading, renewable energy-aware federated learning and battery scheduling, and electric vehicle-assisted edge offloading. These studies can optimize jobs, tasks, or training rounds under energy-constrained conditions, but they do not characterize the stage dependencies in pipelined parallel LLM inference, nor do they consider the high latency sensitivity of autoregressive decoding.

[0007] The third category comprises recent work on optimizing large language model systems. These studies primarily focus on latency, memory efficiency, or training throughput, such as speculative decoding in collaborative scenarios, efficient long contexts and multi-GPU execution, and bubble hiding during training. Other training system and measurement studies have further analyzed the heterogeneous and power-constrained distributed execution characteristics. However, these works have not addressed inference control issues driven by renewable energy conditions, nor have they investigated how semantic decomposition can improve execution resilience under runtime instability caused by renewable energy sources.

[0008] In summary, existing technologies either assume resource stability, consider renewable energy sensing and scheduling only at the job or task level, or focus only on LLM execution optimization without adapting to renewable energy fluctuations. The strategies present a clear dilemma: either they are highly dependent on grid power supply to maintain relatively stable inference latency, or they suffer significant latency jitter when strictly following renewable energy supply. Summary of the Invention

[0009] Therefore, it is necessary to provide a task allocation method, apparatus, device, and medium based on a large language model to address the above-mentioned technical problems, which enables pipeline configuration to adapt to renewable energy conditions that change over time.

[0010] A task allocation method based on a large language model includes: Obtain tasks to be assigned, including: large language model, renewable energy conditions, and distributed server cluster, wherein the distributed server cluster includes multiple edge servers; The large language model is partitioned across edge servers, ensuring each edge server meets partitioning constraints, thus obtaining the initial pipeline configuration result, which includes spatial decision and batch size. Based on renewable energy conditions and multiple edge servers, the energy consumption constraints for each edge server are obtained. Based on multiple edge servers, the effective throughput of the distributed server cluster is obtained. Using spatial decision and batch size as decision variables, and based on partitioning constraints, energy consumption constraints, and effective throughput, a global configuration problem is established. Based on renewable energy conditions and distributed server clusters, the effective capacity of edge servers is obtained; with the goal of each edge server assuming the number of layers it undertakes equal to its effective capacity, an equivalent transformation is performed based on the effective capacity of the edge servers to transform the global configuration problem into a spatial subproblem. Based on the initial results of the pipeline configuration, a feasibility search is performed on the batch size, and discrete projection is executed to solve the spatial subproblem, thereby obtaining the feasible results of the pipeline configuration. Based on the feasible results of the pipeline configuration, the final result of the pipeline configuration is obtained, enabling the distributed edge server cluster to match renewable energy conditions and complete task allocation.

[0011] A task allocation device based on a large language model includes: The first module is used to obtain tasks to be assigned, including: a large language model, renewable energy conditions, and a distributed server cluster, wherein the distributed server cluster includes multiple edge servers. The second module is used to partition the large language model onto various edge servers, ensuring that each edge server meets the partitioning constraints, and to obtain the initial results of the pipeline configuration, which includes spatial decision and batch size; based on renewable energy conditions and multiple edge servers, the energy consumption constraints of each edge server are obtained; based on multiple edge servers, the effective throughput of the distributed server cluster is obtained; using spatial decision and batch size as decision variables, and based on the partitioning constraints, energy consumption constraints, and effective throughput, a global configuration problem is established. The third module is used to obtain the effective capacity of the edge servers based on renewable energy conditions and the distributed server cluster; with the goal of the number of layers undertaken by each edge server equaling the effective capacity, it performs an equivalent transformation based on the effective capacity of the edge servers to transform the global configuration problem into a spatial sub-problem. The fourth module is used to perform a feasibility search on batch size based on the initial results of the pipeline configuration, and to perform discrete projection to solve the spatial subproblem and obtain the feasible results of the pipeline configuration. The fifth module is used to obtain the final result of the pipeline configuration based on the feasible results of the pipeline configuration, so that the distributed edge server cluster can match the renewable energy conditions and complete the task allocation.

[0012] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described method.

[0013] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0014] The aforementioned task allocation method, apparatus, equipment, and medium based on a large language model represent a renewable energy-aware distributed large language model inference method for intermittently powered edge clusters. Unlike approaches that treat edge servers as static resources, this method views the system implementing the method as a controller capable of periodically adjusting the logical model topology based on the cluster's physical energy state. First, at a coarse time scale, the Spatio-Temporal Energy-Elastic Repartition (STEER) algorithm, combined with the current energy state and computing power of nodes, periodically adjusts layer placement and batch size to construct a pipeline configuration that satisfies energy feasibility. Then, at a fine time scale, a semantic decoupling resilient execution mechanism based on an outline is further introduced, transforming the originally holistic text generation process into schedulable point-level branches. This absorbs temporary energy disturbances from slow nodes during runtime and suppresses static energy waste caused by pipeline bubbles. In this way, renewable energy-aware control and distributed autoregressive inference are unified at both coarse and fine time scales. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a task allocation method based on a large language model in one embodiment; Figure 2 This is a schematic diagram of the architecture of a task allocation method based on a large language model in one embodiment; Figure 3 This is a semantic decoupling graph based on an outline in a low-energy toughness mode in one embodiment; Figure 4 This is a graph showing the average latency under different system sizes and workloads in a specific embodiment. Figure 5This is a pipeline bubble rate diagram for a specific embodiment under different system sizes and workloads; Figure 6 This is a diagram showing the percentage of green electricity under different system sizes and workloads in a specific embodiment. Figure 7 This is an energy consumption diagram for a specific embodiment under different system sizes and workloads; Figure 8 This is a diagram showing the ablation experiment results in a specific embodiment; Figure 9 This is a sensitivity analysis diagram of the context length on the average latency in a specific embodiment; Figure 10 This is a sensitivity analysis diagram of inter-server bandwidth to average latency in a specific embodiment; Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0017] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. In the description of this application, "multiple sets" means at least two sets, such as two sets, three sets, etc., unless otherwise explicitly specified.

[0018] In this application, unless otherwise expressly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two elements or the interaction between two elements, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0019] Furthermore, the technical solutions of the various embodiments of this application can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this application.

[0020] This application provides a task allocation method based on a large language model, such as Figure 1 The flowchart shown, in one embodiment, includes: step 101, obtaining tasks to be assigned, the tasks to be assigned including: large language model, renewable energy conditions and distributed server cluster, the distributed server cluster including multiple edge servers.

[0021] Specifically: Obtain tasks to be assigned, including: large language model, renewable energy conditions, and distributed server cluster; Large language models are a type of language model with A multi-layered Transformer large language model; Renewable energy conditions include: static energy consumption, dynamic energy consumption, dispatch cycle determined by the dynamics of renewable energy, and normalized renewable energy; Distributed server cluster Serving a The Transformer large language model with layers, including A heterogeneous edge server with energy harvesting capabilities Obtaining the distributed server cluster includes: obtaining the discrete scheduling period of the distributed server cluster. And normalized computing power, to obtain the computing power of each edge server. (Computing power is the number of floating-point numbers that can be processed per second, measured in FLOPS), memory budget (Memory budget refers to memory size, measured in GB) and energy availability. (Energy availability is the server) In epoch Energy (in joules), and the available bandwidth between adjacent edge servers and the edge network bandwidth.

[0022] In this step, each edge server is a node, and each discrete scheduling period is a period of time.

[0023] Step 102: Divide the large language model into various edge servers, ensuring that each edge server meets the partitioning constraints, to obtain the initial result of the pipeline configuration, which includes spatial decision and batch size; based on renewable energy conditions and multiple edge servers, obtain the energy consumption constraints for each edge server; based on multiple edge servers, obtain the effective throughput of the distributed server cluster; using spatial decision and batch size as decision variables, and based on the partitioning constraints, energy consumption constraints, and effective throughput, establish a global configuration problem.

[0024] Specifically: The large language model is divided into edge servers, and each edge server satisfies the partitioning constraints to obtain the initial result of the pipeline configuration, which includes spatial decision and batch size. The renewable energy conditions include static energy consumption and dynamic energy consumption; the computation latency of each pipeline stage is obtained based on the computing power of each edge server; the communication latency between adjacent pipeline stages is obtained based on the available bandwidth between adjacent edge servers; the pipeline cycle time is obtained based on the computation latency of each pipeline stage and the communication latency between adjacent pipeline stages; the energy consumption of each edge server is obtained based on the static energy consumption and dynamic energy consumption, combined with the pipeline cycle time; the energy consumption constraint for each edge server is that its energy consumption does not exceed the available harvested energy. Based on the edge network bandwidth between adjacent edge servers, the bottleneck migration latency is obtained; the scheduling cycle dynamically determined by renewable energy is obtained, and combined with the bottleneck migration latency, the effective throughput of the distributed server cluster is obtained. Using spatial decision and batch size as decision variables, a global configuration problem is established based on partitioning constraints, energy consumption constraints, and effective throughput.

[0025] More specifically: To perform coarse-grained alignment at epoch boundaries, pipeline parallelism (PP) is employed to divide the large language model into edge servers, ensuring that the number of layers on each edge server meets the partitioning constraints, and that each edge server corresponds to a pipeline stage, thus obtaining the initial result of the pipeline configuration, which includes spatial decisions and batch size. The partitioning constraints include: model integrity constraints, minimum layer constraints, and memory constraints. The model integrity constraint (requiring all layers to be assigned) is:

[0026] In the formula, For spatial decision-making, let the first... Discrete scheduling cycles (epochs) ) assigned to the One edge server The number of Transformer layers, , For the number of edge servers, This represents the total number of layers in the large language model. Minimum layer constraints (to maintain pipeline connectivity and avoid broken phases after energy-aware reconfiguration, each edge server must retain at least one layer constraint) The layer is:

[0027] In the formula, To maintain pipeline connectivity, the edge server should handle the minimum number of Transformer layers. The memory constraint (the coupling between space decision and batch size through the memory constraint) is as follows:

[0028] In the formula, The memory allocated to each layer of parameters. For each layer and each batch size, the KV cache memory is... For batch size, For the first One edge server Memory budget; After obtaining the initial pipeline configuration results, the computation latency for each pipeline stage is calculated based on the computing power of each edge server:

[0029] In the formula, For the first The first stage of the process is in Discrete scheduling cycles (epochs) The computational latency (consisting of computation and communication) The number of floating-point operations required for a single token to pass through a single layer. For the first One edge server Computational power; Based on the available bandwidth between adjacent edge servers, the communication latency between adjacent pipeline stages is obtained:

[0030]

[0031] In the formula, For the communication delay between adjacent flow stages, For the hidden layer dimension, Data precision for each element, The available bandwidth between adjacent edge servers; Parallel pipelines are essentially similar to a synchronous production line, with their effective throughput controlled by the slowest stage. The pipeline cycle time is obtained based on the computational latency of each stage and the communication latency of adjacent stages.

[0032] In the formula, This refers to the production line cycle time. Renewable energy conditions include static energy consumption and dynamic energy consumption. To characterize the energy consumption of a pipeline cycle, the server's energy consumption is modeled as the sum of static energy consumption (the consumption when the node remains powered on, including passive waiting caused by bubbles) and dynamic energy consumption (the consumption during active token processing). Based on static and dynamic energy consumption, combined with the pipeline cycle time, the energy consumption of each edge server is obtained:

[0033] In the formula, For the first Energy consumption of an edge server For the first Static power consumption of an edge server For the first Dynamic energy consumption of an edge server (i.e., the dynamic energy required for a single token to pass through a single layer of Transformer). To utilize renewable energy sources for large-scale model inference as much as possible, we search for configurations that satisfy the following constraint: the energy consumption per epoch in each epoch does not exceed the available harvesting energy for that node. Specifically, the energy consumption of each edge server must not exceed the available harvesting energy as the energy consumption constraint for each edge server.

[0034] In the formula, For the first Energy consumption of an edge server For the first One edge server Energy availability (i.e., the first) One edge server (Available harvested energy); On a coarse timescale, any imbalance at any stage will amplify cycle time and form structural bubbles, thereby simultaneously weakening responsiveness and renewable energy utilization. Repartitioning introduces model weight migration overhead. To avoid migrating the online KV cache, new request reception is paused during repartitioning, waiting for in-transit requests to complete, and the active KV cache is cleared, leaving only static layer weights for migration. The bottleneck migration latency is obtained based on the edge network bandwidth between adjacent edge servers.

[0035]

[0036] In the formula, For bottleneck migration latency, For server The number of layers that need to be migrated into or out in the current epoch relative to the previous epoch. For the edge network bandwidth between servers, For spatial decision-making, For the first The discrete scheduling cycle is assigned to the first One edge server The number of Transformer layers; At the epoch level, the goal is to maximize effective pipeline throughput while meeting green feasibility constraints, and to explicitly account for reconfiguration overhead; by obtaining the scheduling cycle dynamically determined by renewable energy sources and combining it with bottleneck migration latency, the effective throughput of the distributed server cluster is obtained.

[0037] In the formula, For effective throughput, The scheduling cycle is dynamically determined for renewable energy. The proportion of time that is actually available for service within a scheduling cycle. The throughput capacity of the production line itself; Using spatial decision and batch size as decision variables, and based on partitioning constraints, energy consumption constraints, and effective throughput, a global configuration problem is established:

[0038]

[0039]

[0040]

[0041]

[0042] In the formula, For spatial decision-making, let the first... The discrete scheduling cycle is assigned to the first One edge server The number of Transformer layers; For batch size, For efficient throughput; , For the number of edge servers, This represents the total number of layers in the large language model. To maintain pipeline connectivity, the edge server should handle the minimum number of Transformer layers. The memory allocated to each layer of parameters. For each layer and each batch size, the KV cache memory is... For batch size, For the first One edge server Memory budget; Energy consumption for each edge server The harvesting energy available for each edge server.

[0043] In this step, the energy consumption constraint defines the green feasible operating region corresponding to the coarse-grained alignment. If the operation deviates from the ideal balance temporarily due to residual disturbances, it will be handled by the fine-grained resilient execution mechanism.

[0044] Furthermore, the global allocation problem is a mixed-integer nonlinear programming problem (MINLP); its difficulties mainly stem from: the objective function being in non-convex fractional form and the presence of memory constraints. and The existence of bilinear coupling and the introduction of bottleneck coupling by the pipeline cycle time, which is determined by the slowest stage, make it impossible to solve this problem precisely online at each epoch in the dynamic operating environment of renewable energy.

[0045] Step 103: Based on the renewable energy conditions and the distributed server cluster, obtain the effective capacity of the edge servers; with the goal of each edge server assuming the number of layers it undertakes equal to its effective capacity, perform an equivalent transformation based on the effective capacity of the edge servers to transform the global configuration problem into a spatial subproblem.

[0046] Specifically: Based on renewable energy conditions and distributed server clusters, normalized renewable energy and normalized computing power are obtained; based on normalized renewable energy and normalized computing power, the effective capacity of edge servers is obtained. With the goal of ensuring that the number of layers undertaken by each edge server equals its effective capacity, an equivalent transformation is performed based on the effective capacity of the edge servers to transform the global configuration problem into a spatial subproblem.

[0047] More specifically: To achieve real-time renewable energy sensing and control, an agent state abstraction is introduced, and the server's true service potential is not solely determined by computing power. It is inferred that even if a node has strong computing power, allocating an excessive load to it may still violate energy feasibility constraints and create structural bottlenecks if energy is currently scarce. When energy conditions are even more strained, energy availability is emphasized to ensure the feasibility of configuration supporting renewable energy. Based on renewable energy conditions and the distributed server cluster, normalized renewable energy and normalized computing power are obtained. Based on normalized renewable energy and normalized computing power, the effective capacity of the edge server is obtained.

[0048] In the formula, For the first One edge server In epoch Effective capabilities For normalized renewable energy, To normalize computing power, To balance the greenness of operators, To balance the throughput of operators, As a usable energy source, Available computing power; by As a proxy for coupled energy-computing power constraints, with the goal of each edge server assuming the number of layers it undertakes equal to its effective capacity, an equivalent transformation is performed based on the effective capacity of the edge servers to transform the global configuration problem into a spatial subproblem (for fixed...). The original problem can be transformed into a space integer programming problem: in order to reduce bottlenecks and suppress structural bubbles, it is desirable that the number of layers handled by each edge server is equal to its normalized effective capacity.

[0049] In the formula, For the first One edge server In epoch Effective capabilities For edge servers Effective capabilities.

[0050] In this step, the spatial subproblem can be approximated as minimizing the squared error, which means: to make the "actual layer allocation ratio" as close as possible to the "ideal capacity ratio".

[0051] In addition, even if it has already been used Abstracting the joint state of energy and computing power, the space subproblem is fixed. Even under these circumstances, it remains strongly NP-hard; the reason is that: Layer in Dividing the nodes into all possible partitions inherently results in a combinatorial explosion problem; under unconstrained conditions, its configuration space size is... Therefore, in a renewable energy fluctuating environment, attempting an exact solution for each epoch would incur unacceptable online overhead. Based on this, step 104 is further proposed: it approximates the joint optimization problem through two coupled steps, namely, first... Perform a time-dimensional search for feasibility, and then... Perform a deterministic discrete projection to... The space subproblem can be solved approximately in time.

[0052] Step 104: Based on the initial results of the pipeline configuration, perform a feasibility search on the batch size and execute discrete projection to solve the spatial subproblem and obtain the feasible results of the pipeline configuration.

[0053] Specifically: Based on the effective capabilities of the edge servers, the layer-assigned continuous targets are obtained; Based on the initial pipeline configuration, for any batch size, perform floor allocation on the layer-assigned continuous targets to obtain the floor allocation result; calculate the residual value based on the floor allocation result and the layer-assigned continuous targets. When the rounding allocation result satisfies the memory constraints of each edge server partitioning constraint, compensation is performed based on the residual value. Once compensation is complete, the batch size is determined to be feasible. Otherwise, determining the batch size is not feasible; Based on whether the batch size is feasible or not, a binary search is performed to obtain feasible results for the pipeline configuration.

[0054] More specifically: The selection of batch size is expressed as an interval A bounded search problem, in which This is the maximum allowed batch size. As the spatial subproblem shows, the optimal continuous layer depends only on the effective capacity score. ,and Since this is irrelevant, the layer allocation successive objective can be computed all at once at the start of each scheduling epoch, reducing the core problem to: given the candidate batch size... In this case, is it possible to achieve the ideal continuous target? Discretization into a valid integer level partition And without violating memory and energy constraints; based on the effective capacity of the edge server, the layer allocation continuous target is obtained:

[0055] In the formula, Assign continuous targets to the layer; Based on the initial results of the pipeline configuration, for any batch size For continuous targets in the layer, perform floor allocation down to obtain the floor allocation result:

[0056] In the formula, To round down; Calculate the residual value based on the rounded allocation result and the hierarchical allocation of continuous objectives:

[0057] In the formula, The residual value; When determining whether the rounded allocation result satisfies the memory constraints of each edge server partitioning constraint, compensation is performed based on the residual value (compensation based on the residual value includes: calculating the number of unallocated layers). According to residual value Sort all edge servers from largest to smallest, and compensate each edge server with the unallocated layers. Before each compensation, it is necessary to verify again whether the video memory limit is still met after adding a layer. Once the compensation is completed (i.e., the unallocated layers have been allocated), determine that the batch size is feasible. When the rounding allocation result satisfies the memory constraint of each edge server partitioning constraint, that is, when a certain edge server cannot even accommodate the number of layers after rounding down under the current batch size, it is determined that the batch size is not feasible. If the rounding allocation result does not meet the memory constraints of each edge server partitioning constraint, the batch size is deemed infeasible. Depending on whether the batch size is feasible or not (under the premise of strictly satisfying safety constraints, perform a discrete projection of continuous targets with the least possible distortion), for Perform a binary search (if the batch size is feasible, record the current configuration and continue searching towards a larger batch size to improve effective throughput; if the batch size is not feasible, switch to searching towards a smaller batch size; the binary search process is consistent with the monotonicity of the memory budget: as... Increase The terms will expand, which will only further shrink the feasible region. Therefore, it is possible to efficiently locate the maximum feasible batch size without relying on heuristic thresholds, and obtain feasible results for pipeline configuration.

[0058] In this step, coarse-grained spatiotemporal alignment (STEER) is performed. At each scheduling epoch, STEER approximates a spatial subproblem to obtain an energy-feasible configuration. The basic idea is to first calculate a capability-guided layer allocation continuous target based on the current effective capability score of the node; then, under the constraints of system architecture, memory, and energy, search for the largest feasible batch size.

[0059] Coarse-grained spatiotemporal alignment has the following beneficial effects on renewable energy-driven edge inference: First, strict feasibility is guaranteed. Any output configuration maintains architectural layer conservation, i.e. It also satisfies memory constraints; at the same time, through edge server capability and feasibility checks, the resulting configuration will also be consistent with the epoch-level energy feasible region.

[0060] Second, spatial projection can be viewed as an efficient approximation of the spatial subproblem, integer programming. This is because continuous objectives... The calculation is performed only once outside the search loop, so the subsequent residual compensation step essentially constitutes a greedy, minimum-distortion discrete projection: it prioritizes assigning the remaining layers to the nodes with the largest residuals and still have available memory, in order to reduce the squared error between the actual partitioning and the ideal capacity distribution; this globally ordered compensation method can suppress unnecessary layer rearrangement; when the renewable energy conditions of adjacent epochs change only to a moderate degree, They are often smaller, which helps reduce bottleneck migration latency. .

[0061] Third, it has good efficiency and scalability. It uses binary search, so in the time dimension it requires at most [time value missing]. The iteration; the most significant computational cost in each round is the calculation of residual pairs. Sort the nodes, so the complexity of a single round is O(n). In summary, the overall complexity is... This lower operating overhead is crucial for coarse-grained control in the face of macroeconomic fluctuations in renewable energy, because the configuration must be recalculated at each scheduling epoch without introducing additional, non-negligible scheduling delays.

[0062] Alternatively, the relevant algorithm can be written as the following code:

[0063]

[0064] Step 105: Based on the feasible results of the pipeline configuration, obtain the final result of the pipeline configuration, so that the distributed edge server cluster matches the renewable energy conditions and completes the task allocation.

[0065] In this step, the feasible result of the pipeline configuration is taken as the final result of the pipeline configuration.

[0066] In this embodiment, as Figure 2 and Figure 3 As shown, after obtaining feasible results for the pipeline configuration, the process also includes performing fine-grained, robust execution: Based on renewable energy conditions, when the global energy state meets a preset threshold, each batch size is considered a user query. The user query is transformed into an outline, and each key point of the outline is expanded to obtain an expanded sub-request. Based on the tokens generated by each expansion and the maximum number of tokens generated by the expanded sub-request, tokens of the key points are generated to obtain the answer to the user query. The answer to the user query is used as the optimal result of the pipeline configuration. Based on the optimal result of the pipeline configuration, the final result of the pipeline configuration is obtained.

[0067] Specifically: Calculate the global energy state based on renewable energy conditions:

[0068] In the formula, For the global energy state, Normalized renewable energy; When the global energy state meets a preset threshold, the fine-grained resilience execution mode is triggered (global energy state). It is used not only to determine whether to enable fine-grained resilient execution mode, but also to adjust the token quota for each branch after the mode is activated; that is, only when... Furthermore, the semantic decoupling mechanism only takes effect when the current query is determined to be decomposable by a lightweight query-tag; otherwise, the request bypasses the Outline Controller and is executed sequentially on the STEER-aligned pipeline. Each batch size is considered a user query. The user query is transformed into an outline, each outline includes multiple key points, and each key point of the outline is expanded to obtain mutually independent extended sub-requests (key point branches). If the first One query Generated If there are 10 outline points, then the original batch will be converted into a branch pool containing the following number of decoding branches. This transformation converts the original single, continuous stream of response generation into a set of interleaved work units to achieve resilience at a fine-grained level. Calculate the average capability level based on the effective capabilities of the edge servers:

[0069] In the formula, Average ability level For the first One edge server In epoch Effective capability; To achieve low-overhead online control and fairer branch-interleaving execution, the maximum number of tokens generated for extended sub-requests is obtained based on the global energy state and average capability level:

[0070] In the formula, The maximum number of lexical units generated (within the resilient mode). This can be understood as an energy-aware software valve: it shrinks when energy conditions or capabilities are more constrained to increase branch-interleaving scheduling; and it enlarges when the operating state is relatively stable, thereby amortizing context switching overhead through longer consecutive executions. Smaller It is beneficial for enhancing multiplexing capabilities and reducing long-tailed bubbles during rapid energy fluctuations; larger This is more suitable for resilient execution phases where the state is relatively stable, because it allows for longer continuous deployments and reduces the additional costs associated with frequent preemption. To ensure a minimum feasible token allocation and avoid excessive overhead caused by frequent switching in low-energy states, For the global energy state, Average ability level To control the impact of quotas on the global energy state The quantity of response intensity, To control the impact of quotas on average capacity levels The quantity of response intensity; When the generated tokens reach the maximum number of tokens for the extended sub-request, if the end marker EOS does not appear, the key point is further extended and the maximum number of tokens for the extended sub-request is checked again until the end marker EOS appears. The current extension result (current token) is then obtained and used as the token of the key point. (That is, in actual execution, after each service round, if a branch has not yet output EOS, it will be reinserted to the tail of the ready queue; if the branch has reached the current token quota limit, it will be preempted and re-queued to wait for subsequent service.) If the number of tokens generated in each expansion does not reach the maximum number of tokens generated in the expansion sub-request, the current expansion result (current token) is obtained and used as the token of the key point. Based on the lexical units of each key point, the results are integrated to obtain the answer to the user's query. (To maintain the quality of the generated data, free splitting is avoided. Instead, the outline is used as a semantic scaffold, ensuring that each branch revolves around a coherent sub-topic. The entire outline is then reassembled upon completion. This way, if a branch is delayed due to upstream energy fluctuations, the downstream stages can continue processing other ready branches without entering a period of idle waiting, thus suppressing the propagation of energy bubbles caused by runtime disturbances.) All branches share the common prefix KV cache corresponding to the original query and outline, while reserving independent KV segments for their respective subsequent expansions. To manage this branch pool, an Outline Controller is designed to uniformly manage the queue, maintain the KV cache, perform completion checks, and control token quotas. The current prototype uses a global cyclic ready queue. (queue), and introduce a length-aware priority strategy when scheduling across branches; shorter key branches are scheduled first and completed earlier to release memory faster and reduce long-tail bubbles; if multiple branches are of the same length, their parallel relationship is broken according to their order in the original outline; when all point-level branches corresponding to a query are completed, they are reorganized into the final answer in logical order). The answer to the user's query is used as the optimal result for pipeline configuration; Based on the optimal pipeline configuration, the final pipeline configuration is obtained.

[0071] It should be noted that while coarse-grained spatiotemporal alignment can eliminate major mismatches caused by minute- or hourly renewable energy dynamics by realigning the pipeline, it cannot respond to minor energy perturbations within an epoch. In monolithic autoregressive execution mode, even a brief slowdown in the upstream stage can propagate downstream and expand into more bubbles. On edge servers with energy harvesting capabilities, these bubbles not only increase latency but also lead to additional static energy waste, as the waiting stages still need to remain powered on and maintain a large-scale KV cache, yet make almost no effective progress. Therefore, fine-grained resilient execution is proposed. Its core idea is semantic decoupling based on an outline. At the same time, to avoid unnecessary decomposition overhead, this mode is not always enabled but is only activated when the cluster energy state is below a threshold; otherwise, requests still run directly on the coarse-grained spatiotemporally aligned pipeline. This resilient execution mode complements STEER, which deals with the residual renewable energy fluctuations after coarse-grained alignment, while STEER is responsible for macro-level spatiotemporal configuration optimization. By transforming rigid token flow execution into branch-level interleaved execution under an energy-adaptive token budget, it can absorb runtime disturbances without rebuilding the pipeline, thereby reducing downstream bubbles and static energy waste.

[0072] In addition, edge computing platforms based on a unified memory architecture perform context switching to avoid the bandwidth bottleneck caused by data transmission via PCIe in traditional data centers. This allows context switching to be completed in a near-zero-copy manner, and keeps the additional latency overhead introduced by it to a low level.

[0073] The aforementioned task allocation method based on a large language model is a distributed large language model inference method for renewable energy perception in intermittent power supply edge clusters. Unlike approaches that treat edge servers as static resources, this method views the system implementing this method as a controller capable of periodically adjusting the logical model topology based on the physical energy state of the cluster. First, at a coarse time scale, the Spatio-Temporal Energy-Elastic Repartition (STEER) algorithm, combined with the current energy state and computing power of nodes, periodically adjusts layer placement and batch size to construct a pipeline configuration that satisfies energy feasibility. Then, at a fine time scale, a semantic decoupling resilient execution mechanism based on an outline is further introduced, transforming the originally holistic text generation process into schedulable point-level branches. This absorbs temporary energy disturbances from slow nodes during runtime and suppresses static energy waste caused by pipeline bubbles. In this way, renewable energy perception control and distributed autoregressive inference are unified at both coarse and fine time scales.

[0074] At the coarse-grained control layer (coarse-grained spatiotemporal alignment), a compact runtime state view is constructed to describe the service potential of each edge node. Specifically, the node's current energy availability and computing power are combined to define an Effective Capability Score, which characterizes the node's instantaneous service capacity under the combined constraints of current energy and hardware. This is because relying solely on renewable energy status or nominal computing power is insufficient to accurately reflect a node's ability to sustainably undertake inference tasks. Compared to rule-based thresholding or multi-dimensional state comparison methods that treat energy and computing power separately, unifying them into a directly sortable score is more conducive to quickly completing cross-node comparisons and repartitioning within each scheduling cycle. Based on this score, STEER can perform layer repartitioning and batch size adjustment on each epoch to match load distribution with current renewable energy conditions, allowing more capable nodes to undertake more work, thereby keeping the entire pipeline energy-feasible.

[0075] At the fine-grained execution layer (fine-grained resilient execution), semantic decoupling based on outlines is further introduced on top of STEER. It no longer treats a single response as a strictly linear sequence of token generation, but instead first generates a compact outline and then expands each outline point into a semantically relatively independent branch. This abstraction is necessary because energy perturbations typically occur within a single epoch, changing too rapidly to be addressed through repeated pipeline repartitioning. While runtime heuristics that maintain a monolithic generation structure, such as local slowdowns and passive waiting, can react locally, they do not break the sequential dependencies of autoregressive decoding, thus failing to form truly independently schedulable work units. As a result, if a stage slows down even briefly, subsequent stages are still forced to wait, creating pipeline bubbles. By transforming a complete response into several schedulable point-level branches, interleaved execution can be achieved within the pipeline, providing execution-term resilience under runtime instability caused by renewable energy. To maintain the quality of responses, the system does not arbitrarily segment the text. Instead, it uses an outline as a "semantic scaffold," ensuring that each branch revolves around a relatively clear sub-topic during its development, before finally reorganizing. In this way, if a branch is blocked due to short-term energy fluctuations, downstream nodes can still continue processing other ready branches without entering a purely idle waiting state. This reduces the cascading effect of micro-perturbations and minimizes the static energy waste caused by pipeline bubbles.

[0076] Specifically, this application has the following beneficial effects: 1. A distributed large language model inference method for renewable energy perception for edge clusters with intermittent renewable energy power supply is proposed (the system implementing this method is called Glide), which coordinates coarse-grained spatiotemporal alignment and fine-grained resilient execution under a unified control framework.

[0077] 2. A coarse-grained spatiotemporal alignment was designed. By jointly adjusting the layer division and batch size, the pipeline configuration can adapt to the renewable energy conditions that change over time and meet the energy feasibility requirements.

[0078] 3. Based on the coarse-grained configuration provided by coarse-grained spatiotemporal alignment, we further propose fine-grained resilient execution, which is a fine-grained resilient execution mode based on outline semantic decoupling. It transforms the overall generation into schedulable point-level branches to absorb energy disturbances and reduce static energy waste caused by pipeline bubbles.

[0079] 4. A method based on four-node GPUs (Glide) was implemented to simulate heterogeneous distributed edge clusters and evaluated in conjunction with real-world renewable energy trajectories. Experimental results show that, compared with existing strong baselines, Glide can reduce the average end-to-end latency by 60.5% while maintaining 85%–90% green electricity coverage.

[0080] It should also be noted that this application adopts a hybrid energy supply model: the collected renewable energy data defines the main dispatch budget, while the power grid serves only as a fallback option with high penalties to ensure service continuity. This preserves the coupling relationship between renewable energy fluctuations and operational control, while keeping the optimization objective focused on "services supported by renewable energy as much as possible".

[0081] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0082] In one specific embodiment, experimental evaluation is performed.

[0083] The method of this application is implemented based on real models and datasets, and experimental evaluation is carried out around four core questions: (1) whether it can improve the latency-sustainability trade-off under the condition of renewable energy fluctuations; (2) whether STEER and outline-based semantic decoupling bring complementary benefits; (3) whether outline-based semantic decoupling will affect the output semantic quality; (4) whether it is robust to longer contexts, lower communication bandwidth and reconfiguration overhead.

[0084] 1 Experimental Setup 1.1 Trajectory-Driven Simulation Testbed To evaluate the method of this application under controllable and reproducible dynamic conditions of renewable energy, a trajectory-driven simulation testbed of a four-node heterogeneous energy harvesting edge server cluster was constructed. Each logical edge node is mapped to an RTX A6000 GPU and driven by renewable energy trajectories from NSRDB (https: / / nsrdb.nrel.gov / ) from different geographical locations on the same date (July 1, 2024). These four locations correspond to coordinates (35.74, ...). 86.61), (40.22, 112.13), (38.14, 111.71) and (38.98, (110.11), thus providing reproducible heterogeneous energy availability for each node in space and time. To approximate the operating envelope of the edge server, constraints were imposed on each logical node at both the hardware and software levels: power consumption was limited to 60W using NVIDIA-SMI, clock frequency was constrained, and additional latency was injected based on the 3.7x memory bandwidth difference between the RTX A6000 (768GB / s) and the NVIDIA Jetson AGX Orin (approximately 204GB / s). Through these controls, the experimental platform approximates the computing power and memory behavior at the edge server level while maintaining repeatability.

[0085] 1.2 Model and Dataset To reflect more realistic workloads in edge scenarios, the experiments used an INT4-quantized Llama2-7B as the deployed large language model. This configuration reduces the model's memory usage on a single node to a level more consistent with the memory budget of high-end edge servers (such as Jetson AGX Orin), thus ensuring the feasibility of edge deployment. Regarding datasets, the experiments used three relatively new instruction datasets to construct diverse workloads with different cue lengths and task attributes: LiMA, Vicuna-80, and WizardLM.

[0086] 1.3 Comparison with baseline and evaluation indicators Since there has been no previous work simultaneously studying distributed LLM inference with renewable energy awareness, this application selects two comparative baselines from two types of methods: one is performance-oriented inference systems, including Jupiter and DeTransformer; the other is renewable energy-aware scheduling methods, including Pure Green (PG) and Local Greedy (LG). Jupiter is an edge-cooperative LLM inference system employing speculative decoding and outline-based decoding mechanisms; DeTransformer is a cooperative Transformer inference system whose main goal is to reduce communication costs; Pure Green (PG) represents a zero-grid strategy that chooses to wait or throttle when renewable energy is insufficient; Local Greedy (LG) represents a static pipeline strategy that directly switches back to grid power when local renewable energy is insufficient.

[0087] The evaluation metrics include four performance and sustainability indicators: average end-to-end latency, pipeline bubble rate, green ratio, and grid consumption per request. To ensure fairness, Glide's latency statistics include the complete overhead of outline generation, including prefix hints and skeleton inference. Furthermore, this application evaluates output quality using the following metrics: LLM-as-a-Judge (GPT-4 tie / win rate), ROUGE-L, and intra-point cosine similarity based on Sentence-BERT embedding.

[0088] 2. Experimental Results and Analysis 2.1 Overall Performance Figure 4 Figure 5 and Figure 6 Figure 7The proposed method was compared with four baselines under different system sizes and workload intensities. Across all configurations supported by the four-node simulation testbed, the proposed method consistently achieved the best latency-sustainability trade-off. Compared to performance-oriented baselines, the proposed method achieved the lowest average latency and a consistently low bubble rate under conditions of renewable energy fluctuations. This indicates that, on the one hand, STEER can suppress energy mismatch at the macro level, preventing it from evolving into a structural bottleneck; on the other hand, the resilient execution mechanism prevents the remaining runtime imbalances from further amplifying into pipeline bubbles. At scale=4, the proposed method reduced the average latency from 12.0s for Jupiter and 6.5s for DeTransformer to 4.7s, representing reductions of 60.5% and 27.3%, respectively. Its advantages are further amplified with heavier workloads: at batch size=16, the completion time of the proposed method is approximately 1.17s, while PG and LG are approximately 7.0s and 4.7s, respectively. These results demonstrate that Glide maintains good pipeline balance as the scale increases and suppresses bubble amplification under renewable energy fluctuations.

[0089] From a sustainability perspective, Figure 6 Figure 7 The Glide approach lies between two extreme strategies: PG and LG. PG maximizes renewable energy utilization but suffers from significant latency inflation when green electricity is insufficient; LG maintains responsiveness by relying on the grid as a safety net, but at the cost of higher non-renewable energy consumption. The method presented in this application, through spatiotemporal realignment, significantly reduces grid dependence while maintaining high performance. This method maintains approximately 85%–90% green electricity coverage under various workloads and significantly reduces grid energy consumption per request. For example, with a batch size of 16, this method reduces grid energy consumption per request from approximately 8000 J / Req for DeTransformer to approximately 76 J / Req, a reduction of approximately 105 times. Furthermore, some baselines fail to support corresponding configurations under larger scales or heavier loads, further illustrating the limited practical applicability of existing edge inference designs; in contrast, this method, relying on a closed-loop coarse-to-fine granular adaptive mechanism, remains effective even in these scenarios.

[0090] 2.2 Ablation Experiment To analyze the contributions of each key component, the method of the full application was compared with several abridged versions, including: a version that removed coarse-grained alignment; a version that removed fine-grained semantic decoupling; and a variant of "static partitioning + green-aware throttling".

[0091] Figure 8The results show that the complete system achieves the best overall trade-off, with an average latency of 7.10s, a green electricity share of 95.21%, and a bubble rate of 0.43%. Removing the STEER parameter results in the most significant performance degradation: the average latency increases to 23.99s, and the green electricity share decreases to 80.8%. This indicates that coarse-grained alignment is the main mechanism for handling macroeconomic fluctuations in renewable energy.

[0092] Removing semantic decoupling also significantly worsened performance: average latency increased to 10.55s, and the bubble rate rose to 3.45%. This indicates that even with stage-level coarse alignment, fine-grained execution flexibility remains necessary. On the other hand, the scheme relying solely on static partitioning and green throttling performed the worst, with a latency of 32.53s and a green electricity ratio of only 77.81%, demonstrating that throttling alone cannot prevent pipeline bubble amplification. Overall, the experiments show that coarse-grained alignment and fine-grained execution are not substitutes but complements, together achieving the optimal latency-sustainability balance.

[0093] 2.3 Semantic Quality Preservation A natural concern regarding the low-energy resilience of the method in this application is whether enabling outline-based semantic decoupling, which breaks down the answer into multiple independent points for expansion, will lead to context loss, logical breaks, or text duplication. Therefore, the decoupled generation method of this application is compared with a strict sequential generation baseline that uses a complete historical KV cache.

[0094] Table 1: Comparison of semantic quality preservation between this application and the sequential generation baseline

[0095] The experiment sampled complex cues from Vicuna-80 and WizardLM, and measured semantic fidelity using GPT-4 logical consistency assessment, ROUGE-L information retention, and Sentence-BERT intra-point cosine similarity. Table 1 shows the results: on Vicuna-80, Glide's GPT-4 Tie / Win rate was 94.5%, ROUGE-L was 0.88, and intra-point similarity was 0.21; on WizardLM, these metrics were 92.0%, 0.85, and 0.24, respectively. Therefore, Glide incurs only a small, acceptable loss of semantic quality compared to standard sequential generation. The high Tie / Win rate indicates that its human alignment quality is largely maintained; the high ROUGE-L indicates that key information is well preserved; and the low intra-point similarity indicates that different branches do not collapse into repetitive text. Overall, shared outlines can serve as a robust semantic anchor, allowing the system to maintain good output quality while relaxing strict sequential dependencies.

[0096] 2.4 Sensitivity Analysis Figure 9 and Figure 10 The sensitivity of Glide to context length and inter-node bandwidth was evaluated. As the context length increased from 32 tokens to 256 tokens, the latency of all methods increased due to the greater pressure on the KV cache and the longer per-token processing. However, Glide showed the least sensitivity, with its latency increasing only from approximately 2.8s to approximately 5.5s, while DeTransformer increased from approximately 3.9s to approximately 19.3s, and Jupiter from approximately 3.4s to approximately 10.5s. In the 256-token scenario, Glide was still approximately 3.5 times faster than DeTransformer and approximately 1.9 times faster than Jupiter. Glide also demonstrated good robustness when communication bandwidth decreased. When the inter-node bandwidth decreased to 6.25GB / s, Glide's average latency remained at approximately 3.6s, while DeTransformer and Jupiter were approximately 9.1s and 7.8s, respectively, representing advantages of approximately 2.5 times and 2.2 times. This demonstrates that Glide can still maintain pipeline balance well even under increased memory pressure and stronger communication bottlenecks.

[0097] The impact of reconfiguration overhead was further examined, a key issue in dynamic distributed inference. According to the model, repartitioning introduces weight migration latency, while Glide avoids the migration of the online key-value cache. Using INT4 quantization, the parameter size of each Transformer layer in LLaMA2-7B is approximately 110MB; at an edge wireless bandwidth of 50MB / s, migrating one layer takes approximately 2.2 seconds. More generally, this transmission overhead increases linearly with the number of layers redistributed. Since Glide uses a coarse-grained setting of T"epoch"=300"s in the experiments, while STEER typically only repartitions across epochs, the overhead remains relatively small under edge network conditions.

[0098] 2.5 Discussion and Limitations This application's method targets an edge server-level scenario where online control is driven by collected renewable energy. The current prototype runs in a reproducible trajectory-driven simulation environment on real GPU hardware, preserving the basic execution semantics of distributed pipelined inference while imposing controlled renewable energy fluctuations on different nodes. This setup is sufficient to verify the coarse-to-fine granular control logic of the method, but it still abstracts away some real-world factors beyond the scope of current work. These include: native energy harvesting and storage dynamics; real-time remapping overhead; wireless network jitter; and power system transients. Deploying this method to a real renewable energy-driven edge platform in the future will facilitate a more comprehensive analysis of end-to-end system overhead, control stability, and quality-efficiency tradeoffs.

[0099] In summary, the method presented in this application achieves efficient and sustainable edge LLM inference under conditions of renewable energy fluctuations, while addressing energy instability at two time scales: first, macroscopic fluctuations from minutes to hours require system-level energy-load realignment; second, microscopic disturbances from seconds to milliseconds require absorption through runtime resilience during the execution phase. Based on this understanding, a coarse-to-fine system design is adopted: at the coarse-grained level, STEER is used for spatiotemporal alignment; at the fine-grained level, outline-based semantic decoupling is used for resilient execution. The former is responsible for periodically and dynamically adjusting the pipeline configuration to match current energy conditions based on the current renewable energy status and node capabilities, while the latter is responsible for improving execution resilience by relaxing the rigid sequential dependencies of traditional monolithic decoding, absorbing residual runtime disturbances without rebuilding the pipeline, thereby jointly supporting renewable energy-driven distributed autoregressive inference. On a trajectory-driven simulation platform based on a four-node GPU, evaluation results using real solar and wind trajectories show that, compared to a strong baseline, Glide reduces average end-to-end latency by 60.5% while maintaining an 85%–90% green electricity share. Further ablation experiments and semantic quality assessments demonstrate that the coarse-grained alignment and fine-grained execution mechanisms are complementary, and resilient execution significantly improves performance while largely maintaining semantic consistency in responses without significantly compromising response quality. Therefore, the design principles of the method presented in this application are promising and can be used to support sustainable distributed LLM services under intermittent renewable energy conditions.

[0100] This application also provides a task allocation device based on a large language model, which in one embodiment includes: a first module, a second module, a third module, a fourth module, and a fifth module, wherein: The first module is used to obtain tasks to be assigned, including: a large language model, renewable energy conditions, and a distributed server cluster, wherein the distributed server cluster includes multiple edge servers. The second module is used to partition the large language model onto various edge servers, ensuring that each edge server meets the partitioning constraints, and to obtain the initial results of the pipeline configuration, which includes spatial decision and batch size; based on renewable energy conditions and multiple edge servers, the energy consumption constraints of each edge server are obtained; based on multiple edge servers, the effective throughput of the distributed server cluster is obtained; using spatial decision and batch size as decision variables, and based on the partitioning constraints, energy consumption constraints, and effective throughput, a global configuration problem is established. The third module is used to obtain the effective capacity of the edge servers based on renewable energy conditions and the distributed server cluster; with the goal of the number of layers undertaken by each edge server equaling the effective capacity, it performs an equivalent transformation based on the effective capacity of the edge servers to transform the global configuration problem into a spatial sub-problem. The fourth module is used to perform a feasibility search on batch size based on the initial results of the pipeline configuration, and to perform discrete projection to solve the spatial subproblem and obtain the feasible results of the pipeline configuration. The fifth module is used to obtain the final result of the pipeline configuration based on the feasible results of the pipeline configuration, so that the distributed edge server cluster can match the renewable energy conditions and complete the task allocation.

[0101] For specific limitations regarding a task allocation device based on a large language model, please refer to the limitations of a task allocation method based on a large language model mentioned above, which will not be repeated here. Each module in the above device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0102] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a task allocation method based on a large language model. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0103] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0104] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method described above.

[0105] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0106] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0107] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0108] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended application documents.

Claims

1. A task allocation method based on a large language model, characterized in that, include: Obtain tasks to be assigned, including: large language model, renewable energy conditions, and distributed server cluster, wherein the distributed server cluster includes multiple edge servers; The large language model is partitioned across edge servers, ensuring each edge server meets partitioning constraints, thus obtaining the initial pipeline configuration result, which includes spatial decision and batch size. Based on renewable energy conditions and multiple edge servers, the energy consumption constraints for each edge server are obtained. Based on multiple edge servers, the effective throughput of the distributed server cluster is obtained. Using spatial decision and batch size as decision variables, and based on partitioning constraints, energy consumption constraints, and effective throughput, a global configuration problem is established. Based on renewable energy conditions and distributed server clusters, the effective capacity of edge servers is obtained; with the goal of each edge server assuming the number of layers it undertakes equal to its effective capacity, an equivalent transformation is performed based on the effective capacity of the edge servers to transform the global configuration problem into a spatial subproblem. Based on the initial results of the pipeline configuration, a feasibility search is performed on the batch size, and discrete projection is executed to solve the spatial subproblem, thereby obtaining the feasible results of the pipeline configuration. Based on the feasible results of the pipeline configuration, the final result of the pipeline configuration is obtained, enabling the distributed edge server cluster to match renewable energy conditions and complete task allocation.

2. The task allocation method based on a large language model according to claim 1, characterized in that, After obtaining a feasible pipeline configuration, the following is also included: Based on renewable energy conditions, when the global energy status meets the preset threshold, each batch size is considered as one user query. The user query is converted into an outline, and each key point of the outline is expanded to obtain an expanded sub-request. Based on the vocabulary generated in each expansion and the maximum number of vocabulary generated in the expansion sub-request, the vocabulary of the key points is generated to obtain the answer to the user's query; The answer to the user's query is used as the optimal result for pipeline configuration; Based on the optimal pipeline configuration, the final pipeline configuration is obtained.

3. The task allocation method based on a large language model according to claim 2, characterized in that, Based on the initial results of the pipeline configuration, a feasibility search is performed on the batch size, and discrete projection is executed to solve the spatial subproblems, yielding feasible results for the pipeline configuration, including: Based on the effective capabilities of the edge servers, the layer-assigned continuous targets are obtained; Based on the initial pipeline configuration, for any batch size, perform floor allocation on the layer-assigned continuous targets to obtain the floor allocation result; calculate the residual value based on the floor allocation result and the layer-assigned continuous targets. When the rounding allocation result satisfies the memory constraints of each edge server partitioning constraint, compensation is performed based on the residual value. Once compensation is complete, the batch size is determined to be feasible. Otherwise, determining the batch size is not feasible; Based on whether the batch size is feasible or not, a binary search is performed to obtain feasible results for the pipeline configuration.

4. The task allocation method based on a large language model according to any one of claims 1 to 3, characterized in that, The large language model is partitioned across edge servers, ensuring each edge server meets partitioning constraints, thus obtaining the initial pipeline configuration. This pipeline configuration includes spatial decisions and batch size considerations, including: The large language model is divided into edge servers, and each edge server satisfies the partitioning constraints to obtain the initial result of the pipeline configuration, which includes spatial decision and batch size. The partitioning constraints include: model integrity constraints, minimum layer constraints, and memory constraints. The model integrity constraints are: In the formula, For spatial decision, the number of discrete scheduling periods allocated to the edge server is represented as The number of Transformer layers of the edge server , , The number of edge servers is The total number of layers of the large language model; The minimum layer constraint is: In the formula, To keep the minimal number of Transformer layers that the edge server takes on the pipeline connection. The video memory constraint is: In the formula, Memory occupied by each layer parameter, KV cache memory of each batch size for each layer, Batch size, Memory budget of the Edge server .

5. The task allocation method based on a large language model according to any one of claims 1 to 3, characterized in that, Based on renewable energy conditions and multiple edge servers, the energy consumption constraints for each edge server are obtained, including: Renewable energy conditions include static energy consumption and dynamic energy consumption; The computation latency of each pipeline stage is obtained based on the computing power of each edge server; the communication latency between adjacent pipeline stages is obtained based on the available bandwidth between adjacent edge servers; and the pipeline cycle time is obtained based on the computation latency of each pipeline stage and the communication latency between adjacent pipeline stages. Based on static and dynamic energy consumption, combined with pipeline cycle time, the energy consumption of each edge server is obtained: In the formula, For the first Energy consumption of an edge server For the first Static power consumption of an edge server For the first Dynamic power consumption of an edge server The production line cycle time. For spatial decision-making, Batch size; The energy consumption constraint for each edge server is that its energy consumption does not exceed the available energy for collection.

6. The task allocation method based on a large language model according to any one of claims 1 to 3, characterized in that, Based on multiple edge servers, the effective throughput of the distributed server cluster is obtained, including: The bottleneck migration latency is obtained based on the edge network bandwidth between adjacent edge servers: In the formula, For bottleneck migration latency, For server The number of layers that need to be migrated into or out in the current epoch relative to the previous epoch. The memory allocated to each layer of parameters. For the edge network bandwidth between servers, For spatial decision-making, For the first The discrete scheduling cycle is assigned to the first Edge servers The number of Transformer layers; By obtaining the scheduling cycle dynamically determined by renewable energy sources and combining it with bottleneck migration latency, the effective throughput of the distributed server cluster can be obtained.

7. The task allocation method based on a large language model according to any one of claims 1 to 3, characterized in that, Using spatial decision and batch size as decision variables, and based on partitioning constraints, energy consumption constraints, and effective throughput, a global configuration problem is established, including: In the formula, For spatial decision-making, let the first... The discrete scheduling cycle is assigned to the first One edge server The number of Transformer layers; For batch size, For efficient throughput; , For the number of edge servers, This represents the total number of layers in the large language model. To maintain pipeline connectivity, the edge server should handle the minimum number of Transformer layers. The memory allocated to each layer of parameters. For each layer and each batch size, the KV cache memory is... For batch size, For the first One edge server Memory budget; Energy consumption for each edge server The harvesting energy available for each edge server. 8.A task allocation apparatus based on a large language model, characterized by comprising: include: The first module is used to obtain tasks to be assigned, including: a large language model, renewable energy conditions, and a distributed server cluster, wherein the distributed server cluster includes multiple edge servers. The second module is used to partition the large language model onto various edge servers, ensuring that each edge server meets the partitioning constraints, and to obtain the initial results of the pipeline configuration, which includes spatial decision and batch size; based on renewable energy conditions and multiple edge servers, the energy consumption constraints of each edge server are obtained; based on multiple edge servers, the effective throughput of the distributed server cluster is obtained; using spatial decision and batch size as decision variables, and based on the partitioning constraints, energy consumption constraints, and effective throughput, a global configuration problem is established. The third module is used to obtain the effective capacity of the edge servers based on renewable energy conditions and the distributed server cluster; with the goal of the number of layers undertaken by each edge server equaling the effective capacity, it performs an equivalent transformation based on the effective capacity of the edge servers to transform the global configuration problem into a spatial sub-problem. The fourth module is used to perform a feasibility search on batch size based on the initial results of the pipeline configuration, and to perform discrete projection to solve the spatial subproblem and obtain the feasible results of the pipeline configuration. The fifth module is used to obtain the final result of the pipeline configuration based on the feasible results of the pipeline configuration, so that the distributed edge server cluster can match the renewable energy conditions and complete the task allocation. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images