Task processing method, device and system and electronic equipment

Abstract

The invention provides a task processing method, device and system and electronic equipment, and relates to the technical field of computers. The method comprises the following steps: allocating a plurality of LLM components to a cloud node and / or edge nodes according to resource data of the cloud node and each edge node and resource demand data of the plurality of LLM components; detecting whether a plurality of LLM tasks in a next task batch are received or not in the process that each LLM component processes the plurality of LLM tasks in the current task batch on the cloud node and / or the edge node; if it is determined that the latter task batch is received, whether processing of the current task batch needs to be interrupted or not is judged based on the value density of the LLM tasks in the two adjacent task batches; if yes, processing of the current task batch is interrupted, and based on the task processing requirement of the next task batch, LLM components are redistributed on the cloud node and / or the edge node, and processing the plurality of LLM tasks in the next task batch on the cloud node and / or the edge node based on the reallocated LLM component.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a task processing method, device, electronic device, storage medium, and computer program product. Background Art

[0002] In recent years, the edge computing landscape has undergone significant changes as connected devices have become ubiquitous across sectors such as the Industrial Internet of Things, healthcare, smart cities, and personal electronics. Unlike traditional computing environments, these edge devices typically have limited resources and require efficient processing power to meet the real-time demands of applications such as predictive maintenance, health monitoring, autonomous vehicles, and intelligent decision-making systems. These applications demand low-latency responses and strong performance, posing challenges for deploying LLM models on these devices. LLMs such as GPT-4, PaLM-2, and LLaMA-2 have emerged as powerful AI technologies, built on transformer architectures with billions of parameters. While these models have demonstrated remarkable capabilities in natural language processing, their high computational intensity and memory requirements make them challenging to deploy on resource-constrained edge devices. Integrating LLMs into edge computing applications offers significant potential for enhancing user interaction and decision-making at the edge, but directly deploying LLM models on edge devices presents several challenges, including large model size, high computational requirements, and high memory usage.

[0003] From the above, we can see that the problem of difficulty in deploying LLM models on edge devices still needs to be solved. Summary of the Invention

[0004] This application provides a task processing method, device, system, and electronic device that can solve the problem of difficulty in deploying LLM models on edge devices in related technologies. The technical solution is as follows:

[0005] According to one aspect of the present application, a task processing method is applied to a task processing system, wherein the task processing system includes a cloud node and at least one edge node; the method includes: allocating multiple LLM components to the cloud node and / or edge node based on resource data of the cloud node and each edge node, and resource requirement data of multiple LLM components; the LLM components are used to process LLM tasks in a task processing model; in the process of each LLM component processing multiple LLM tasks in a current task batch on the cloud node and / or the edge node, detecting whether multiple LLM tasks in a subsequent task batch are received; if it is determined that the subsequent task batch is received, judging whether it is necessary to interrupt the processing of the current task batch based on the value density of each LLM task in two adjacent task batches; if so, interrupting the processing of the current task batch, and reallocating LLM components on the cloud node and / or the edge node based on the task processing requirements of the subsequent task batch, so as to process the multiple LLM tasks in the subsequent task batch on the cloud node and / or the edge node based on the reallocated LLM components.

[0006] According to one aspect of the present application, a task processing device is characterized in that it is applied to a task processing system, wherein the task processing system includes a cloud node and at least one edge node; the device includes: a component allocation module, which is used to allocate multiple LLM components to the cloud node and / or edge node based on resource data of the cloud node and each edge node, and resource requirement data of multiple LLM components; the LLM components are used to process LLM tasks in a task processing model; a task processing module, which is used to detect whether multiple LLM tasks in a subsequent task batch are received during the process of each LLM component processing multiple LLM tasks in a current task batch on the cloud node and / or edge node; a task interruption module, which is used to determine whether it is necessary to interrupt the processing of the current task batch based on the value density of each LLM task in two adjacent task batches if it is determined that the subsequent task batch is received; if so, interrupt the processing of the current task batch, and reallocate the LLM components on the cloud node and / or edge node based on the computing requirements of the subsequent task batch, so as to process the multiple LLM tasks in the subsequent task batch on the cloud node and / or edge node based on the reallocated LLM components.

[0007] According to one aspect of the present application, a task processing system is characterized in that the task processing system includes a main control module, a cloud node and at least one edge node; the main control module is used to allocate multiple LLM components to the cloud node and / or edge node according to resource data of the cloud node and each edge node, and resource demand data of multiple LLM components; the LLM components are used to process LLM tasks in the task processing model; in the process of each LLM component processing multiple LLM tasks in a current task batch on the cloud node and / or edge node, it is detected whether multiple LLM tasks in a subsequent task batch are received; if it is determined that multiple LLM tasks in a subsequent task batch are received, the LLM components are detected. For the next task batch, based on the value density of each LLM task in the two adjacent task batches, it is determined whether it is necessary to interrupt the processing of the current task batch; if so, the processing of the current task batch is interrupted, and based on the computing requirements of the next task batch, the LLM components are reallocated on the cloud nodes and / or edge nodes to process multiple LLM tasks in the next task batch on the cloud nodes and / or edge nodes based on the reallocated LLM components; the cloud nodes are used to run the allocated LLM components and process the allocated LLM tasks; the edge nodes are used to run the allocated LLM components and process the allocated LLM tasks.

[0008] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the task processing method described above is implemented.

[0009] According to one aspect of the present application, a storage medium stores a computer program thereon, and when the computer program is executed by one or more processors, the task processing method described above is implemented.

[0010] According to one aspect of the present application, a computer program product includes a computer program, and when the computer program is executed by one or more processors, the task processing method described above is implemented.

[0011] The beneficial effects of the technical solution provided by this application are:

[0012] In the above technical solution, a cloud-edge collaboration framework is adopted to deploy lightweight LLM components in a resource-constrained edge node computing environment. The architecture can achieve efficient LLM task allocation between cloud nodes and edge nodes, ensuring low-latency processing and optimal resource utilization for real-time applications in various edge computing scenarios such as industrial Internet of Things, smart cities and healthcare systems, thereby effectively solving the problem of difficulty in deploying LLM models on edge devices in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts.

[0014] Figure 1 It is a schematic diagram of a task processing system;

[0015] Figure 2 is a hardware structure diagram of an electronic device according to an exemplary embodiment;

[0016] Figure 3 is a flowchart showing a task processing method according to an exemplary embodiment;

[0017] Figure 4 yes Figure 3 A flowchart of an embodiment corresponding to step 310 in an embodiment;

[0018] Figure 5 yes Figure 4 A flowchart of an embodiment corresponding to step 311 in an embodiment;

[0019] Figure 6 yes Figure 3 A flowchart of step 330 in one embodiment corresponding to the embodiment;

[0020] Figure 7 yes Figure 3 A flowchart of an embodiment corresponding to step 350 in an embodiment;

[0021] Figure 8 It is a schematic diagram of a specific implementation of a task processing system in an application scenario;

[0022] Figure 9 is a structural block diagram of a task processing device according to an exemplary embodiment;

[0023] Figure 10 The figure is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0024] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0025] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0026] As mentioned earlier, deploying LLM models directly on edge devices faces several challenges, including large model size, high computational requirements, and large memory usage.

[0027] Deploying large language models (LLMs) in edge computing environments presents multiple challenges. While cloud-based approaches can leverage powerful computing resources, they often suffer from high latency, typically exceeding 200 milliseconds, which is unacceptable for real-time applications requiring low latency. Furthermore, transmitting sensitive data to centralized cloud servers poses serious privacy risks. On the other hand, approaches that rely entirely on edge devices often sacrifice model accuracy and functionality due to computational limitations, making them incapable of handling complex inference tasks, which are essential in modern edge computing scenarios.

[0028] While collaborative frameworks combining cloud and edge computing attempt to address these shortcomings, existing technologies often lack efficient resource management strategies, making it difficult to adapt to fluctuating network conditions or heterogeneous device capabilities under dynamic workloads. Some systems employ static model partitioning or fixed quantization levels, which cannot dynamically adjust to real-time network status or device resources. Furthermore, these frameworks often lack effective task priority management mechanisms, treating all requests equally and ignoring the varying urgency of tasks in real-world applications, leading to increased latency and reduced resource utilization for critical tasks. Furthermore, existing solutions suffer from high energy and memory overhead, making them impractical for battery-powered edge devices. This inefficiency further limits the system's scalability in heterogeneous edge environments, as devices with varying computing capabilities must collaborate efficiently to achieve optimal performance. Overall, existing approaches struggle to simultaneously balance performance, energy efficiency, and scalability in diverse and unpredictable edge computing scenarios.

[0029] Current approaches to address these issues face numerous limitations. For example, while offloading all LLM computational tasks to a centralized cloud can simplify the burden on edge devices, it introduces significant latency and privacy issues, especially in time-sensitive applications. Techniques such as knowledge distillation reduce computational requirements by extracting smaller models from large ones, but often at the expense of model accuracy. Similarly, model compression techniques such as quantization and pruning can reduce computational and storage overhead but struggle to maintain model performance when processing complex tasks. Some edge solutions deploy LLMs closer to end devices through techniques such as adaptive quantization and feature map caching, but still struggle to find an optimal balance between accuracy and resource constraints. Collaborative frameworks that employ early exit mechanisms and cloud-based context management can dynamically allocate computational tasks, but often perform suboptimally in the face of unpredictable network conditions or device heterogeneity. Federated learning approaches allow for efficient model fine-tuning on edge devices, but achieving a good balance between communication overhead, training time, and model performance remains challenging.

[0030] Overall, existing solutions primarily focus on reducing model size or improving cloud-edge collaborative processing, but they do not fully meet the demands of complex edge computing applications. These demands include dynamic adaptability, energy efficiency optimization, task priority management, and real-time responsiveness, while also ensuring optimal resource utilization. An ideal edge computing framework should balance latency, energy consumption, and performance in highly dynamic environments, providing a superior solution for future edge inference applications.

[0031] From the above, it can be seen that the relevant technology still has the defect of being difficult to deploy the LLM model on edge devices.

[0032] To this end, the task processing method provided in this application can effectively improve the accuracy of task processing. Accordingly, the task processing method is applicable to a task processing device, which can be deployed in an electronic device. The electronic device can be a computer device configured with a von Neumann architecture, for example, the computer device includes a desktop computer, a laptop computer, a server, etc.

[0033] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0034] See also Figure 1 , Figure 1 The diagram is a schematic diagram of a task processing system according to an exemplary embodiment.

[0035] like Figure 1 As shown, a task processing system includes a main control module 110, a cloud node 130 and at least one edge node 150;

[0036] The main control module 110 is configured to allocate multiple LLM components to cloud nodes and / or edge nodes based on resource data of the cloud nodes and edge nodes, and resource requirement data of the multiple LLM components; the LLM components are configured to process LLM tasks in a task processing model; while each LLM component is processing multiple LLM tasks in a current task batch on the cloud node and / or edge node, detect whether multiple LLM tasks in a subsequent task batch are received; if it is determined that the subsequent task batch is received, determine whether it is necessary to interrupt the processing of the current task batch based on the value density of each LLM task in two adjacent task batches; if so, interrupt the processing of the current task batch, and reallocate the LLM components on the cloud nodes and / or edge nodes based on the task processing requirements of the subsequent task batch, so that the multiple LLM tasks in the subsequent task batch are processed on the cloud nodes and / or edge nodes based on the reallocated LLM components;

[0037] Cloud node 130, for running assigned LLM components and processing assigned LLM tasks;

[0038] The edge node 150 is configured to run the assigned LLM components and process the assigned LLM tasks.

[0039] See also Figure 2 , Figure 2 This is a hardware structure diagram of an electronic device according to an exemplary embodiment. Figure 1 The main control module 110 and the cloud node 130 in the task processing system are shown.

[0040] It should be noted that the electronic device is only an example adapted for this application and cannot be considered to provide any limitation on the scope of use of this application. The electronic device cannot be interpreted as needing to rely on or must have Figure 2 One or more components of exemplary electronic device 200 are shown.

[0041] The hardware structure of the electronic device 200 may vary greatly due to different configurations or performances, such as Figure 2 As shown, the electronic device 200 includes a power supply 210 , an interface 230 , at least one memory 250 , and at least one central processing unit (CPU) 270 .

[0042] Specifically, the power supply 210 is used to provide operating voltage for various hardware devices on the electronic device 200 .

[0043] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. Figure 1The interaction between the terminal 100 and the electronic device 200 in the implementation environment is shown.

[0044] Of course, in other examples adapted by this application, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, etc. Figure 2 As shown, this does not constitute a specific limitation.

[0045] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon include an operating system 251, application 253 and data 255, etc. The storage method can be temporary storage or permanent storage.

[0046] Among them, the operating system 251 is used to manage and control the various hardware devices and application programs 253 on the electronic device 200 to enable the central processing unit 270 to calculate and process the massive data 255 in the memory 250. It can be WindowsServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0047] The application 253 is a computer program formed by computer-readable instructions based on the operating system 251 to perform at least one specific task, and may include at least one module ( Figure 2 Each module may include corresponding computer-readable instructions. For example, the task processing device may be considered as an application 253 deployed on the electronic device 200.

[0048] The data 255 may be photos, pictures, etc. stored in a disk, or may be LLM tasks, etc. stored in the memory 250 .

[0049] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read computer programs stored in the memory 250, thereby performing operations and processing on the massive amount of data 255 in the memory 250. For example, the task processing method is completed by the central processing unit 270 reading the application program 253 stored in the memory 250.

[0050] In addition, the present application can also be implemented through hardware circuits or hardware circuits combined with software. Therefore, the implementation of the present application is not limited to any specific hardware circuits, software, or a combination of the two.

[0051] See also Figure 3 , the embodiment of the present application provides a task processing method, which is applicable to an electronic device, for example, the electronic device may be Figure 1The main control module 110 in the task processing system is shown. The hardware structure of the electronic device can be as follows: Figure 2 shown.

[0052] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0053] like Figure 3 As shown, the method may include the following steps:

[0054] Step 310 : Allocate multiple LLM components to cloud nodes and / or edge nodes according to resource data of the cloud nodes and edge nodes, and resource requirement data of multiple LLM components.

[0055] The LLM component is used to process LLM tasks in the task processing model.

[0056] First of all, it should be noted that cloud nodes are cloud devices with high-performance computing capabilities, such as cloud servers or data center nodes; edge nodes are devices deployed at the edge of the network and have certain computing resources, such as edge computing servers, edge inference gateways, etc.; among them, the task processing capabilities of edge nodes are lower than those of cloud nodes, but they have lower access latency and localized processing advantages.

[0057] Regarding LLM components, they can be functional units in the task processing model, used to perform LLM tasks related to natural language processing in the task processing system, including but not limited to language understanding, generation, question answering, and reasoning.

[0058] A task processing model refers to a large language model (LLM) that has language understanding and generation capabilities and can support a variety of natural language processing tasks. A task processing model is a deep learning model trained using a large amount of text data, such as ChatGPT and DeepSeek.

[0059] Resource data may refer to data on how much resources the cloud node / edge node itself possesses, such as the current available computing power, available memory capacity, available bandwidth, energy consumption cost, etc. of the cloud node or edge node.

[0060] Resource requirement data refers to the data on how much resources the LLM component needs to run, such as computing requirements, memory requirements, bandwidth requirements, etc.

[0061] It's understood that the resource data corresponding to cloud nodes or edge nodes must meet the resource requirements of LLM components; otherwise, the LLM components will not run smoothly. For example, if the available computing power of the cloud node exceeds the computing requirements of the LLM component, and the available computing power of the edge node is less than the computing requirements of the LLM component, then the LLM component cannot run on the edge node and must be allocated to the cloud node.

[0062] In one possible implementation, such as Figure 4 As shown, step 310 may include the following steps:

[0063] Step 311 , based on the resource requirement data of each LLM component, the resource data of the cloud node, and the resource data corresponding to each edge node, determine a corresponding cloud node or edge node for each LLM component according to a third constraint condition;

[0064] The third constraint is a condition for determining whether the LLM component can run on a cloud node or an edge node.

[0065] Specifically, the third constraint condition may include that the resource data corresponding to the cloud node or edge node needs to be greater than the resource requirement data of the LLM component, for example, the available computing power and available memory capacity of the cloud node or edge node need to be greater than the quantitative computing requirements and quantitative memory requirements corresponding to the LLM component.

[0066] In one possible implementation, such as Figure 5 As shown, step 311 may further include the following steps:

[0067] Step 3111 : Based on the resource requirement data of each LLM component, the quantitative computing requirement and the quantitative memory requirement corresponding to each LLM component are calculated.

[0068] First, it should be noted that the resource requirement data may include the computing requirements and memory requirements of the LLM component. Then, based on the computing requirements and memory requirements, the quantitative computing requirements and quantitative memory requirements corresponding to the LLM component may be calculated.

[0069] Specifically, the quantization computing requirements can be calculated based on the computing requirements and the quantization factor, and the quantization memory requirements can be calculated based on the memory requirements and the quantization factor. The quantization factor is the scaling factor used when mapping floating-point values ​​to integer representations. It is the proportional coefficient required to compress the model's weights, activations, etc. from high precision (such as FP32) to low precision (such as INT8, INT4). The quantization factor can be obtained based on maximum value (Min-Max) static quantization, based on minimum mean square error (MSE), or based on minimum KL divergence, and is not limited here.

[0070] By reducing the accuracy of model weights and activations through quantization factors, memory and computing requirements are greatly reduced, enabling large-scale task processing models to be deployed on edge nodes with limited resources, thereby improving the scalability of heterogeneous edge environments.

[0071] Step 3113: Based on the resource data corresponding to the cloud node, determine the available computing power and available memory capacity corresponding to the cloud node.

[0072] Specifically, the local system monitoring module of the cloud node can read in real time or regularly report parameter information such as current processing power and free memory, so as to determine the available computing power and available memory capacity corresponding to the cloud node.

[0073] Step 3115: Based on the resource data corresponding to each edge node, determine the available computing power and available memory capacity corresponding to each edge node.

[0074] Specifically, the local system monitoring module of the edge node can read in real time or regularly report parameter information such as current processing power and free memory, thereby determining the available computing power and available memory capacity corresponding to the edge node.

[0075] Step 3117, based on the quantified computing requirements and quantified memory requirements corresponding to each LLM component, the available computing power and available memory capacity corresponding to the cloud node, and the available computing power and available memory capacity corresponding to each edge node, determine the corresponding cloud node or edge node for each LLM component.

[0076] Specifically, the available computing power and available memory capacity of the node or edge node need to be greater than the quantitative computing requirements and quantitative memory requirements corresponding to the LLM component. Then, the corresponding cloud node or edge node can be determined for the LLM component.

[0077] Through the above process, a quantization-aware method is introduced to divide and distribute LLM components between cloud nodes and edge nodes. LLM components can be strategically allocated between the cloud and edge layers based on computing requirements, memory availability, and network limitations, and the accuracy levels of different layers of the model can be dynamically adjusted. By reducing the accuracy of the task processing model weights and activations, the memory and computing requirements are greatly reduced, enabling large-scale task processing models to be deployed on edge nodes with limited resources, thereby improving the scalability of heterogeneous edge environments.

[0078] Step 313: Using a dynamic programming method, each LLM component is assigned to an edge node or a cloud node for execution.

[0079] First of all, it should be noted that there may be both cloud nodes and edge nodes that can run LLM components smoothly, or there may be multiple edge nodes that can run LLM components at the same time. In this case, dynamic programming methods can be used to allocate the optimal nodes to LLM components.

[0080] Regarding the dynamic programming approach, the processing and communication delays between consecutive LLM components on different nodes are calculated. The total delay is calculated as the sum of the processing and communication delays, and a DP table is updated to track the minimum delay for each LLM component-node assignment. Finally, the node (edge ​​node / cloud node) with the minimum total delay is selected and assigned to the LLM component.

[0081] It should be noted that the LLM component is a component in the task processing model, specifically, it may correspond to different model layers in the task processing model. Then, continuous LLM components may refer to adjacent model layers in the task processing model.

[0082] Regarding processing delay, the processing delay of the LLM component when running on a cloud node or edge node can be obtained by dividing the quantified computing demand corresponding to the LLM component by the available computing power corresponding to the cloud node or edge node.

[0083] Regarding the communication delay, it can be calculated by formula (1):

[0084]

[0085] Among them, l refers to the number of the model layer corresponding to LLM, β l-1，l It refers to the output data volume of layer l-1 (that is, the input of layer l); It refers to the network bandwidth between nodes at layer l-1, for example, layer 3 is at the edge device and layer 4 is at the cloud node.

[0086] A DP table (Dynamic Programming Table) is a two-dimensional or multi-dimensional table structure used to store optimal solutions to subproblems in dynamic programming methods, avoiding repeated computations and enabling state transitions, ultimately leading to the global optimal solution. Specifically, a DP table records the minimum cost (i.e., total latency) for each model layer when running on a particular node. The DP table can be used for state transitions in subsequent layers, allowing backward tracing from the DP table to obtain the globally optimal component (model layer)-node allocation path.

[0087] Through the above process, dynamic programming methods can be used to adapt to different workloads and ensure that LLM components are effectively distributed between cloud nodes and edge nodes. This adaptive resource management prevents any single node from being overloaded and ensures that the system can efficiently scale in heterogeneous edge environments with different computing power and memory levels.

[0088] Furthermore, the dynamic programming approach is able to minimize energy consumption by processing as many LLM components as possible on the edge device, reducing the need for offloading to cloud nodes. This is particularly beneficial for battery-powered edge devices such as IoT sensors, wearables, and other mobile devices, where energy efficiency is critical for long-term operation in remote or mobile environments.

[0089] Step 330 : During the process of processing the multiple LLM tasks in the current task batch on the cloud node and / or edge node, each LLM component detects whether multiple LLM tasks in the next task batch are received.

[0090] First of all, it should be noted that in the process of each LLM component processing multiple LLM tasks in the current task batch on the cloud node and / or edge node, since the current task batch includes multiple LLM tasks, the urgency or priority of different LLM tasks is different. In the process of processing multiple LLM tasks in the current task batch, the task processing system can prioritize the various LLM tasks in the current task batch in order to improve task processing efficiency and resource utilization, and execute tasks according to the sorting results.

[0091] In one possible implementation, such as Figure 6 As shown, step 330 may include the following steps:

[0092] Step 331 : Calculate the value density corresponding to each LLM task based on the task attributes corresponding to each LLM task in the current task batch.

[0093] Among them, the task attributes can include the task arrival time, task deadline and priority value corresponding to the LLM task. Then, the value density corresponding to each LLM task can be calculated by formula (2).

[0094]

[0095] Among them, ρ j is the value density, is the task arrival time, refers to the task deadline, v j Refers to the priority value.

[0096] The priority value can be determined based on the user level weight, task value, service level, etc. corresponding to the LLM task.

[0097] Step 333 : Determine the priority order of each LLM task based on the value density corresponding to each LLM task.

[0098] In step 335 , each LLM component processes each LLM task on the cloud node and / or edge node according to the priority order.

[0099] Then, the higher the value density, the higher the priority of the LLM task. The task processing system will arrange the LLM tasks in order from large to small value density and execute the task with the highest priority first.

[0100] It should be further explained that the resources required to process different LLM tasks are different. Since the resource data of edge nodes and cloud nodes are different, if the resource data of edge nodes can meet the needs of LLM tasks, then LLM tasks will be processed on edge nodes first. If the resource data of edge nodes cannot meet the needs of LLM tasks, then LLM tasks need to be transferred to cloud nodes for execution.

[0101] In one possible implementation, step 330 may include: based on the task processing requirements of each LLM task, the resource data corresponding to the cloud node, and the resource data corresponding to each edge node, allocating a cloud node or an edge node to each LLM task according to the first constraint condition, so that each LLM task completes task processing on the corresponding cloud node or edge node.

[0102] The first constraint is used to determine feasible cloud nodes and edge nodes for the LLM task. The first constraint may include Among them, φ n (t) refers to the available computing power of the cloud node or edge node. Refers to the computational requirements of LLM task processing, M n Refers to the available memory capacity of the cloud node or edge node. Refers to the content capacity required for LLM task processing, b e (t) refers to the available bandwidth corresponding to the cloud node or edge node, r j Refers to the bandwidth required for LLM task processing.

[0103] Among them, task processing requirements may include computing requirements, content requirements and bandwidth requirements. The computing requirements can be determined based on the total computing amount required to execute the LLM task. The content capacity required for the LLM task can be determined based on the data volume of the LLM task. The data volume and processing time of the LLM task uploaded and output can be input to determine the bandwidth required for the LLM task. No specific limitations are made here.

[0104] It should be noted that if the LLM task is an urgent task, the LLM task must be responded to within seconds, and there is no need to wait for the LLM component to be redeployed before starting. Task scheduling is given priority to ensure response to real-time requests.

[0105] Specifically, when an LLM task arrives, it prioritizes the currently deployed LLM component structure and the resource status of each node to quickly complete task allocation and entry node selection, ensuring timely task execution. Simultaneously, the task processing system asynchronously analyzes the match between the resource requirements of the task batch and the current LLM component distribution in the background. If the edge node cannot meet the task requirements, the missing components are loaded from the cloud or MEC node in real time. Alternatively, if the deployment is found to be unreasonable or there is room for optimization, the quantitative perception layer placement algorithm can be triggered to redeploy some or all LLMs for subsequent task batches.

[0106] Step 350: If it is determined that the next task batch is received, then based on the value density of each LLM task in two adjacent task batches, it is determined whether the processing of the current task batch needs to be interrupted.

[0107] First, task batches are divided based on time slots. LLM tasks within each time slot are grouped into a batch. When new LLM tasks arrive, if they belong to the current time slot, they are added to the current task batch; if they belong to a new time slot, they form the next task batch.

[0108] It's understandable that the arrival time of the current task batch is earlier than the arrival time of the subsequent task batch. However, this doesn't necessarily mean that the "first-arriving task batch" is more important. If the subsequent task batch includes urgent, high-value, or high-service-level LLM tasks, then, in order to improve the overall task processing efficiency and task value output of the task processing system, when the task processing system detects the arrival of the subsequent task batch while processing the current task batch, it can evaluate it based on the cumulative value density and estimated energy consumption of the two batches of tasks. If the cumulative value density of the current task batch is higher and the execution cost is lower, the task processing system can interrupt the task processing of the current task batch and prioritize the subsequent task batch to achieve efficient resource utilization and service quality assurance.

[0109] In one possible implementation, such as Figure 7 As shown, step 350 may further include the following steps:

[0110] Step 351 : Calculate the cumulative value density corresponding to the current task batch and the next task batch based on the value density corresponding to each LLM task in two adjacent task batches.

[0111] Step 353: If the relationship between the cumulative value density corresponding to the next task batch and the cumulative value density corresponding to the current task batch satisfies the third constraint condition, the energy consumption corresponding to the current task batch and the energy consumption corresponding to the next task batch are determined.

[0112] The third constraint condition is used to determine whether to trigger task preemption, that is, to determine whether to interrupt the current task batch and give priority to the next user batch.

[0113] In one possible implementation, the third constraint is in, is the cumulative value density of the next task batch, is the cumulative value density of the current task, and γ is the preemption threshold.

[0114] Regarding the preemption threshold, it is used to control the sensitivity of triggering task batch preemption. The preemption threshold can prevent frequent preemption and optimize resource utilization. The value of the preemption threshold can comprehensively consider the task urgency, node resource status and communication cost, thereby balancing the relationship between delay optimization and increased energy consumption brought about by task migration.

[0115] Regarding the comparison of energy consumption, energy efficiency can be further optimized to prevent preemption from causing deterioration of the overall energy consumption of the system. In one possible implementation, step 353 may include: calculating energy consumption based on the task processing requirements of the current task batch, the resource data corresponding to the cloud node, and the resource data corresponding to each edge node, to obtain the energy consumption corresponding to the current task batch; calculating energy consumption based on the task processing requirements of the next task batch, the resource data corresponding to the cloud node, and the resource data corresponding to each edge data, to obtain the energy consumption corresponding to the next task batch.

[0116] The task processing requirements may refer to the computing requirements, memory requirements, bandwidth requirements, etc. required to process each LLM task in a task batch.

[0117] It should be noted that resource data can also include node energy consumption; then, based on the task processing requirements and the energy consumption in the resource data, the energy consumption corresponding to the current task batch and the energy consumption corresponding to the next task batch can be calculated.

[0118] Step 355 : When the energy consumption corresponding to the next task batch is less than the energy consumption corresponding to the current task batch, the next task batch is processed first.

[0119] Through the above process, the Value Density First (VDF) scheduling algorithm with dynamic preemption is introduced. This algorithm determines the priority of tasks based on factors such as urgency, computational complexity, and resource efficiency. It ensures optimal resource allocation by grouping tasks into batches and allowing higher-priority tasks to preempt lower-priority tasks when necessary. This mechanism helps ensure the real-time responsiveness of edge computing applications by minimizing the latency of critical tasks.

[0120] Step 370: interrupt the processing of the current task batch, and reallocate the LLM components on the cloud nodes and / or edge nodes based on the task processing requirements of the next task batch, so as to process multiple LLM tasks in the next task batch on the cloud nodes and / or edge nodes based on the reallocated LLM components.

[0121] Specifically, the processing flow of the current task batch can be interrupted first, and the computing resources, memory resources, bandwidth resources, etc. allocated to the current task batch in the cloud nodes and edge nodes can be released. Subsequently, based on the task processing requirements corresponding to the multiple LLM tasks in the next task batch and the current resource data of each node, the task processing system allocates the LLM nodes to cloud nodes or edge nodes for task processing.

[0122] The task processing system recalculates the quantified computational and memory requirements of each LLM component, combines resource data from cloud nodes and edge nodes, and utilizes quantitatively aware layer placement technology to determine the optimal deployment location for each LLM component. After deployment, the system processes multiple LLM tasks from the next task batch on cloud nodes and / or edge nodes based on the new LLM component allocation structure, ensuring that the next task batch receives priority execution after preemption, and enabling dynamic resource scheduling and optimal utilization.

[0123] Through the above process, this solution adopts a cloud-edge collaboration framework for deploying lightweight LLM components in resource-constrained edge node computing environments. The architecture can achieve efficient LLM task distribution between cloud nodes and edge nodes, ensuring low-latency processing and optimal resource utilization for real-time applications in various edge computing scenarios such as industrial Internet of Things, smart cities, and healthcare systems.

[0124] Figure 8 This diagram illustrates a specific implementation of a task processing system for an application scenario. In this application scenario, the task processing system implements a task processing methodology validated through extensive simulations and field experiments using real-world datasets and CIoT scenarios. The task processing system was tested in a multi-layer cloud-edge setup, where edge nodes represent various CIoT devices (such as smart home hubs, wearables, and sensors), while cloud nodes run in high-performance data centers.

[0125] The application scenario uses two widely recognized CIoT datasets:

[0126] MedBIoT: This dataset is used for IoT-based botnet detection in healthcare settings and includes sensor data from various connected medical devices.

[0127] Smart Home: A dataset consisting of smart home sensor data, including temperature, motion, and humidity data read by IoT devices used in daily home management.

[0128] The task processing system is implemented on a test platform consisting of five edge nodes and a cloud server. The edge nodes are equipped with Intel Xeon D-2146NT CPU and NVIDIA TM GeForce GTX 1650GPU, while the cloud server uses high-end NVIDIA TM RTX 2080Ti GPU. Network conditions varied from 10Mbps to 100Mbps to simulate different bandwidth availability levels for real-world CIoT applications.

[0129] Key Performance Indicators

[0130] Latency: This task processing system significantly reduces inference latency by processing LLM tasks locally on edge nodes. In real-time scenarios, edge layer processing can reduce latency by up to 80% compared to traditional cloud-only approaches.

[0131] Energy efficiency: By executing LLM task inference locally on edge nodes, the energy consumption of the task processing system is optimized. Compared with pure cloud solutions, energy consumption is reduced by 50% because less data is transmitted over the network.

[0132] Throughput: The throughput (measured in the number of tasks processed per unit time) of lightweight models deployed on edge nodes increased by 40% due to reduced reliance on cloud computing and dynamic batch scheduling that optimizes resource allocation.

[0133] Resource Utilization: The task processing system effectively balances the computing load between the cloud and the edge, ensuring efficient utilization of both resources. This dynamic resource management improves overall system throughput by 20%, avoiding overloading edge nodes while also keeping communication costs low.

[0134] In this application scenario, by leveraging cloud-edge collaboration, dynamic task scheduling, and quantization-aware deployment, the proposed task processing system not only improves latency and energy efficiency but also ensures scalability for real-time CIoT applications. These results demonstrate the ability of the proposed task processing system to effectively handle the unique challenges faced by resource-constrained CIoT environments.

[0135] In addition, lightweight LLMs (such as TinyLlama-1.1B) can be efficiently executed on edge devices while leveraging the computing power of cloud resources for more complex operations. The integration of these LLM components significantly reduces latency (80%), improves throughput by 40%, reduces energy consumption and memory usage, while maintaining model accuracy under different edge device and network conditions.

[0136] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0137] The following are embodiments of the apparatus of the present application, which can be used to execute the task processing method involved in the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the task processing method involved in the present application.

[0138] See also Figure 9 In an embodiment of the present application, a task processing device 900 is provided, including but not limited to: a component allocation module 910, a task processing module 930, and a task interruption module 950.

[0139] The component allocation module 910 is configured to allocate multiple LLM components to cloud nodes and / or edge nodes based on resource data of the cloud nodes and edge nodes and resource requirement data of multiple LLM components; the LLM components are configured to process LLM tasks in the task processing model;

[0140] The task processing module 930 is configured to detect whether multiple LLM tasks in a subsequent task batch are received during the process of each LLM component processing multiple LLM tasks in a current task batch on a cloud node and / or an edge node;

[0141] The task interruption module 950 is used to determine whether it is necessary to interrupt the processing of the current task batch based on the value density of each LLM task in the two adjacent task batches if it is determined that the next task batch has been received; if so, interrupt the processing of the current task batch, and reallocate the LLM components on the cloud nodes and / or edge nodes based on the computing requirements of the next task batch, so as to process multiple LLM tasks in the next task batch on the cloud nodes and / or edge nodes based on the reallocated LLM components.

[0142] It should be noted that the task processing device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when performing task processing. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the task processing device will be divided into different functional modules to complete all or part of the functions described above.

[0143] In addition, the task processing device and the task processing method provided in the above embodiments belong to the same concept, wherein the specific manner in which each module performs operations has been described in detail in the method embodiments and will not be repeated here.

[0144] See also Figure 10 In an embodiment of the present application, an electronic device 4000 is provided. The electronic device 4000 may include: a desktop computer, a laptop computer, a server, etc.

[0145] exist Figure 10 In the embodiment, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0146] Data exchange between the processor 4001 and the memory 4003 can be achieved via at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, for example. The communication bus 4002 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0147] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0148] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0149] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store a computer program in the form of instructions or data structures and can be accessed by the electronic device 400, but is not limited to these.

[0150] The memory 4003 stores a computer program, and the processor 4001 can read the computer program stored in the memory 4003 through the communication bus 4002 .

[0151] The computer program is executed by one or more processors 4001 to implement the task processing methods in the above-mentioned embodiments.

[0152] In addition, an embodiment of the present application provides a storage medium on which a computer program is stored. The computer program is executed by one or more processors to implement the task processing method as described above.

[0153] A computer program product is provided in an embodiment of the present application, including a computer program, which is executed by one or more processors to implement the task processing method described above.

[0154] Compared with related technologies, this solution adopts a cloud-edge collaboration framework to deploy lightweight LLM components in resource-constrained edge node computing environments. The architecture can achieve efficient LLM task distribution between cloud nodes and edge nodes, ensuring low-latency processing and optimal resource utilization for real-time applications in various edge computing scenarios such as industrial Internet of Things, smart cities, and healthcare systems.

[0155] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A task processing method, characterized in that: Applied to a task processing system, the task processing system includes a cloud node and at least one edge node; the method includes: Allocating multiple LLM components to the cloud nodes and / or edge nodes based on resource data of the cloud nodes and edge nodes and resource requirement data of multiple LLM components; the LLM components are used to process LLM tasks in the task processing model; During the process of each of the LLM components processing the multiple LLM tasks in a current task batch on the cloud node and / or the edge node, detecting whether multiple LLM tasks in a subsequent task batch are received; If it is determined that the latter task batch is received, determining whether it is necessary to interrupt the processing of the current task batch based on the value density of each LLM task in the two adjacent task batches; If yes, interrupt the processing of the current task batch, and reallocate LLM components on the cloud node and / or the edge node based on the task processing requirements of the subsequent task batch, so as to process multiple LLM tasks in the subsequent task batch on the cloud node and / or the edge node based on the reallocated LLM components.

2. The method according to claim 1, wherein The process in which each LLM component processes multiple LLM tasks in a current task batch on a cloud node and / or an edge node includes: Calculating the value density corresponding to each of the LLM tasks based on the task attributes corresponding to each of the LLM tasks in the current task batch; Determining a priority order of each of the LLM tasks based on the value density corresponding to each of the LLM tasks; Each of the LLM components processes each of the LLM tasks on the cloud node and / or edge node according to the priority order.

3. The method according to claim 1, wherein The process in which each LLM component processes multiple LLM tasks in a current task batch on a cloud node and / or an edge node includes: Based on the task processing requirements of each LLM task, the resource data corresponding to the cloud node, and the resource data corresponding to each edge node, cloud nodes or edge nodes are allocated to each LLM task according to the first constraint condition, so that each LLM task completes task processing on the corresponding cloud node or edge node.

4. The method according to claim 1, wherein The determining whether it is necessary to interrupt the processing of a current task batch based on the value density of each LLM task in two adjacent task batches includes: Calculate the cumulative value density corresponding to the current task batch and the next task batch based on the value density corresponding to each of the LLM tasks in two adjacent task batches; If the relationship between the cumulative value density corresponding to the subsequent task batch and the cumulative value density corresponding to the current task batch satisfies the second constraint condition, determining the energy consumption corresponding to the current task batch and the energy consumption corresponding to the subsequent task batch; In the case that the energy consumption corresponding to the subsequent task batch is less than the energy consumption corresponding to the current task batch, the subsequent task batch is processed first.

5. The method according to claim 4, wherein The determining of the energy consumption corresponding to the current task batch and the energy consumption corresponding to the subsequent task batch includes: Performing energy consumption calculation based on the task processing requirements of the current task batch, the resource data corresponding to the cloud node, and the resource data corresponding to each edge node to obtain the energy consumption corresponding to the current task batch; Energy consumption is calculated based on the task processing requirements of the subsequent task batch processing, the resource data corresponding to the cloud node, and the resource data corresponding to each edge data to obtain the energy consumption corresponding to the subsequent task batch.

6. The method according to any one of claims 1 to 5, characterized in that The multiple LLM components are allocated to the cloud nodes and / or edge nodes according to the resource data of the cloud nodes and each edge node and the resource demand data of the multiple LLM components. Determining a corresponding cloud node or edge node for each LLM component according to a third constraint based on the resource requirement data of each LLM component, the resource data of the cloud node, and the resource data corresponding to each edge node; By using a dynamic programming method, each of the LLM components is respectively assigned to the edge node or the cloud node for execution.

7. The method according to claim 6, wherein The determining, based on the resource requirement data of each LLM component, the resource data of the cloud node, and the resource data corresponding to each edge node, respectively, for each LLM component according to a third constraint condition, a corresponding cloud node or edge node includes: Calculating the quantitative computing requirements and the quantitative memory requirements corresponding to each LLM component based on the resource requirement data of each LLM component; Determining available computing power and available memory capacity corresponding to the cloud node based on resource data corresponding to the cloud node; Determining the available computing power and available memory capacity corresponding to each edge node based on the resource data corresponding to each edge node; Based on the quantitative computing requirements and the quantitative memory requirements corresponding to each LLM component, the available computing power and the available memory capacity corresponding to the cloud node, and the available computing power and the available memory capacity corresponding to each edge node, a corresponding cloud node or edge node is determined for each LLM component.

8. A task processing device, characterized in that: Applied to a task processing system, the task processing system includes a cloud node and at least one edge node; the device includes: a component allocation module for allocating multiple LLM components to cloud nodes and / or edge nodes based on resource data of the cloud nodes and edge nodes and resource requirement data of multiple LLM components; the LLM components are used to process LLM tasks in the task processing model; The task processing module is used to detect whether multiple LLM tasks in a subsequent task batch are received during the process of each LLM component processing multiple LLM tasks in a current task batch on the cloud node and / or edge node; The task interruption module is used to determine whether it is necessary to interrupt the processing of the current task batch based on the value density of each LLM task in the two adjacent task batches if it is determined that the next task batch has been received; if so, interrupt the processing of the current task batch, and reallocate LLM components on cloud nodes and / or edge nodes based on the computing requirements of the next task batch, so as to process multiple LLM tasks in the next task batch on cloud nodes and / or edge nodes based on the reallocated LLM components.

9. A task processing system, characterized in that: The task processing system includes a main control module, a cloud node and at least one edge node; The main control module is used to allocate multiple LLM components to cloud nodes and / or edge nodes based on the resource data of the cloud nodes and each edge node, and the resource demand data of multiple LLM components; the LLM components are used to process LLM tasks in the task processing model; During the process of processing multiple LLM tasks in a current task batch on the cloud node and / or edge node, each LLM component detects whether multiple LLM tasks in a subsequent task batch are received; If it is determined that the next task batch is received, it is determined whether the processing of the current task batch needs to be interrupted based on the value density of each LLM task in the two adjacent task batches; If yes, interrupt the processing of the current task batch, and reallocate the LLM components on the cloud nodes and / or edge nodes based on the computing requirements of the next task batch, so as to process multiple LLM tasks in the next task batch on the cloud nodes and / or edge nodes based on the reallocated LLM components; The cloud node is used to run the assigned LLM components and process the assigned LLM tasks; The edge node is used to run the assigned LLM components and process the assigned LLM tasks.

10. An electronic device comprising at least one processor and at least one memory, wherein: The memory stores a computer program, wherein when the computer program is executed by the processor, the task processing method according to any one of claims 1 to 7 is implemented.

Images