A large-scale computing power allocation and scheduling system for edge computing

Abstract

The present invention provides a large-model computing power allocation and scheduling system for edge computing, which relates to the field of electrical digital data processing, and includes an edge monitoring module, an edge computing module, an edge management module and a central allocation module. The edge monitoring module is used to monitor the usage status of edge devices, the edge computing module processes tasks based on computing power resources, the edge management module is used to control and manage the operation of edge devices, and the central allocation module is used to control the allocation of computing power resources and task data. The system can allocate computing power resources according to the computing power usage status of edge nodes, so that the tasks of large models can be processed more quickly, thereby improving the operation efficiency of large models.

Description

Technical Field

[0001] The present invention relates to the field of electrical digital data processing, and in particular to a large-model computing power allocation and scheduling system for edge computing. Background Art

[0002] In the large model inference stage, since the inference process usually requires strong computing resources and low latency, edge computing can reduce the computing scale of the model by pruning and quantizing the model. However, the computing power resources of edge nodes are relatively small, and task accumulation cannot be avoided when processing tasks. Therefore, it is necessary to allocate computing power and schedule tasks among edge nodes to improve the operating efficiency of the entire large model.

[0003] The foregoing discussion of the background art is intended only to facilitate an understanding of the present invention. This discussion does not acknowledge or admit that any of the material referred to is part of the common general knowledge.

[0004] Now many computing power allocation systems have been developed. After a lot of searching and reference, it is found that the existing computing power allocation systems are such as the system disclosed in the publication number CN115718664B. These system methods generally include: sampling subsystem, evaluation subsystem, allocation scheduling subsystem and visualization subsystem. First, the sampling subsystem samples data from the edge computing platform to obtain data information; secondly, the evaluation subsystem evaluates the data information to obtain the capacity allocation percentage; then, the allocation scheduling subsystem allocates computing power to all mobile edge devices of the edge computing platform based on the capacity allocation percentage to obtain the scheduling result; finally, the visualization subsystem visualizes the scheduling result. However, this system is only based on the capacity of the edge device for scheduling, rather than the real-time operating status. The computing power allocation is not flexible enough and cannot effectively improve the operating efficiency when processing large model tasks. Summary of the invention

[0005] The purpose of the present invention is to propose a large-model computing power allocation and scheduling system for edge computing in response to the existing deficiencies.

[0006] The present invention adopts the following technical solution:

[0007] A large-model computing power allocation and scheduling system for edge computing, including an edge monitoring module, an edge computing module, an edge management module and a central allocation module;

[0008] The edge monitoring module is used to monitor the usage status of the edge device, the edge computing module processes the task based on the computing resources, the edge management module is used to control and manage the operation of the edge device, and the central allocation module is used to control the allocation of computing resources and task data;

[0009] The edge monitoring module includes a resource management unit, a data monitoring unit and a data reporting unit. The resource management unit is used to record the computing resource information of the edge device, the data monitoring unit is used to perform real-time detection of the resource usage data of the device, and the data reporting unit is used to report the monitored information to the central allocation module;

[0010] The edge computing module includes a local computing unit, an external computing unit and an allocation execution unit, wherein the local computing unit is used to process local computing tasks, the external computing unit is used to process allocated external computing tasks, and the allocation execution unit is used to allocate the use rights of computing resources to corresponding tasks based on allocation instructions;

[0011] The edge management module includes a task management unit, a data management unit and a resource warning unit. The task management unit is used to manage the tasks generated by the large model, the data management unit is used to manage the data required for the tasks, and the resource warning unit is used to warn of insufficient resources.

[0012] The central allocation module includes a monitoring data storage unit, an edge resource allocation unit and a task data scheduling unit. The monitoring data storage unit is used to store the monitoring data reported by all edge nodes, the edge resource allocation unit is used to allocate computing resources to the scheduling tasks, and the task data scheduling unit is used to schedule the task data to the corresponding edge nodes;

[0013] Further, the data monitoring unit includes a monitoring application processor, a real-time detection processor and a load calculation processor, wherein the monitoring application processor is used to obtain path information of computing resources, the real-time detection processor performs real-time detection of computing resources based on the path information, and the load calculation processor is used to calculate the load information of the edge node;

[0014] The load calculation processor calculates the load value L of the edge node according to the following formula:

[0015] ;

[0016] Among them, C is the computing power currently occupied by the edge node, C max is the maximum computing power of the edge node, M is the current memory usage of the edge node, and M max is the maximum memory capacity of the edge node, is the relative adjustment coefficient of computing power and memory;

[0017] Further, the resource warning unit includes a warning detection processor and a scheduling judgment processor, the warning detection processor is used to receive the load value and judge whether it is in a warning state, and the scheduling judgment processor is used to judge whether to perform task transfer in the warning state;

[0018] After receiving the warning signal, the scheduling judgment processor determines whether to perform task transfer according to the following inequality:

[0019] ;

[0020] Among them, V d is the minimum amount of resources required to run the task, V a is the total local resource amount, T0 is the waiting time required for local operation, T1 is the time to complete the task locally, T is the estimated time to transfer the task, and T2 is the time to complete the task on the external edge node;

[0021] When all inequalities meet the conditions, the corresponding tasks are transferred;

[0022] Further, the edge resource allocation unit includes a task transfer processor, a node selection processor and a resource configuration processor, the task transfer processor is used to receive the task information transferred by the edge node, the node selection processor selects the edge node to be deployed based on the task information and the real-time monitoring data of each edge node, and the resource configuration processor generates a corresponding task configuration data packet based on the selected edge node and sends it to the corresponding edge node;

[0023] Furthermore, the process of the node selection processor selecting an edge node for a transfer task includes the following steps:

[0024] S1. Filter out the edge nodes whose loads meet the requirements of running the transfer task, which are called target nodes.

[0025] S2. Calculate the allocation index P of the target node according to the following formula:

[0026] ;

[0027] Where D is the quantized distance between the source node and the target node. is the trade-off coefficient;

[0028] S3. Select the target node with the smallest deployment index as the edge node for transit task deployment.

[0029] The beneficial effects achieved by the present invention are:

[0030] This system monitors edge nodes, and the monitoring results can be used to determine whether to schedule a task and to which edge node to schedule the task. When making scheduling decisions, the time consumption before and after scheduling is compared based on actual conditions, rather than blindly scheduling when computing power is tight. This can effectively avoid negative optimization. After deciding on task scheduling, the optimal edge node can be selected for scheduling by performing quantitative calculations on edge nodes, effectively improving the efficiency of task operation in large models.

[0031] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and description and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the overall structural framework of the present invention;

[0033] Figure 2 This is a schematic diagram of the edge monitoring module of the present invention;

[0034] Figure 3 This is a schematic diagram of the edge computing module of the present invention;

[0035] Figure 4 This is a schematic diagram of the edge management module of the present invention;

[0036] Figure 5 This is a schematic diagram of the central distribution module of the present invention;

[0037] Figure 6 This is a comparison diagram of the running time of the large model of the present invention. DETAILED DESCRIPTION

[0038] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual sizes. It is stated in advance. The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention.

[0039] Embodiment 1: This embodiment provides a large model computing power allocation and scheduling system for edge computing, combined with Figure 1 , including edge monitoring module, edge computing module, edge management module and central distribution module;

[0040] The edge monitoring module is used to monitor the usage status of the edge device, the edge computing module processes the task based on the computing resources, the edge management module is used to control and manage the operation of the edge device, and the central allocation module is used to control the allocation of computing resources and task data;

[0041] The edge monitoring module includes a resource management unit, a data monitoring unit and a data reporting unit. The resource management unit is used to record the computing resource information of the edge device, the data monitoring unit is used to perform real-time detection of the resource usage data of the device, and the data reporting unit is used to report the monitored information to the central allocation module;

[0042] The edge computing module includes a local computing unit, an external computing unit and an allocation execution unit, wherein the local computing unit is used to process local computing tasks, the external computing unit is used to process allocated external computing tasks, and the allocation execution unit is used to allocate the use rights of computing resources to corresponding tasks based on allocation instructions;

[0043] The edge management module includes a task management unit, a data management unit and a resource warning unit. The task management unit is used to manage the tasks generated by the large model, the data management unit is used to manage the data required for the tasks, and the resource warning unit is used to warn of insufficient resources.

[0044] The central allocation module includes a monitoring data storage unit, an edge resource allocation unit and a task data scheduling unit. The monitoring data storage unit is used to store the monitoring data reported by all edge nodes, the edge resource allocation unit is used to allocate computing resources to the scheduling tasks, and the task data scheduling unit is used to schedule the task data to the corresponding edge nodes;

[0045] The data monitoring unit includes a monitoring application processor, a real-time detection processor and a load calculation processor, wherein the monitoring application processor is used to obtain path information of computing resources, the real-time detection processor performs real-time detection of computing resources based on the path information, and the load calculation processor is used to calculate the load information of the edge node;

[0046] The load calculation processor calculates the load value L of the edge node according to the following formula:

[0047] ;

[0048] Among them, C is the computing power currently occupied by the edge node, C max is the maximum computing power of the edge node, M is the current memory usage of the edge node, and M max is the maximum memory capacity of the edge node, is the relative adjustment coefficient of computing power and memory;

[0049] The resource warning unit includes a warning detection processor and a scheduling judgment processor, wherein the warning detection processor is used to receive the load value and judge whether it is in a warning state, and the scheduling judgment processor is used to judge whether to perform task transfer in the warning state;

[0050] After receiving the warning signal, the scheduling judgment processor determines whether to perform task transfer according to the following inequality:

[0051] ;

[0052] Among them, V d is the minimum amount of resources required to run the task, V a is the total local resource amount, T0 is the waiting time required for local operation, T1 is the time to complete the task locally, T is the estimated time to transfer the task, and T2 is the time to complete the task on the external edge node;

[0053] When all inequalities meet the conditions, the corresponding tasks are transferred;

[0054] The edge resource allocation unit includes a task transfer processor, a node selection processor and a resource configuration processor, wherein the task transfer processor is used to receive the task information transferred by the edge node, the node selection processor selects the edge node to be deployed based on the task information and the real-time monitoring data of each edge node, and the resource configuration processor generates a corresponding task configuration data packet based on the selected edge node and sends it to the corresponding edge node;

[0055] The process of the node selection processor selecting an edge node for a transfer task includes the following steps:

[0056] S1. Filter out the edge nodes whose loads meet the requirements of running the transfer task, which are called target nodes.

[0057] S2. Calculate the allocation index P of the target node according to the following formula:

[0058] ;

[0059] Where D is the quantized distance between the source node and the target node. is the trade-off coefficient;

[0060] S3. Select the target node with the smallest deployment index as the edge node for transit task deployment.

[0061] Embodiment 2: This embodiment includes all the contents of embodiment 1, and provides a large-model computing power allocation and scheduling system for edge computing, including an edge monitoring module, an edge computing module, an edge management module, and a central allocation module;

[0062] The edge monitoring module is used to monitor the usage status of the edge device, the edge computing module processes the task based on the computing resources, the edge management module is used to control and manage the operation of the edge device, and the central allocation module is used to control the allocation of computing resources and task data;

[0063] Combination Figure 2 The edge monitoring module includes a resource management unit, a data monitoring unit and a data reporting unit. The resource management unit is used to record the computing resource information of the edge device, the data monitoring unit is used to perform real-time detection of the resource usage data of the device, and the data reporting unit is used to report the monitored information to the central allocation module;

[0064] Combination Figure 3 , the edge computing module includes a local computing unit, an external computing unit and an allocation execution unit, the local computing unit is used to process local computing tasks, the external computing unit is used to process allocated external computing tasks, and the allocation execution unit is used to allocate the use rights of computing resources to corresponding tasks based on allocation instructions;

[0065] Combination Figure 4 The edge management module includes a task management unit, a data management unit and a resource warning unit. The task management unit is used to manage the tasks generated by the large model, the data management unit is used to manage the data required for the tasks, and the resource warning unit is used to warn of insufficient resources;

[0066] Combination Figure 5 The central allocation module includes a monitoring data storage unit, an edge resource allocation unit and a task data scheduling unit. The monitoring data storage unit is used to store the monitoring data reported by all edge nodes, the edge resource allocation unit is used to allocate computing resources to the scheduling tasks, and the task data scheduling unit is used to schedule the task data to the corresponding edge nodes;

[0067] The resource management unit includes a resource path register and a path delivery processor, wherein the resource path register is used to store the path information of the computing power resources, and the path delivery processor is used to receive the monitoring application and provide the corresponding path information;

[0068] The data monitoring unit includes a monitoring application processor, a real-time detection processor and a load calculation processor, wherein the monitoring application processor is used to obtain path information of computing resources, the real-time detection processor performs real-time detection of computing resources based on the path information, and the load calculation processor is used to calculate the load information of the edge node;

[0069] The load calculation processor calculates the load value L of the edge node according to the following formula:

[0070] ;

[0071] Among them, C is the computing power currently occupied by the edge node, C max is the maximum computing power of the edge node, M is the current memory usage of the edge node, and M max is the maximum memory capacity of the edge node, is the relative adjustment coefficient of computing power and memory;

[0072] The data reporting unit includes a load editing processor, a central upload processor and an instant transmission processor, wherein the load editing processor is used to edit the load value to generate a monitoring data packet, the central upload processor is used to upload the monitoring data packet to the central distribution module, and the instant transmission processor is used to directly send the load value to the resource warning unit;

[0073] The local computing unit includes a channel configuration processor and a computing core processor, wherein the channel configuration processor is used to establish an exclusive channel between computing resources and tasks, and the computing core processor processes tasks by using computing resources;

[0074] The external computing unit includes a permission verification processor, an external configuration processor and an external computing processor, wherein the permission verification processor is used to verify the resource usage permission of the external task, the external configuration processor is used to establish an exclusive channel between the computing power resources and the external task, and the external computing processor processes the external task by using the computing power resources;

[0075] The allocation execution unit includes an instruction interpretation processor and a permission configuration processor, wherein the instruction interpretation processor is used to receive and interpret the allocation instruction, and the permission configuration processor grants the use permission of the computing power resource to the corresponding external task based on the interpretation result;

[0076] The task management unit includes a task receiving processor, a task encoding processor and a task queue processor, wherein the task receiving processor is used to receive task information assigned by the large model, the task encoding processor encodes the task based on the task information, and the task queue processor is used to generate a task queue to process the task dispatching process;

[0077] The data management unit includes a data acquisition processor, a data conversion processor and a data injection processor. The data acquisition processor acquires corresponding data based on task information. The data conversion processor is used to convert the acquired data into data directly used by the task. The data injection processor is used to inject the converted data into the corresponding task.

[0078] The resource warning unit includes a warning detection processor and a scheduling judgment processor, wherein the warning detection processor is used to receive the load value and judge whether it is in a warning state, and the scheduling judgment processor is used to judge whether to transfer the task in the warning state, that is, whether the new task in the task queue is to be scheduled to the external edge node for processing;

[0079] The early warning detection processor is provided with an early warning value. When the load value exceeds the early warning value, it is in an early warning state, and the early warning detection processor sends out an early warning signal;

[0080] After receiving the warning signal, the scheduling judgment processor determines whether to perform task transfer according to the following inequality:

[0081] ;

[0082] Among them, V d is the minimum amount of resources required to run the task, V a is the total local resource amount, T0 is the waiting time required for local operation, T1 is the time to complete the task locally, T is the estimated time to transfer the task, and T2 is the time to complete the task on the external edge node;

[0083] When all inequalities meet the conditions, the corresponding tasks are transferred;

[0084] The edge resource allocation unit includes a task transfer processor, a node selection processor and a resource configuration processor, wherein the task transfer processor is used to receive the task information transferred by the edge node, the node selection processor selects the edge node to be deployed based on the task information and the real-time monitoring data of each edge node, and the resource configuration processor generates a corresponding task configuration data packet based on the selected edge node and sends it to the corresponding edge node;

[0085] The process of the node selection processor selecting an edge node for a transfer task includes the following steps:

[0086] S1. Filter out the edge nodes whose loads meet the requirements of running the transfer task, which are called target nodes.

[0087] S2. Calculate the allocation index P of the target node according to the following formula:

[0088] ;

[0089] Where D is the quantized distance between the source node and the target node. is the trade-off coefficient;

[0090] S3, select the target node with the smallest allocation index as the edge node for transit task allocation;

[0091] The task data scheduling unit includes a data transfer processor and a deployment execution processor, wherein the data transfer processor is used to receive the task data transferred by the edge node, and the deployment execution processor sends the task data to the corresponding edge node based on the deployment information of the task;

[0092] The difference between task information and task data is that task information is general content, while task data is content used to perform tasks;

[0093] The following is some code information:

[0094] class CentralAllocationModule:

[0095] def __init__(self):

[0096] self.monitor_data = {}

[0097] def store_monitor_data(self, node_id, monitor_data):

[0098] self.monitor_data[node_id] = monitor_data

[0099] def allocate_resources(self, task_id, node_id):

[0100] # Allocate resources to a task

[0101] pass

[0102] def schedule_task_data(self, task_data, node_id):

[0103] # Schedule task data to the specific edge node

[0104] pass

[0105] class NodeSelector:

[0106] def __init__(self):

[0107] self.target_nodes = []

[0108] def filter_target_nodes(self, nodes, load_threshold):

[0109] self.target_nodes = [node for node in nodes if node.load <load_threshold]

[0110] return self.target_nodes

[0111] def calculate_allocation_index(self, node_distance, weight):

[0112] return node_distance + weight

[0113] def select_best_node(self, nodes):

[0114] min_index = float('inf')

[0115] best_node = None

[0116] for node in nodes:

[0117] allocation_index = self.calculate_allocation_index(node.distance, node.weight)

[0118] if allocation_index < min_index:

[0119] min_index = allocation_index

[0120] best_node = node

[0121] return best_node

[0122] class TaskDataDispatcher:

[0123] def __init__(self):

[0124] pass

[0125] def dispatch_task_data(self, task_data, node):

[0126] # Dispatch the task data to the chosen node

[0127] Pass

[0128] By running the big data multiple times for comparison, we can get the following Figure 6 's effect diagram.

[0129] We now use 50 groups of samples to conduct experiments, and measure the processing time of the common large model and the optimized model for the samples, and obtain the following results: Figure 6 Comparison effect diagram.

[0130] The contents disclosed above are only preferred feasible embodiments of the present invention, and do not limit the protection scope of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention specification and drawings are included in the protection scope of the present invention. In addition, the elements therein can be updated as technology develops.

Claims

1. A large-model computing power allocation and scheduling system for edge computing, characterized in that: It includes edge monitoring module, edge computing module, edge management module and central distribution module; The edge monitoring module is used to monitor the usage status of the edge device, the edge computing module processes the task based on the computing resources, the edge management module is used to control and manage the operation of the edge device, and the central allocation module is used to control the allocation of computing resources and task data; The edge monitoring module includes a resource management unit, a data monitoring unit and a data reporting unit. The resource management unit is used to record the computing resource information of the edge device, the data monitoring unit is used to perform real-time detection of the resource usage data of the device, and the data reporting unit is used to report the monitored information to the central allocation module; The edge computing module includes a local computing unit, an external computing unit and an allocation execution unit, wherein the local computing unit is used to process local computing tasks, the external computing unit is used to process allocated external computing tasks, and the allocation execution unit is used to allocate the use rights of computing resources to corresponding tasks based on allocation instructions; The edge management module includes a task management unit, a data management unit and a resource warning unit. The task management unit is used to manage the tasks generated by the large model, the data management unit is used to manage the data required for the tasks, and the resource warning unit is used to warn of insufficient resources. The central allocation module includes a monitoring data storage unit, an edge resource allocation unit and a task data scheduling unit. The monitoring data storage unit is used to store the monitoring data reported by all edge nodes, the edge resource allocation unit is used to allocate computing resources to the scheduling tasks, and the task data scheduling unit is used to schedule the task data to the corresponding edge nodes; The data monitoring unit includes a monitoring application processor, a real-time detection processor and a load calculation processor, wherein the monitoring application processor is used to obtain path information of computing resources, the real-time detection processor performs real-time detection of computing resources based on the path information, and the load calculation processor is used to calculate the load information of the edge node; The load calculation processor calculates the load value L of the edge node according to the following formula: ； Among them, C is the computing power currently occupied by the edge node, C max is the maximum computing power of the edge node, M is the current memory usage of the edge node, and M max is the maximum memory capacity of the edge node, is the relative adjustment coefficient of computing power and memory; The data reporting unit includes a load editing processor, a central upload processor and an instant transmission processor, wherein the load editing processor is used to edit the load value to generate a monitoring data packet, the central upload processor is used to upload the monitoring data packet to the central distribution module, and the instant transmission processor is used to directly send the load value to the resource warning unit; The resource warning unit includes a warning detection processor and a scheduling judgment processor, wherein the warning detection processor is used to receive the load value and judge whether it is in a warning state, and the scheduling judgment processor is used to judge whether to perform task transfer in the warning state; After receiving the warning signal, the scheduling judgment processor determines whether to perform task transfer according to the following inequality: ； Among them, V d is the minimum amount of resources required to run the task, V a is the total local resource amount, T0 is the waiting time required for local operation, T1 is the time to complete the task locally, T is the estimated time to transfer the task, and T2 is the time to complete the task on the external edge node; When all inequalities meet the conditions, the corresponding tasks are transferred; The edge resource allocation unit includes a task transfer processor, a node selection processor and a resource configuration processor, wherein the task transfer processor is used to receive the task information transferred by the edge node, the node selection processor selects the edge node to be deployed based on the task information and the real-time monitoring data of each edge node, and the resource configuration processor generates a corresponding task configuration data packet based on the selected edge node and sends it to the corresponding edge node; The process of the node selection processor selecting an edge node for a transfer task includes the following steps: S1. Filter out the edge nodes whose loads meet the requirements of running the transfer task, which are called target nodes. S2. Calculate the allocation index P of the target node according to the following formula: ； Where D is the quantized distance between the source node and the target node. is the trade-off coefficient; S3. Select the target node with the smallest deployment index as the edge node for transit task deployment.

Images