Edge computing scheduling system based on knowledge graph
By using knowledge graphs to build a dynamic scheduling system in edge computing, the problem of insufficient flexibility of traditional scheduling methods in complex environments is solved, and more efficient resource utilization and performance optimization is achieved.
Patent Information
- Application Number
- CN202411236873.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-03
AI Technical Summary
Traditional edge computing scheduling methods cannot effectively deal with complex real-time scenarios and heterogeneous networks, resulting in inflexible task allocation and scheduling in unstable network connections and device heterogeneous environments, and performance is not optimized.
Adopting an edge computing scheduling system based on knowledge graphs, the edge server computing resource characterization module, scheduling module and visualization module can collect and analyze computing resource data in real time, and build a dynamic knowledge graph to realize accurate description of edge server resources and task scheduling. , optimize node utilization and computing performance.
It improves scheduling efficiency and accuracy, shortens system failure response time, supports concurrent scheduling of complex task topology, and improves the utilization rate and computing performance of edge computing nodes.
Smart Images

Figure CN119065852A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of edge computing scheduling, and in particular to an edge computing scheduling system based on a knowledge graph. Background Art
[0002] Edge computing refers to placing computing resources and application services on devices or nodes at the edge of the network to accelerate data processing and reduce network latency. With the development of technologies such as the Internet of Things and 5G, edge computing is increasingly being used in various scenarios, such as smart homes, smart cities, and autonomous driving. However, edge computing faces some challenges, such as unstable network connections, heterogeneity between different devices, and complex application scenarios, which bring great difficulties to edge computing scheduling. Traditional scheduling methods are usually based on static rules or heuristic algorithms and cannot adapt to complex real-time scenarios. Knowledge graph is a graphical model that represents knowledge. It can capture the relationship between entities and the connection between entity attributes. The edge computing scheduling system based on knowledge graph can use the characteristics of knowledge graph to describe the various entities and their attributes involved in edge computing, and perform real-time decision inference through the inference engine, thereby achieving efficient resource scheduling and task allocation.
[0003] In existing technologies, the common edge computing scheduling methods are mainly based on static rules or heuristic algorithms, which cannot flexibly cope with complex real-time scenarios. Although some scheduling systems may utilize the idea of distributed computing, they lack a deep understanding of heterogeneous networks and entity relationships.
[0004] General traditional method technical solution description: Task submission: Users submit computing tasks to the edge computing system.
[0005] Task Scheduling: Using static rules or heuristic algorithms, the system selects appropriate edge computing nodes to perform tasks based on predefined rules and policies. This may be based on information such as node load, distance, and performance.
[0006] Task execution: The selected edge computing nodes execute the assigned tasks.
[0007] Result return: After execution is completed, the result is returned to the user or system.
[0008] However, these traditional methods have some defects and cannot cope with the complex heterogeneous networks, real-time requirements, and deep understanding of entity relationships in edge computing.
[0009] The disadvantages of the prior art are: Since traditional methods use static rules or heuristic algorithms for task scheduling, the system is not adaptable enough to the changing network conditions and task loads in edge computing. Static rules lead to insufficient flexibility. In a dynamic environment, since the rules cannot be adjusted in real time, the system's task allocation and scheduling are not flexible enough, which may lead to non-optimal performance.
[0010] In addition, because traditional methods usually do not have a deep understanding of the entity relationships in edge computing, they are only based on information such as node load and distance, and cannot fully consider the complex relationships between nodes. Without a deep understanding of entity relationships, in scenarios where edge computing involves multiple nodes working together, traditional methods may not be able to fully tap the potential of entity relationships, resulting in limited scheduling effects. Summary of the invention
[0011] The purpose of this invention is to improve the flexibility and intelligence of edge computing scheduling.
[0012] The technical problems to be solved by the present invention are: The present invention aims to solve the key problem in the field of edge computing scheduling, that is, the dilemma that traditional scheduling methods cannot meet real-time requirements when facing the challenges of unstable network connections, device heterogeneity and complex application scenarios. By introducing an edge computing scheduling system based on knowledge graphs, the invention combines visualization technology with innovatively designed computing resource representation and task scheduling models to achieve accurate description of edge server resources, real-time task scheduling and system status visualization, thereby improving the utilization rate and computing performance of edge computing nodes.
[0013] Technical solution of the present invention: An edge computing scheduling system based on knowledge graph includes: an edge server computing power resource representation module, an edge server scheduling module, and an edge server visualization module.
[0014] in: The edge server computing resource representation module includes resource collection and resource modeling functions. This module is responsible for collecting computing resource data of edge servers, extracting relevant features through entity recognition technology, and building a knowledge graph to provide effective scheduling basis and data support for the edge server scheduling module and edge server visualization module.
[0015] The edge server scheduling module includes task decomposition, task matching, task scheduling, and node management functions. It can associate the computing resources and computing tasks of the edge server, and match nodes and schedule scheduling in the knowledge graph.
[0016] The edge server visualization module includes two parts: computing resource visualization and computing task visualization, which are used to display the characteristic attributes and task load of edge computing nodes.
[0017] The present invention analyzes and decomposes the tasks, and expresses the characteristics of edge computing nodes based on the knowledge graph, and matches the appropriate edge computing nodes according to the topological structure of the tasks; by characterizing the computing resources of the edge server, the potential connections between features of different dimensions are explored, thereby realizing computing task matching and scheduling, and achieving the purpose of optimizing the utilization of edge computing nodes and improving computing performance.
[0018] Beneficial Effects It is reflected in the following aspects: Scheduling efficiency: Existing scheduling algorithms are generally based on predefined rules or strategies, which are not flexible enough and difficult to cope with dynamically changing environments. This invention combines dynamic knowledge graphs for scheduling optimization, which enables the scheduling algorithm to adapt to changes in real time, improves scheduling efficiency and accuracy, and improves scheduling efficiency by about 10%-30%.
[0019] System failure response: The existing technology responds slowly to system failures, resulting in task loss or the need to reschedule. Thanks to the computing power data and status reported in real time by the proxy program, the present invention can detect anomalies in a timely manner and automatically reschedule, shortening the system failure response time by more than 30%.
[0020] Visualization: The visualization in the prior art is usually a static chart that can only provide limited information and has poor interactivity; the present invention provides a real-time interactive interface that allows users to dynamically view and manipulate data, thereby better understanding and analyzing the system status.
[0021] In terms of task topology: the existing technology usually only supports sequential scheduling and cannot effectively handle complex task topologies, which limits the scheduling capability of the system; the present invention can achieve concurrent scheduling by analyzing task topology. This method supports more complex task structures and improves the flexibility and efficiency of scheduling. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a system architecture diagram of the present invention; Figure 2 A resource representation workflow diagram for the present invention; Figure 3 It is a system workflow diagram of the present invention; Figure 4 This is a schematic diagram of the knowledge graph representation of edge server resources in Example 1 of the present invention; Figure 5 This is a task topology diagram after the task set of Example 1 of the present invention is disassembled. DETAILED DESCRIPTION
[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0024] like Figure 1 As shown, an edge computing scheduling system based on knowledge graph includes: an edge server computing power resource characterization module, an edge server scheduling module, and an edge server visualization module.
[0025] in: The edge server computing resource representation module includes resource collection and resource modeling functions. This module is responsible for collecting computing resource data of edge servers, extracting relevant features through entity recognition technology, and building a knowledge graph to provide effective scheduling basis and data support for the edge server scheduling module and edge server visualization module.
[0026] The edge server scheduling module includes task decomposition, task matching, task scheduling, and node management functions. It can associate the computing resources and computing tasks of the edge server, and match nodes and schedule them in the graph.
[0027] The edge server visualization module includes two parts: computing resource visualization and computing task visualization, which are used to display the characteristic attributes and task load of edge computing nodes.
[0028] Each functional module is described in detail as follows: Edge server computing resource representation module The computing power resources of the edge server are collected through the agent program deployed on the edge server. The collected data will be used to build the knowledge graph; the graph data is dynamically updated according to the access status of the edge server nodes; and the scheduling basis and visualization data support are provided for the edge server scheduling module and the edge server visualization module.
[0029] The details are as follows: The edge server computing power resource characterization module includes two main steps: first, by deploying an agent program on each edge server, it continuously monitors and collects resource information such as computing power, memory usage, storage capacity and network bandwidth in real time, and then sends the data to the central system through the MQTT protocol; secondly, the scheduling system summarizes the received data and uses the BERT (Bidirectional Encoder Representations from Transformers) model (existing technology) in the Transformer technology to perform context modeling and key entity extraction of resource information, so as to extract the attribute information of computing power resources.
[0030] The edge server computing power resource characterization module adopts a relationship definition method based on expert knowledge to complete the definition of edge node relationships: The representation of edge node relationships is defined as <one node> →<with relation> →<other node> (1) Represents an edge node (<edge_node> ) and its affiliated computing center (<center_node> ) are related to each other: <edge_node> →<belongs_to> →<center_node> Indicates that the edge node belongs to the computing center.
[0031] The extracted node attributes are defined as: <edge_node> →<with attribute> → <attribute>(2) Represents an edge node (<edge_node> ) has a property ( <attribute>), and the relationship between the attribute and the edge node is given by<with attribute> Connection, specifically: <edge_node> →<with attribute> → <name> <edge_node>→<with attribute>→<device_id> <edge_node>→<with attribute>→<cpu_power> <edge_node>→<with attribute>→<cpu_usage> <edge_node>→<with attribute>→<mem_total> <edge_node>→<with attribute>→<mem_usage> <edge_node>→<with attribute>→<disk_total> <edge_node>→<with attribute>→<disk_usage> <edge_node>→<with attribute>→ <ip> <edge_node>→<with attribute>→ <online> <edge_node>→<with attribute>→ <bandwidth> Among them, name indicates the node name, device_id indicates the device ID, cpu_power indicates the CPU computing power, cpu_usage indicates the CPU usage, mem_total indicates the total memory, mem_usage indicates the memory usage, disk_total indicates the total disk, disk_usage indicates the disk usage, ip indicates the external IP address, online indicates whether the node is online, and bandwidth indicates the bandwidth.
[0032] The knowledge representation of edge node status is defined as: <status> → <stateful>→<status_expression> (2) Indicates the node status ( <status>) is stateful ( <stateful>), and the node status expression (<status_expression> ) and this state have a relationship: <online> → <stateful>→<edge_node.online = 1> <offline> → <stateful>→<edge_node.online = 0> They represent the online and offline states of the node respectively, and establish a connection between the online attribute of the node and the state.
[0033] The knowledge representation of edge node features is defined as: <feature> → <require>→<feature_expression> (3) Represents a node feature ( <feature>) needs to meet certain conditions ( <require>), and the condition is given by (<feature_expression> )express: <edge_node meet the task's cpu requirements> → <require>→<edge_node.cpu_power*(1-edge_node.cpu_usage)> =task's cpu requirements> Indicates that the node meets the CPU computing requirements of the task, requiring that the node's CPU computing capacity minus the used part (that is, CPU utilization) is greater than or equal to the computing requirements of the task.
[0034] <edge_node meet the task's mem requirements> → <require>→<edge_node.cpu_power*(1-edge_node.mem_usage)> = task's mem requirements> Indicates that the node meets the memory requirements of the task, requiring the available memory size of the node to be greater than or equal to the memory requirements of the task.
[0035] <edge_node meet the task's disk requirements> → <require>→<edge_node.cpu_power*(1-edge_node.disk_usage)> =task's disk requirements> Indicates that the node meets the disk requirements of the task, requiring the available disk space of the node to be greater than or equal to the disk requirements of the task.
[0036] <edge_node not in service> → <require>→<edge_node.online = 0> Indicates that the node is unavailable and requires the node's online status to be offline.
[0037] <edge_node in service> → <require>→<edge_node.online = 1> Indicates that the node is in available state and requires the node's online state to be online.
[0038] The knowledge representation of the scheduling operation is defined as: <operator> → <action>→<operator_expression> (4) Represents a scheduling operation ( <operator>) needs to perform a specific action ( <action>), and the action is performed by (<operator_expression> )express: <search edge_nodes> → <action>→<find nearby available edge_nodes> Represents searching for available edge nodes (<search edge_nodes> ) needs to perform an action to find nearby available nodes (<find nearby available edge_nodes> ), and returns it.<search edge_nodes> Represents the operation of searching for available edge nodes, based on conditions such as node characteristics and resource requirements.<find nearby availableedge_nodes> It indicates specific actions, such as searching for nearby available nodes based on node location information, and filtering available nodes based on node status and characteristics.
[0039] <issue task> → <action>→<issue image> <issue task data>→ <action>→<issue docker config> Represents the operation of issuing task images and container configuration files, and converting them into specific actions.
[0040] The weight representation of the scheduling operation is defined as: <operator_expression> →<with weight> → <weight>(5) Represents a scheduling operation (<operator_expression> ) The corresponding weight when it needs to be executed ( <weight>), and the weight is given by<with weight> express: <find nearby available edge_nodes> →<with weight> → <3> <issue image> →<with weight> → <2> <issue docker config> →<with weight> → <1> They represent the corresponding weights for operations such as finding nearby available nodes, sending task images, and container configuration files.
[0041] <find nearby available edge_nodes> ,<issue image> and<issue dockerconfig> Represents three different scheduling operations,<with weight> It indicates that a connection is established between the weight and the operation, that is, the weight value must be considered when the operation needs to be executed. <3> , <2> and <1> They represent the weight values of the three operations respectively. The larger the value, the higher the priority.<operator_expression> Can be any form of scheduling operation expression. <weight>It represents the weight value of the corresponding operation, which is usually used to measure factors such as the priority, performance and availability of tasks or nodes.<with weight> This indicates that a connection is established between the weight and the operation, that is, the weight value must be considered when the operation needs to be performed.
[0042] The execution representation of a scheduling operation is defined as: <operator_expression> → <call>→<external functon> (6) A scheduling operation (<operator_expression> ) needs to be executed when the external function is called (<external function> ), and the function is given by <call>express: <find nearby available edge_nodes> → <call>→<search functon> <issue image>→ <call>→<image issue functon> <issue docker config>→ <call>→<docker issue functon> They define external functions that are called when searching for nearby available nodes, sending task images, and container configuration files.<find nearby available edge_nodes> ,<issue image> and<issue docker config> Represents three different scheduling operations, <call>Indicates that a connection is established between the operation and the function, that is, the corresponding external function must be called when the operation needs to be executed.<edge_node search function> ,<image issue function> and<docker issue function> They represent the names of the external functions that need to be called.
[0043] Based on the above relationship definitions, the extracted entities are eventually mapped to the knowledge graph to establish relationships and dependencies between resources. This process ensures the comprehensive collection and accurate modeling of edge server resource data, providing reliable data support and decision-making basis for edge computing scheduling. Based on the universality and adaptability of the above model definitions, as time goes by and the access status of edge server nodes changes, the module will dynamically update the graph based on new data. This means that if a new edge server joins the system, its computing power resource data will be included in the graph, thereby maintaining the real-time and accuracy of the graph.
[0044] Edge server scheduling module The edge server scheduling module realizes efficient scheduling of computing tasks through steps such as task decomposition, task matching, task scheduling and node management. First, the module performs topological analysis on the computing tasks submitted by the user, identifies each subtask and its dependencies, refines the tasks and sorts them into the task queue in topological order. Then, the feature vector is obtained by computing the task mapping relationship, and the node feature information provided by the edge server computing power resource characterization module is used for matching. The best execution node is found by comprehensively considering factors such as the node's computing power, reliability, distance and network delay. Then, the decomposition and matching subtasks are arranged into a complete computing process, and the task image is deployed to the selected node to ensure that the tasks are executed in sequence. Finally, the module manages the node status by monitoring the health status and resource utilization of the node in real time to ensure that the tasks are assigned to available and suitable nodes, thereby optimizing the scheduling effect and resource utilization of the system.
[0045] The details are as follows: Edge server scheduling module, which includes task decomposition, task matching, task scheduling, and node management: Task decomposition: When a user submits a computing task, the edge server scheduling module first performs a topological analysis on the task. This step includes identifying the various subtasks in the task, clarifying the interdependencies between tasks, and breaking down the complex computing task into a series of subtasks. According to the order determined by the topological sorting, these subtasks are included in the task queue one by one for subsequent processing.
[0046] Task matching: After the task is decomposed, the scheduling module takes the computing task from the task queue. The scheduling module uses the edge node feature information provided by the edge server computing resource characterization module. By matching the computing task feature vector with the node feature vector, the module can find the best edge computing node in the knowledge graph to execute the task. The matching process comprehensively considers the attributes of the node, such as computing power, memory, reliability, network latency, etc., and uses a weighted scoring model to complete the matching. Where S represents the score of matching between tasks and edge nodes, P i Represents the edge node attributes, W i It represents the weight of the attribute in the matching task process. Its value is associated with the task characteristics. Finally, all nodes are sorted according to the matching score, and the node with the highest score is selected to perform the task.
[0047] Task orchestration: The subtasks that have been disassembled and matched will be orchestrated into a complete computing process. This step ensures that the tasks are run on the edge server nodes in the correct order and dependencies. During the orchestration process, the task image is deployed to the matching edge computing node to complete the task scheduling.
[0048] Node management: The status and availability of edge server nodes are monitored, including monitoring the node health and resource utilization information so that the available resources of the node can be considered when scheduling tasks. Node management helps ensure that tasks are assigned to reliable and suitable nodes.
[0049] Edge Server Visualization Module The edge server visualization module is used to design a knowledge graph-based visualization system based on the constructed orchestration and scheduling model to display the system workflow in order to achieve intelligent decision-making. The main functions of the visualization system include two parts: edge server computing power resource visualization and computing task scheduling visualization. (1) The edge server computing power resource visualization can display the feature information of the edge computing cluster. After the edge computing cluster is expanded, the attribute information of all edge server nodes in the cluster can be obtained. (2) The computing task scheduling visualization can display the task load status of the edge computing cluster in real time and display the workflow of computing task scheduling.
[0050] The details are as follows: The edge server visualization module is designed to show the system workflow to administrators through a visual interface to help support intelligent decision-making. This module mainly consists of two parts: edge server computing resource visualization and computing task scheduling visualization: 1. Edge server computing power resource visualization. This module is designed to present the computing power resource information of each edge server node in the edge computing cluster, such as processor performance, memory capacity, storage capacity, etc. defined in the edge server computing power resource characterization module, which helps to quickly understand the resource situation of each node. Through the visualization system, administrators can obtain the overall characteristic information of the edge computing cluster. When the administrator expands the cluster, detailed attribute information of each edge server node can be obtained. This visualization enables administrators to better evaluate the available resources.
[0051] 2. Visualization of computing task scheduling. This module is designed to display the task load status and task scheduling workflow of the edge computing cluster in real time. Administrators can use this visualization interface to track the execution status of tasks, view the tasks being executed, tasks waiting to be executed, and tasks that have been completed. This real-time visual feedback helps administrators understand the load of the cluster and promptly identify potential problems, such as resource bottlenecks or overloads, so that they can take necessary measures to optimize scheduling and resource allocation.
[0052] The working principle of the present invention is: The present invention analyzes, processes, selects and extracts features from the computing power resource data of the edge server, constructs a knowledge graph, and further processes the graph relationship; the system disassembles and analyzes the computing tasks submitted by the user, and adds the decomposed subtasks to the task queue according to the topological structure of the task for scheduling and scheduling; the system uses task images to carry computing tasks and data, and can migrate between different edge computing power nodes; the system takes tasks from the task queue, constructs feature vectors of computing tasks and edge nodes, queries in the knowledge graph to identify a subset of nodes that matches the task feature vector, and scores each candidate node by comprehensively considering the multi-dimensional attributes of the node and the task requirements, and finally selects the node with the highest score as the best execution point for the computing task, and sends the task image to the node for execution.
[0053] like Figure 2 As shown in the figure, the resource characterization workflow of the edge server computing power resource characterization module is as follows: S1. Computing power collection of edge server nodes: Agent deployment: An agent is deployed on each edge server to continuously monitor and record the resource usage of the edge server.
[0054] Resource collection: The agent collects resource information of the edge server in real time, including computing power, memory usage, storage capacity, network bandwidth, etc.
[0055] Resource reporting: After collecting resource information, the agent packages the data and sends it to the scheduling system through the MQTT protocol.
[0056] S2. Resource modeling of edge server nodes: Resource aggregation: After receiving resource information from all edge servers, the scheduling system aggregates the data.
[0057] Entity recognition: The BERT model in Transformer technology is used to process the aggregated resource information. The specific steps include: text encoding, using BERT's bidirectional encoder to model the context of resource information; entity extraction, identifying key entities (such as computing power, memory type, etc.) through the model's self-attention mechanism, and extracting corresponding features from the text.
[0058] Graph modeling: Based on the identified entities, graph modeling is performed to map the extracted entities into the knowledge graph, where the relationships between entities are modeled according to predefined rules. The modeled graph shows the associations and dependencies between resources.
[0059] Graph update and maintenance: The knowledge graph is updated regularly to reflect the latest resource status and relationships to ensure the accuracy and timeliness of the graph.
[0060] Through this process, comprehensive collection and accurate modeling of edge server resource data can be achieved, and reliable data support and decision-making basis can be provided for edge computing scheduling.
[0061] like Figure 3 , which is the workflow of the edge computing scheduling system of the present invention: In this embodiment, the user submits a computing task to the system. Once the system receives the task, the scheduling process starts. The following steps are included: S1. User submits task: The user submits computing tasks to the system, and the system enters the scheduling process after receiving the tasks.
[0062] S2. Task analysis and decomposition: After receiving the task, the edge server scheduling module analyzes and decomposes the task. The system divides the task into multiple subtasks according to its topological structure for better management and allocation. This decomposition can simplify complex tasks into smaller, independent task units.
[0063] S3. Subtasks are added to the task queue: The disassembled subtasks are added to the task queue, which is a list of tasks waiting to be processed. These subtasks will wait for subsequent task matching and scheduling.
[0064] S4. Regularly check the task queue: The system regularly checks the task queue to see if there are any unprocessed tasks waiting. If the queue is empty, the system may enter a dormant state and wait for new tasks to be added.
[0065] S5. Task matching: When there are pending subtasks in the task queue, the system will perform task matching. The knowledge graph is used to comprehensively consider the resource requirements of the subtasks and the computing resources of the edge server nodes, including computing power, reliability, distance, and network delay factors, and obtain the edge server node with the highest matching degree from the knowledge graph to determine the best execution node.
[0066] S6. Node matching and task delivery: If the match is successful, the system will assign the subtask to the best node and then deliver the task to the node, that is, the subtask will be executed on the selected edge server.
[0067] S7, Task execution or priority adjustment: The subtask starts to execute on the node. If the system does not find a suitable node in the task matching phase, the subtask will wait. In this case, the system adjusts the priority of the task and uses the knowledge graph to update the node status information so that the next round of matching can be carried out when the appropriate computing power conditions are met in the future.
[0068] S8, loop execution: The process is executed in a loop until all subtasks are assigned and completed. If a new task is submitted, the whole process will start again.
[0069] This process describes how the edge server scheduling module processes tasks submitted by users. Task decomposition, task matching, and task scheduling are the core steps of this module, ensuring that tasks are efficiently assigned to appropriate edge server nodes to maximize resource utilization and task execution efficiency.
[0070] Example 1 Construct a complex computing task set with obvious topological structure and schedule it using the scheduling system of the present invention. The decomposition and scheduling of the task set can optimize resource allocation through knowledge graph and achieve efficient concurrent execution.
[0071] Edge server resources: Server A: 6 TFLOPS computing power, 32 GB memory, 256 GB storage, 10 Gbps bandwidth Server B: 15 TFLOPS of computing power, 8 GB of memory, 256 GB of storage, 20 Gbps of bandwidth Server C: 6 TFLOPS of computing power, 20 GB of memory, 512 GB of storage, 5 Gbps of bandwidth Compute a set of tasks (tasks have a topological structure): Task 1: Computing requirement 8 TFLOPS, memory requirement 4 GB, storage requirement 200 GB (parent task) Task 2: Computing requirement 4 TFLOPS, memory requirement 24 GB, storage requirement 200 GB (subtask, dependent on Task 1) Task 3: Computing requirement 4 TFLOPS, memory requirement 4 GB, storage requirement 500 GB (subtask, dependent on Task 1) Task 4: Computing requirement 4 TFLOPS, memory requirement 10 GB, storage requirement 200 GB (subtask, dependent on Task 2) Resource collection: Deploy agents on edge servers A, B, and C to collect computing power, memory, storage, bandwidth and other resource data in real time, and transmit them to the scheduling system through the MQTT protocol Knowledge graph construction: Data processing: The collected resource data is processed using the BERT model to identify computing power, memory, storage, and bandwidth as entities.
[0072] Graph construction: Based on resource data and predefined relationships, a knowledge graph is constructed, including the attribute characteristics of entities and the topological structure of resource nodes. The knowledge graph representation of edge server resources is as follows: Figure 4 As shown, the details are as follows: Server A: Server A → with attribute → cpu_power: 6 TFLOPS Server A → with attribute → mem_total: 32 GB Server A → with attribute → disk_total: 256 GB Server A → with attribute → bandwidth: 10 Gbps Server B: Server B → with attribute → cpu_power: 15 TFLOPS Server B → with attribute → mem_total: 8 GB Server B → with attribute → disk_total: 256 GB Server B → with attribute → bandwidth: 20 Gbps Server C: Server C → with attribute → cpu_power: 6 TFLOPS Server C → with attribute → mem_total: 20 GB Server C → with attribute → disk_total: 512 GB Server C → with attribute → bandwidth: 5 Gbps Real-time update: New resource data is collected from edge servers in real time, and the knowledge graph is dynamically updated to ensure that the latest resource status and relationships are reflected in the graph.
[0073] Task topology analysis: The task topology after the calculation task set is decomposed is as follows Figure 5 As shown, Task 1 is the parent task; Task 2 (subtask) depends on Task 1; Task 3 (subtask) depends on Task 1; Task 4 (subtask) depends on Task 2.
[0074] Task matching: According to the resource requirements of each task in the task set, attribute weights are assigned (for example, the computing requirement of task 1 is the largest among all tasks, so the computing power weight of this task is also the largest), as shown in the following table According to the resource requirements of each edge node in the node set in the knowledge graph, the attribute value is assigned (for example, the computing requirement of server B is the largest among all tasks, so the attribute value of the computing power of this node is also the largest), as shown in the following table When matching, select the edge nodes that meet the conditions, and perform weighted cumulative matching based on the attribute weights of each task and the corresponding attribute values of the edge nodes to select the edge node with the largest matching value, as follows: Task 1: Only server B meets the computing requirements, so it is assigned to server B with the strongest computing power for processing.
[0075] Task 2: Both Server A and Server C meet the conditions. The matching score of Server A is 1*1+4*3+1*1=14, and the score of Server C is 1*1+4*2+1*3=12, that is, Server A will get a higher score, so it will be assigned to Server A with larger memory for processing.
[0076] Task 3: Only server C meets the storage requirement, so C is assigned to server C with the strongest storage capacity for processing.
[0077] Task 4: Both Server A and Server C can meet the conditions, but they need to rely on Task 2 to execute, not Task 3. Therefore, if Task 3 is still running on Server C, then C will be matched after subtracting the resource requirements of Task 2. At this time, the conditions are no longer met, so Server A is directly matched. If Task 3 has been completed at this time, then matching is performed from Servers A and C, and the calculation method is the same as Task 2, that is, matching Server A.
[0078] Scheduling process: Resource matching: By obtaining real-time data from the knowledge graph, the system analyzes the resource requirements of each subtask and matches them with the resource status of the edge server, and selects the best target scheduling node (i.e., execute search edge_nodes → action → find nearby available edge_nodes from the knowledge graph); sends the task image to the selected edge node (i.e., execute issue task → action → issue image from the knowledge graph); sends the task's docker configuration file to the edge node (i.e., execute issue task data → action → issue docker config from the knowledge graph).
[0079] Dynamic adjustment: If a server has insufficient resources or crashes, the latest data provided by the knowledge graph is used to reschedule tasks. For example, if Task 3 crashes during execution, it is transferred to another server with sufficient resources to ensure smooth execution of the task.
[0080] Compared with the existing methods, the method of the present invention has the following advantages: Existing technology: 1. Resource monitoring: Scheduling systems usually use static data or regularly updated strategies, which makes it difficult to quickly reflect real-time changes in resource status.
[0081] 2. Task scheduling: Based on static resource information and predefined strategies, it is not flexible enough and difficult to cope with dynamically changing environments.
[0082] 3. Visualization: Visualization is usually a static chart that can only provide limited information and has poor interactivity.
[0083] The technology of the present invention: 1. Real-time resource collection: Resource data is collected in real time through the agent program, and the MQTT protocol is used for data transmission to reflect the latest resource status and ensure the real-time and accuracy of scheduling decisions.
[0084] 2. Accurate modeling: Use the BERT model for entity recognition and context modeling to accurately map resource information into the knowledge graph.
[0085] 3. Dynamic scheduling: Combine dynamic knowledge graphs for task scheduling. Through knowledge graphs, computing tasks and edge computing resources are matched, and more factors are considered to optimize resource allocation.
[0086] 4. It has good support for the topology of tasks. By disassembling tasks and analyzing their topological structure, it can schedule and execute multiple subtasks concurrently, support more complex task structures, and improve scheduling flexibility and efficiency.
[0087] 5. Advanced visualization: Use D3.js and Gin framework to provide a real-time, interactive visualization interface to enhance the management convenience of the system.
[0088] Prior art vs. the present invention The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any slight modification, equivalent substitution and improvement made to the above embodiment based on the technical essence of the present invention should be included in the protection scope of the technical solution of the present invention.< / call> < / call> < / call> < / call> < / call> < / call> < / weight> < / weight> < / weight> < / action> < / action> < / action> < / action> < / operator> < / action> < / operator> < / require> < / require> < / require> < / require> < / require> < / require> < / feature> < / require> < / feature> < / stateful> < / offline> < / stateful> < / online> < / stateful> < / status> < / stateful> < / status> < / bandwidth> < / online> < / ip> < / name> < / attribute> < / attribute>
Claims
1. An edge computing scheduling system based on knowledge graph, characterized in that: include: Edge server computing power resource representation module, edge server scheduling module, edge server visualization module; in: The edge server computing resource representation module includes resource collection and resource modeling functions. This module is responsible for collecting computing resource data of edge servers, extracting relevant features through entity recognition technology, and building a knowledge graph to provide effective scheduling basis and data support for the edge server scheduling module and edge server visualization module; The edge server scheduling module includes task decomposition, task matching, task scheduling, and node management functions. It can associate the computing resources and computing tasks of the edge server, and match nodes and schedule scheduling in the knowledge graph. The edge server visualization module includes two parts: computing resource visualization and computing task visualization, which are used to display the characteristic attributes and task load of edge computing nodes.
2. The edge computing scheduling system as claimed in claim 1, characterized in that: The edge server computing resource representation module completes the computing resource collection of the edge server through the agent program deployed on the edge server. The collected data will be used to build the knowledge graph; dynamically update the graph data according to the access status of the edge server node; provide scheduling basis and visualization data support for the edge server scheduling module and edge server visualization module; the details are as follows: The edge server computing power resource characterization module includes two main steps: first, by deploying an agent program on each edge server, it continuously monitors and collects resource information such as computing power, memory usage, storage capacity and network bandwidth in real time, and then sends the data to the central system through the MQTT protocol; secondly, the scheduling system summarizes the received data and uses the BERT model in the Transformer technology to perform context modeling and key entity extraction of resource information, so as to extract the attribute information of computing power resources.
3. The edge computing scheduling system as claimed in claim 1, characterized in that: The edge server scheduling module achieves efficient scheduling of computing tasks through steps such as task decomposition, task matching, task scheduling and node management. First, the module performs topological analysis on the computing tasks submitted by the user, identifies each subtask and its dependencies, refines the tasks and sorts them into the task queue in topological order; then, the feature vector is obtained by computing the task mapping relationship, and the node feature information provided by the edge server computing resource characterization module is used for matching, and the best execution node is found by comprehensively considering factors such as the node's computing power, reliability, distance and network delay; then, the subtasks after decomposition and matching are arranged into a complete computing process, and the task images and containers are deployed to the selected nodes to ensure that the tasks are executed in sequence; finally, the module manages the node status by real-time monitoring of the node's health status and resource utilization to ensure that the tasks are assigned to available and suitable nodes, optimizing the system's scheduling effect and resource utilization.
4. The edge computing scheduling system as claimed in claim 3, characterized in that: The edge server scheduling module includes task decomposition, task matching, task scheduling, and node management: Task decomposition: When a user submits a computing task, the edge server scheduling module first performs a topological analysis on the task. This step includes identifying the various subtasks in the task, clarifying the interdependencies between tasks, and breaking down the complex computing task into a series of subtasks. According to the order determined by the topological sorting, these subtasks are included in the task queue one by one for subsequent processing; Task matching: After task decomposition, the scheduling module takes out computing tasks from the task queue. The scheduling module uses the edge node feature information provided by the edge server computing resource characterization module. By matching the computing task feature vector with the node feature vector, the module can find the best edge computing node in the knowledge graph to execute the task. The matching process comprehensively considers the attributes of the node and uses a weighted scoring model to complete the matching, as follows: Where S represents the score of matching between tasks and edge nodes, P i Represents the edge node attributes, W i Represents the weight of this attribute in the matching task process. Its value is associated with the task characteristics. Finally, all nodes are sorted according to the matching score, and the node with the highest score is selected to perform the task; Task orchestration: The decomposed and matched subtasks will be orchestrated into a complete computing process. This step ensures that the tasks are run on the edge server nodes in the correct order and dependencies. During the orchestration process, the task image is deployed to the matching edge computing node to complete the task scheduling. Node management: The status and availability of edge server nodes will be detected, including monitoring the health status and resource utilization information of the nodes so that the available resources of the nodes can be considered when scheduling tasks; Node management helps ensure that tasks are assigned to reliable and suitable nodes.
5. The edge computing scheduling system according to claim 1, characterized in that: The edge server visualization module is used to design a visualization system based on the knowledge graph according to the constructed orchestration scheduling model to display the system workflow in order to achieve intelligent decision-making; the main functions of the visualization system include two parts: visualization of edge server computing resources and visualization of computing task scheduling; (1) The edge server computing resource visualization can display the characteristic information of the edge computing cluster. After the edge computing cluster is expanded, the attribute information of all edge server nodes in the cluster can be obtained; (2) The computing task scheduling visualization can display the task load status of the edge computing cluster in real time and display the workflow of computing task scheduling; The details are as follows: The edge server visualization module is designed to display the system workflow to administrators through a visual interface to help support intelligent decision-making; the module mainly consists of two parts: edge server computing resource visualization and computing task scheduling visualization: 1) Edge server computing power resource visualization: This module is designed to present the computing power resource information of each edge server node in the edge computing cluster, such as processor performance, memory capacity, storage capacity, etc. defined in the edge server computing power resource characterization module, which helps to quickly understand the resource situation of each node. Through the visualization system, administrators can obtain the overall characteristic information of the edge computing cluster. When the administrator expands the cluster, detailed attribute information of each edge server node can be obtained; this visualization enables administrators to better evaluate the available resources; 2) Computing task scheduling visualization: This module aims to display the task load status and task scheduling workflow of the edge computing cluster in real time. Administrators can use this visualization interface to track the execution status of tasks, view the tasks being executed, tasks waiting to be executed, and tasks that have been completed. This real-time visualization feedback helps administrators understand the load of the cluster and promptly identify potential problems, such as resource bottlenecks or overloads, so that they can take necessary measures to optimize scheduling and resource allocation.
6. The edge computing scheduling system as claimed in claim 1, characterized in that: The resource characterization workflow of the edge server computing power resource characterization module is as follows: S1. Computing power collection of edge server nodes: Agent deployment: An agent is deployed on each edge server to continuously monitor and record the resource usage of the edge server; Resource collection: The agent collects resource information of edge servers in real time, including computing power, memory usage, storage capacity, network bandwidth, etc. Resource reporting: After collecting resource information, the agent packages the data and sends it to the scheduling system through the MQTT protocol; S2. Resource modeling of edge server nodes: Resource aggregation: After receiving resource information from all edge servers, the scheduling system aggregates the data; Entity recognition: The BERT model in Transformer technology is used to process the aggregated resource information. The specific steps include: text encoding, using BERT's bidirectional encoder to model the context of resource information; entity extraction, identifying key entities through the model's self-attention mechanism and extracting corresponding features from the text; Graph modeling: Based on the identified entities, graph modeling is performed to map the extracted entities into the knowledge graph, where the relationships between entities are modeled according to predefined rules. The modeled graph shows the associations and dependencies between resources. Graph update and maintenance: Regularly update the knowledge graph to reflect the latest resource status and relationships to ensure the accuracy and timeliness of the graph; Through this process, comprehensive collection and accurate modeling of edge server resource data can be achieved, and reliable data support and decision-making basis can be provided for edge computing scheduling.
7. The edge computing scheduling system as claimed in claim 1, characterized in that: The edge computing scheduling system workflow includes the following steps: S1. User submits task: The user submits the computing task to the system, and the system enters the scheduling process after receiving the task; S2. Task analysis and decomposition: After receiving the task, the edge server scheduling module analyzes and decomposes the task; the system divides the task into multiple subtasks according to its topological structure for better management and allocation. This decomposition can simplify complex tasks into smaller, independent task units; S3, subtasks are added to the task queue: The disassembled subtasks are added to the task queue, which is a list of tasks waiting to be processed. These subtasks will wait for subsequent task matching and scheduling; S4. Regularly check the task queue: The system regularly checks the task queue to see if there are any unprocessed tasks waiting; if the queue is empty, the system may enter a dormant state and wait for new tasks to be added; S5, Task Matching: When there are pending subtasks in the task queue, the system will perform task matching; the knowledge graph is used to comprehensively consider the resource requirements of the subtasks and the computing resources of the edge server nodes, including computing power, reliability, distance, and network delay factors, and obtain the edge server node with the highest matching degree from the knowledge graph to determine the best execution node; S6, node matching and task delivery: If the match is successful, the system will assign the subtask to the best node and then deliver the task to the node, that is, the subtask will be executed on the selected edge server; S7, Task execution or priority adjustment: The subtask starts to execute on the node. If the system does not find a suitable node in the task matching phase, the subtask will wait; in this case, the system adjusts the priority of the task and uses the knowledge graph to update the node status information so that the next round of matching can be carried out when the appropriate computing power conditions are met in the future; S8. Loop execution: The process is executed in a loop until all subtasks are assigned and completed; if a new task is submitted, the whole process will start again.
8. The edge computing scheduling system as claimed in claim 1, characterized in that: The edge server computing power resource characterization module adopts a relationship definition method based on expert knowledge to complete the definition of edge node relationships: The representation of edge node relationships is defined as <one node> →<with relation> →<other node> (1) Represents an edge node (<edge_node> ) and its affiliated computing center (<center_node> ) are related to each other: <edge_node> →<belongs_to> →<center_node> Indicates that the edge node belongs to the computing center; The extracted node attributes are defined as: <edge_node> → <with attribute> → <attribute> (2)< / attribute> Represents an edge node (<edge_node> ) has a property ( <attribute> ), and the relationship between the attribute and the edge node is given by<with attribute> Connection, specifically:< / attribute> <edge_node> → <with attribute> → <name>< / name> <edge_node> →<with attribute> →<device_id> <edge_node> →<with attribute> →<cpu_power> <edge_node> →<with attribute> →<cpu_usage> <edge_node> →<with attribute> →<mem_total> <edge_node> →<with attribute> →<mem_usage> <edge_node> →<with attribute> →<disk_total> <edge_node> →<with attribute> →<disk_usage> <edge_node> → <with attribute> → <ip>< / ip> <edge_node> → <with attribute> → <online>< / online> <edge_node> → <with attribute> → <bandwidth>< / bandwidth> Among them, name indicates the node name, device_id indicates the device ID, cpu_power indicates the CPU computing power, cpu_usage indicates the CPU usage, mem_total indicates the total memory, mem_usage indicates the memory usage, disk_total indicates the total disk, disk_usage indicates the disk usage, ip indicates the external IP address, online indicates whether the node is online, and bandwidth indicates the bandwidth; The knowledge representation of edge node status is defined as: <status> → <stateful> →<status_expression> (2)< / stateful> < / status> Indicates the node status ( <status>) is stateful ( <stateful> ), and the node status expression (<status_expression> ) and this state have a relationship:< / stateful> < / status> <online> → <stateful> →<edge_node.online = 1>< / stateful> < / online> <offline> → <stateful> →<edge_node.online = 0>< / stateful> < / offline> They represent the online and offline states of the node respectively, and establish a connection between the online attribute of the node and the state; The knowledge representation of edge node features is defined as: <feature> → <require> →<feature_expression> (3)< / require> < / feature> Represents a node feature ( <feature>) needs to meet certain conditions ( <require> ), and the condition is given by (<feature_expression> )express:< / require> < / feature> <edge_node meet the task's cpu requirements> → <require> →<edge_node.cpu_power*(1-edge_node.cpu_usage) > =task's cpu requirements>< / require> Indicates that the node meets the CPU computing requirements of the task, which requires that the node's CPU computing capacity minus the used part, that is, the CPU utilization rate, is greater than or equal to the computing requirements of the task; <edge_node meet the task's mem requirements> → <require> →<edge_node.cpu_power*(1-edge_node.mem_usage) > = task's mem requirements>< / require> Indicates that the node meets the memory requirement of the task, requiring the available memory size of the node to be greater than or equal to the memory requirement of the task; <edge_node meet the task's disk requirements> → <require> →<edge_node.cpu_power*(1-edge_node.disk_usage) > =task's disk requirements>< / require> Indicates that the node meets the disk requirements of the task, requiring the available disk space of the node to be greater than or equal to the disk requirements of the task; <edge_node not in service> → <require> →<edge_node.online = 0>< / require> Indicates that the node is unavailable and requires the node's online status to be offline; <edge_node in service> → <require> →<edge_node.online = 1>< / require> Indicates that the node is in available state, requiring the node's online state to be online; The knowledge representation of the scheduling operation is defined as: <operator> → <action> →<operator_expression> (4)< / action> < / operator> Represents a scheduling operation ( <operator>) needs to perform a specific action ( <action> ), and the action is performed by (<operator_expression> )express:< / action> < / operator> <search edge_nodes> → <action>→<find nearby available edge_nodes>< / action> Represents searching for available edge nodes (<search edge_nodes> ) needs to perform an action to find nearby available nodes (<find nearby available edge_nodes> ), and return it;<search edge_nodes> Represents the operation of searching for available edge nodes, based on conditions such as node characteristics and resource requirements;<find nearby availableedge_nodes> It indicates specific actions, such as searching for nearby available nodes based on node location information, or filtering available nodes based on node status and features; <issue task> → <action> →<issue image>< / action> <issue task data> → <action> →<issue docker config>< / action> Represents the operation of issuing task images and container configuration files, and converts them into specific actions; The weight representation of the scheduling operation is defined as: <operator_expression> → <with weight> → <weight> (5)< / weight> Represents a scheduling operation (<operator_expression> ) The corresponding weight when it needs to be executed ( <weight> ), and the weight is given by<with weight> express:< / weight> <find nearby available edge_nodes> →<with weight> → <3> <issue image> →<with weight> → <2> <issue docker config> →<with weight> → <1> They represent the weights corresponding to operations such as searching for nearby available nodes, sending task images, and container configuration files. <find nearby available edge_nodes>,<issue image> and<issue docker config> Represents three different scheduling operations,<with weight> It means that a connection is established between the weight and the operation, that is, the weight value must be considered when the operation needs to be executed; <3> , <2> and <1> They represent the weight values of the three operations respectively. The larger the value, the higher the priority. In this representation,<operator_expression> Can be any form of scheduling operation expression, <weight> It represents the weight value of the corresponding operation.<with weight> This means that a connection is established between the weight and the operation, that is, the weight value must be considered when the operation needs to be executed;< / weight> The execution representation of a scheduling operation is defined as: <operator_expression> → <call> →<external functon> (6)< / call> A scheduling operation (<operator_expression> ) needs to be executed when the external function is called ( <externalfunction>), and the function is given by <call> express:< / call> < / externalfunction> <find nearby available edge_nodes> → <call> →<search functon>< / call> <issue image> → <call> →<image issue functon>< / call> <issue docker config> → <call> →<docker issue functon>< / call> They respectively define external functions that are called when searching for nearby available nodes, sending task images, and container configuration files. In this representation,<find nearby available edge_nodes> ,<issue image> and<issue docker config> Represents three different scheduling operations, <call> Indicates that a connection is established between the operation and the function, that is, the corresponding external function must be called when the operation needs to be executed;<edge_node search function> ,<image issue function> and<docker issue function> They represent the names of the external functions that need to be called;< / call> Based on the above relationship definitions, the extracted entities are eventually mapped to the knowledge graph to establish relationships and dependencies between resources. This process ensures the comprehensive collection and accurate modeling of edge server resource data, providing reliable data support and decision-making basis for edge computing scheduling; based on the universality and adaptability of the above model definition, as time goes by and the access status of edge server nodes changes, the module will dynamically update the graph based on new data; this means that if a new edge server joins the system, its computing power resource data will be included in the graph, thereby maintaining the real-time and accuracy of the graph.
Citation Information
Cited By
Computing power scheduling method and system of ARM cluster server
CN119248519A
Large language model optimization method and system for edge device
CN119918622A
A Large Language Model Optimization Method and System for Edge Devices
CN119918622B
Method and system for cross-domain management and intelligent allocation of computing power resources
CN120315881A
Smart community metadata interaction method and system based on edge computing framework
CN120803756A
