A task scheduling system and method

Abstract

The embodiments of the present application provide a task scheduling system, method, electronic device and storage medium, wherein the scheduling system comprises at least a central cloud and a plurality of edge nodes communicatively connected to the central cloud, and each edge node comprises a plurality of execution units; wherein the central cloud is used to obtain task configuration information corresponding to the target data stream in response to streaming information for the target data stream, and generate a data processing task for the streaming information according to the task configuration information; obtain node operating parameters corresponding to each edge node, and select a first edge node that meets a first preset condition corresponding to the data processing task from each edge node according to the node operating parameters; the first edge node is used to receive the data processing task, and obtain the unit operating parameters of each execution unit, and select a first execution unit that meets a second preset condition from each execution unit according to the unit operating parameters; the first execution unit is used to execute the data processing task for the target data stream.

Description

Technical Field

[0001] The present application relates to the field of edge computing technology, and in particular to a task scheduling system and a task scheduling method. Background Art

[0002] Videos and pictures have become the main carriers of data content and the main means of information dissemination due to their strong information carrying capacity. 5G's large bandwidth, low latency, and wide connection characteristics have activated cloud video monitoring, cloud gaming, the Internet of Things and other scene applications. The extension from consumer Internet to industrial Internet has further promoted the explosion of terminal applications and view data. Among them, relying on the infrastructure of the edge cloud, through view computing control and scheduling, it provides video monitoring equipment with nearby access, audio and video processing, video file transfer and other functions, which can provide users with more cost-effective product solutions. However, since the streaming media processing service is deployed on the edge node, while reducing costs, it also introduces the instability of the edge node into the data processing process, which can easily lead to large bandwidth consumption and high costs during the execution of data tasks, as well as serious network fluctuations and instability due to long links. Summary of the invention

[0003] The embodiments of the present application provide a task scheduling system and method to solve or partially solve the problems of large bandwidth consumption, high cost and unstable transmission in the execution process of tasks based on edge computing.

[0004] The embodiment of the present application discloses a task scheduling system, the scheduling system at least includes a central cloud and a plurality of edge nodes connected to the central cloud in communication, each of the edge nodes includes a plurality of execution units; wherein:

[0005] The central cloud is used to respond to the push stream information for the target data stream, obtain the task configuration information corresponding to the target data stream, and generate a data processing task for the push stream information according to the task configuration information; obtain the node operation parameters corresponding to each of the edge nodes, and select a first edge node that meets the first preset condition corresponding to the data processing task from each of the edge nodes according to the node operation parameters;

[0006] The first edge node is used to receive the data processing task, obtain unit operation parameters of each of the execution units, and select a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters;

[0007] The first execution unit is used to execute the data processing task for the target data stream.

[0008] Optionally, the central cloud includes at least several service layers and task control services connected to the service layers; wherein,

[0009] The service layer is used to obtain task configuration information corresponding to the target data stream in response to the streaming information of the target data stream, and generate a data processing task for the streaming information according to the task configuration information;

[0010] The task management service is used to obtain the node operating parameters corresponding to each of the edge nodes, and select a first edge node that meets the first preset conditions corresponding to the data processing task from each of the edge nodes according to the node operating parameters, and send the data processing task to the first edge node.

[0011] Optionally, the first edge node includes at least a node management service and a plurality of execution unit management services connected to the node management service, each of the execution unit management services corresponds to one execution unit; wherein,

[0012] The execution unit management and control service is used to obtain unit operation parameters of the execution unit to which it belongs, and send the unit operation parameters to the node management and control service;

[0013] The node management and control service is used to respond to receiving the data processing task and select a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters of each of the execution units, and send the data processing task to a first execution unit management and control service corresponding to the first execution unit;

[0014] The first execution unit is used to obtain the data processing service sent by the first execution unit management service and execute the data processing service corresponding to the target data flow.

[0015] Optionally, the first edge node further includes an edge cache layer; wherein,

[0016] The node management and control service is further used to obtain the task execution result of the data processing task by the first execution unit and the reporting information corresponding to the task execution result if it is detected that the communication connection with the central cloud is disconnected;

[0017] The edge cache layer is used to store the task execution result and the reporting information.

[0018] Optionally, the node management and control service is further configured to send a synchronization instruction to the edge cache layer if it is detected that the communication connection with the central cloud is restored;

[0019] The edge cache layer is also used to synchronize the task execution result and the reporting information to the task management and control service of the central cloud.

[0020] Optionally, the task management service is used to obtain the node heartbeat of the first edge node; if the node heartbeat of the first edge node times out, select a second edge node that meets the first preset condition according to the node operation parameters corresponding to other edge nodes except the first edge node; and migrate the data processing task to the second edge node;

[0021] The second edge node is used to receive the data processing task, obtain unit operation parameters of each of the execution units, and select a second execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters;

[0022] The second execution unit is used to execute the data processing task for the target data stream.

[0023] Optionally, the node management and control service is further used to obtain the unit heartbeat of the first execution unit; if the unit heartbeat of the first execution unit times out, select a third execution unit that meets the second preset condition according to the node operation parameters corresponding to other execution units except the first execution unit; and send the data processing task to the second execution unit management and control service corresponding to the third execution unit;

[0024] The third execution unit is used to obtain the data processing service sent by the second execution unit management service, and execute the data processing service corresponding to the target data flow.

[0025] Optionally, the node management and control service is further configured to send a task migration request to the task management and control service of the central cloud if the node operation parameters of each execution unit to which it belongs do not meet the second preset condition;

[0026] The task management and control service is used to select a second edge node that meets the first preset condition according to node operating parameters corresponding to other edge nodes except the first edge node; and migrate the data processing task to the second edge node.

[0027] Optionally, the central cloud is also used to obtain bandwidth cost information corresponding to each of the edge nodes, and to extract a first operating parameter associated with the data processing task from the node operating parameters corresponding to the edge nodes; calculate the node score corresponding to the edge node based on the bandwidth cost information and the first operating parameter, and use the edge node with the highest node score as the first edge node to execute the data processing task.

[0028] Optionally, the first edge node is also used to extract a second operating parameter associated with the data processing task from the unit operating parameters corresponding to each of the execution units; calculate the unit score corresponding to the execution unit according to the second operating parameter, and use the execution unit with the highest unit score as the first execution unit to execute the data processing task.

[0029] The present application also provides a method for scheduling a task, including:

[0030] In response to the push stream information for the target data stream, acquiring task configuration information corresponding to the target data stream, and generating a data processing task for the push stream information according to the task configuration information;

[0031] Node operating parameters corresponding to each edge node are obtained, and a first edge node that meets a first preset condition corresponding to the data processing task is selected from the edge nodes according to the node operating parameters, wherein the first edge node is used to execute the data processing task for the target data flow.

[0032] Optionally, selecting, from the edge nodes according to the node operation parameter, a first edge node that meets a first preset condition corresponding to the data processing task includes:

[0033] Obtaining bandwidth cost information corresponding to each of the edge nodes;

[0034] Extracting a first operating parameter associated with the data processing task from the node operating parameters corresponding to the edge node;

[0035] A node score corresponding to the edge node is calculated according to the bandwidth cost information and the first operating parameter, and the edge node with the highest node score is used as the first edge node for executing the data processing task.

[0036] Optionally, the method further comprises:

[0037] Obtaining the synchronization instruction sent by the first edge node and the task execution result and reporting information corresponding to the data processing task;

[0038] In response to the synchronization instruction, the task execution result and the reporting information are synchronized.

[0039] Optionally, the method further comprises:

[0040] Obtaining the node heartbeat of the first edge node;

[0041] If the node heartbeat of the first edge node times out, a second edge node that meets the first preset condition is selected based on the node operating parameters corresponding to other edge nodes except the first edge node, and the data processing task is migrated to the second edge node. The second edge node is used to execute the data processing task for the target data flow.

[0042] The present application also provides a method for scheduling a task, including:

[0043] Obtain node operation parameters and send the node operation parameters to the central cloud, the central cloud is used to respond to the push stream information for the target data stream, obtain the task configuration information corresponding to the target data stream, and generate a data processing task for the push stream information according to the task configuration information;

[0044] Receiving the data processing task and obtaining unit operation parameters of each execution unit;

[0045] A first execution unit that meets a second preset condition is selected from each of the execution units according to the unit operation parameter, and the first execution unit is used to execute the data processing task for the target data stream.

[0046] Optionally, selecting a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameter includes:

[0047] Extracting a second operating parameter associated with the data processing task from the unit operating parameters corresponding to each of the execution units;

[0048] The unit score corresponding to the execution unit is calculated according to the second operating parameter, and the execution unit with the highest unit score is used as the first execution unit for executing the data processing task.

[0049] Optionally, it also includes:

[0050] If it is detected that the communication connection with the central cloud is disconnected, obtaining the task execution result of the data processing task by the first execution unit and the reporting information corresponding to the task execution result;

[0051] The task execution result and the reporting information are stored in a preset edge cache layer.

[0052] Optionally, it also includes:

[0053] If it is detected that the communication connection with the central cloud is restored, a synchronization instruction is sent to the edge cache layer, and the edge cache layer is used to respond to the synchronization instruction and synchronize the task execution result and the reporting information to the central cloud.

[0054] Optionally, it also includes:

[0055] Obtaining a unit heartbeat of the first execution unit;

[0056] If the unit heartbeat of the first execution unit times out, a third execution unit that meets the second preset condition is selected based on the node operating parameters corresponding to other execution units except the first execution unit, and the data processing task is sent to the third execution unit. The third execution unit is used to execute the data processing task for the target data stream.

[0057] Optionally, it also includes:

[0058] If the node operating parameters of each of the execution units do not meet the second preset condition, a task migration request is sent to the central cloud. The central cloud is used to select a second edge node that meets the first preset condition based on the node operating parameters corresponding to other edge nodes except the first edge node, and migrate the data processing task to the second edge node.

[0059] The embodiment of the present application further provides a task scheduling method, which is applied to a scheduling system, wherein the scheduling system includes a central cloud and a plurality of edge nodes that are communicatively connected to the central cloud, each of the edge nodes includes a plurality of execution units, and the method includes:

[0060] The central cloud obtains task configuration information corresponding to the target data stream in response to the streaming information of the target data stream, and generates a data processing task for the streaming information according to the task configuration information;

[0061] The central cloud obtains node operating parameters corresponding to each edge node, and selects a first edge node that meets a first preset condition corresponding to the data processing task from the edge nodes according to the node operating parameters;

[0062] The first edge node obtains unit operation parameters of each of the execution units, and selects a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters;

[0063] The first execution unit executes the data processing task for the target data stream.

[0064] Optionally, the central cloud includes at least several service layers and a task control service connected to the service layer; the central cloud obtains task configuration information corresponding to the target data stream in response to the push stream information of the target data stream, and generates a data processing task for the push stream information according to the task configuration information, including:

[0065] The service layer obtains task configuration information corresponding to the target data stream in response to the streaming information of the target data stream, and generates a data processing task for the streaming information according to the task configuration information;

[0066] The task management service obtains node operating parameters corresponding to each of the edge nodes, and selects a first edge node that meets a first preset condition corresponding to the data processing task from each of the edge nodes according to the node operating parameters, and sends the data processing task to the first edge node.

[0067] Optionally, the first edge node includes at least a node management service and a plurality of execution unit management services connected to the node management service, each of the execution unit management services corresponding to one of the execution units; the first edge node obtains unit operation parameters of each of the execution units, and selects a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters, including:

[0068] The execution unit management and control service obtains the unit operation parameters of the execution unit to which it belongs, and sends the unit operation parameters to the node management and control service;

[0069] The node management and control service, in response to receiving the data processing task, selects a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters of each of the execution units, and sends the data processing task to a first execution unit management and control service corresponding to the first execution unit;

[0070] The first execution unit executes the data processing task for the target data stream, including:

[0071] The first execution unit obtains the data processing service sent by the first execution unit management service, and executes the data processing service corresponding to the target data flow.

[0072] Optionally, the first edge node further includes an edge cache layer, and the method further includes:

[0073] If the node management and control service detects that the communication connection with the central cloud is disconnected, then obtaining the task execution result of the data processing task by the first execution unit and the reporting information corresponding to the task execution result;

[0074] The task execution result and the reporting information are stored in the edge cache layer.

[0075] Optionally, also include

[0076] If the node management and control service detects that the communication connection with the central cloud is restored, it sends a synchronization instruction to the edge cache layer;

[0077] The edge cache layer synchronizes the task execution result and the reporting information to the task management and control service of the central cloud.

[0078] Optionally, it also includes:

[0079] The task management service obtains the node heartbeat of the first edge node; if the node heartbeat of the first edge node times out, selects a second edge node that meets the first preset condition according to the node operation parameters corresponding to other edge nodes except the first edge node; and migrates the data processing task to the second edge node;

[0080] The second edge node receives the data processing task, obtains unit operation parameters of each of the execution units, and selects a third execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters;

[0081] The first execution unit executes the data processing task for the target data stream.

[0082] Optionally, it also includes:

[0083] The node management and control service obtains the unit heartbeat of the first execution unit; if the unit heartbeat of the first execution unit times out, selects a third execution unit that meets the second preset condition according to the node operation parameters corresponding to other execution units except the first execution unit; and sends the data processing task to the second execution unit management and control service corresponding to the third execution unit;

[0084] The third execution unit obtains the data processing service sent by the second execution unit management service, and executes the data processing service corresponding to the target data flow.

[0085] Optionally, it also includes:

[0086] If the node control service detects that the node operating parameters of each execution unit to which it belongs do not meet the second preset condition, it sends a task migration request to the task control service of the central cloud;

[0087] The task management service selects a second edge node that meets the first preset condition according to node operating parameters corresponding to other edge nodes except the first edge node; and migrates the data processing task to the second edge node.

[0088] Optionally, selecting, from the edge nodes according to the node operation parameter, a first edge node that meets a first preset condition corresponding to the data processing task includes:

[0089] Obtaining bandwidth cost information corresponding to each of the edge nodes;

[0090] Extracting a first operating parameter associated with the data processing task from the node operating parameters corresponding to the edge node;

[0091] The node score corresponding to the edge node is calculated according to the bandwidth cost information and the first operating parameter, and the edge node with the highest node score is used as the first edge node for executing the data processing task.

[0092] Optionally, selecting a first execution unit that meets a second preset condition from the various execution units according to the unit operation parameters includes:

[0093] Extracting a second operating parameter associated with the data processing task from the unit operating parameters corresponding to each of the execution units;

[0094] The unit score corresponding to the execution unit is calculated according to the second operating parameter, and the execution unit with the highest unit score is used as the first execution unit for executing the data processing task.

[0095] The embodiments of the present application include the following advantages:

[0096] In an embodiment of the present application, for a scheduling system, it may include a central cloud and several edge nodes that are communicatively connected to the central cloud, each edge node may include several execution units, and in the process of scheduling tasks, the central cloud may respond to the push information for the target data stream, obtain the task configuration information corresponding to the target data stream, and generate a data processing task for the push information according to the task configuration information; obtain the node operating parameters corresponding to each edge node, and select a first edge node that meets the first preset condition corresponding to the data processing task from each edge node according to the node operating parameters, and select a suitable node for executing the data processing task through the node operating parameters of the edge node, which can not only ensure the effectiveness of task execution, but also execute the data processing task through a suitable edge node, which can ensure the balance of the edge node operation performance, and reduce bandwidth consumption and bandwidth cost, and, in the process of executing the data processing task through a suitable edge node, the edge node can also obtain the unit operating parameters of each execution unit, and select the first execution unit that meets the second preset condition from each execution unit according to the unit operating parameters, and execute the data processing task for the target data stream through the first execution unit, and through scheduling inside the edge node, the corresponding processing cost can be further reduced, and the efficiency of task execution can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] Figure 1 It is a structural block diagram of a task scheduling system provided in an embodiment of the present application;

[0098] Figure 2 It is a structural block diagram of the central cloud provided in the embodiment of the present application;

[0099] Figure 3 is a structural block diagram of an edge node provided in an embodiment of the present application;

[0100] Figure 4 is a structural block diagram of an edge node provided in an embodiment of the present application;

[0101] Figure 5 is a structural block diagram of a scheduling system provided in an embodiment of the present application;

[0102] Figure 6 It is a flowchart of a method for scheduling tasks provided in an embodiment of the present application;

[0103] Figure 7 It is a flowchart of a method for scheduling tasks provided in an embodiment of the present application;

[0104] Figure 8 It is a flowchart of a method for scheduling tasks provided in an embodiment of the present application;

[0105] Fig. 9 It is a schematic diagram of a network device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0106] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0107] As an example, in the view computing product solution, relying on the edge cloud infrastructure, view computing management and scheduling can provide video monitoring equipment with nearby access, audio and video processing, video file transfer and other functions, providing users with a more cost-effective product solution. However, since the streaming media processing service is deployed on the edge node, while reducing costs, it will also introduce the unstable hardware and software environment of the edge node into the scheduling system, affecting the stability of the system operation.

[0108] For example, taking live streaming media processing as an example, after the user terminal pushes the stream, the data stream can be forwarded to the central computer room through multiple physical computer rooms. The central computer room can be deployed with computing and processing capabilities such as video recording and transcoding. After transcoding in the central computer room, the corresponding transcoded data can be stored in the OSS cluster (Object Storage Service) of the central computer room. In this process, if the user terminal pushes the stream and forwards it to the central computer room through the CDN (Content Delivery Network) computer room, and then performs the stream pulling processing, it will undoubtedly lead to an increase in bandwidth costs. In addition, if the user terminal is directly connected to the central computer room, the network will be prone to fluctuations and instability due to the long network link.

[0109] In this regard, one of the core inventions of this application is to optimize the scheduling system of edge tasks and provide a high-availability and low-cost task scheduling method for cloud-edge collaboration. Specifically, for the scheduling system, it can include a central cloud and several edge nodes connected to the central cloud in communication. Each edge node can include several execution units. In the scheduling process of the task, the central cloud can select the node that is most suitable for the data processing task according to the node operation parameters of each edge node as the edge node for executing the data processing task, which can not only ensure the effectiveness of task execution, but also execute the data processing task through the appropriate edge node, which can ensure the balance of edge node operation performance and reduce bandwidth consumption and bandwidth cost. In addition, for the edge node that executes the task, it can further select the execution unit that is most suitable for the data processing task according to the unit operation parameters of each execution unit for executing the data processing task, so as to further reduce the corresponding processing cost and improve the efficiency of task execution. Furthermore, in the process of executing the task, when an exception occurs between the edge node or execution unit that executes the task, the internal migration of the edge node or the migration across the edge node can also be realized, which improves the high availability of the scheduling system and the stability of the edge service.

[0110] In order to help those skilled in the art better understand the technical solutions in the embodiments of the present application, some technical terms involved in the embodiments of the present application are explained and described below:

[0111] The central cloud can be a cloud computing platform built on the cloud computing data center infrastructure based on the capabilities of cloud computing technology. It can effectively store and manage huge amounts of data by summarizing and analyzing complex, large-scale global data processing tasks.

[0112] Edge cloud / edge cloud computing: referred to as edge cloud, is a cloud computing platform built on edge infrastructure based on the core of cloud computing technology and edge computing capabilities, forming an elastic cloud platform with comprehensive computing, network, storage, security and other capabilities at the edge, and forming an end-to-end technical architecture of "cloud-edge-end collaboration" with the central cloud and IoT terminals. By placing network forwarding, storage, computing, intelligent data analysis and other tasks at the edge for processing, it reduces response latency, relieves pressure on the central cloud, reduces bandwidth costs, and provides cloud services such as full-network scheduling and computing power distribution.

[0113] Edge nodes: The basic architecture of edge cloud is composed of centralized control by the center and multiple distributed edge nodes covering all locations of the entire network. It is a typical one-to-many control model. The forms of edge cloud nodes include but are not limited to: edge IDC (Internet data center), edge gateway, edge device, etc. It should be noted that in the embodiment of the present application, the edge node can be an edge unit that can be used for edge cloud computing based on edge IDC.

[0114] Data streams may include image streams, video streams, audio streams, etc. Different data streams may correspond to different task configuration information. For example, for video streams, they may correspond to screenshot configuration information, recording configuration information, etc. Different task configuration information may be used to configure different data processing tasks so that the edge node can perform corresponding data processing tasks, such as taking screenshots of video streams, recording video streams, etc.

[0115] The operating parameters may include node operating parameters and unit operating parameters. For the unit operating parameters, they may include the CPU, GPU, disk, memory, bandwidth and other usage rates of an execution unit. Since an edge node may include several execution units, the node operating parameters may be the sum of the unit operating parameters of all execution units, such as the sum of CPU usage, GPU usage, disk usage, memory usage, bandwidth usage, etc. The performance status of the edge node or execution unit may be detected through the operating parameters, so as to select the edge node or execution unit suitable for the data processing task according to the operating parameters.

[0116] Specifically, refer to Figure 1 , shows a structural block diagram of a task scheduling system provided in an embodiment of the present application, wherein the scheduling system comprises at least a central cloud and a plurality of edge nodes connected to the central cloud in communication, and each of the edge nodes comprises a plurality of execution units; wherein,

[0117] The central cloud is used to respond to the push stream information for the target data stream, obtain the task configuration information corresponding to the target data stream, and generate a data processing task for the push stream information according to the task configuration information; obtain the node operation parameters corresponding to each of the edge nodes, and select a first edge node that meets the first preset condition corresponding to the data processing task from each of the edge nodes according to the node operation parameters;

[0118] The first edge node is used to receive the data processing task, obtain unit operation parameters of each of the execution units, and select a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters;

[0119] The first execution unit is used to execute the data processing task for the target data stream.

[0120] Optionally, for view computing, it can include edge computing scenarios such as live broadcast, video monitoring, video recognition, image recognition, and image acquisition. Different edge computing scenarios can correspond to different data processing tasks. In the process of scheduling data processing tasks, the scheduling system can select appropriate edge nodes to perform corresponding data processing tasks according to the edge computing scenarios that actually need to be processed.

[0121] In the embodiment of the present application, for the central cloud in the scheduling system, it can be a central cloud server; for the edge node, it can be an edge unit that can be used for edge cloud computing based on the edge IDC, which can be a computer cluster or a single computer device. Among them, there can be a one-to-many relationship between the central cloud and the edge node, and there can also be a one-to-many relationship between the edge node and the execution unit. Optionally, when the edge node is a computer cluster, the execution unit can be a computer device under the computer cluster; when the edge node is a single computer device, the execution unit can be a unit in the computer device for data processing, etc., and the present application does not limit this.

[0122] In the process of scheduling data processing tasks, the central cloud can obtain the task configuration information corresponding to the target data stream based on the push stream information received for the target data stream, so as to generate the corresponding data processing task according to the task configuration information. Among them, for the target data stream, it can be the data stream involved in the edge computing scenario (image stream, video stream, audio stream, etc.); the push stream information can be the information of the user terminal pushing the data stream to the scheduling system. When the system receives the data stream sent by the user terminal, it can process the data stream accordingly, and can perform the corresponding processing operation through the edge node in the scheduling system. Therefore, when the user terminal sends the data stream to the system, the central cloud can generate the corresponding data processing task according to the task configuration information corresponding to the pushed data stream, so that the edge node can perform the corresponding data processing task.

[0123] In a specific implementation, the task configuration information can be static information. The same type of data stream can correspond to multiple different task configuration information, or it can correspond to only one type of task configuration information. For example, for a video stream, it can correspond to screenshot configuration information, recording configuration information, etc. For the screenshot configuration information, it can include the video stream ID, screenshot format, screenshot cycle (capture an image every 5 seconds), the screenshot save location, etc., then the central cloud can generate a screenshot task for the video stream based on these screenshot configuration information; for the recording configuration information, it can include the video stream ID, recording file format, recording file generation cycle (generate a recording file every 15 minutes), the recording file save location, etc., then the central cloud can generate a recording task for the video stream based on these screenshot configuration information, so that when the system receives the corresponding streaming information, it can generate the corresponding data processing task according to the task configuration information, and send it to the appropriate edge node, and the edge node will execute the corresponding data processing task.

[0124] In an alternative embodiment, referring to Figure 2, shows a structural block diagram of the central cloud provided in an embodiment of the present application, in which several service layers and task control services connected to the service layers may be included. Among them, the service layer can be used to generate and send corresponding data processing tasks to the task management service according to the task configuration information corresponding to the data flow; the task management service can be used to be responsible for the control of edge nodes, the control of tasks, etc. Specifically, the service layer can respond to the push stream information for the target data flow, obtain the task configuration information corresponding to the target data flow, and generate the data processing task for the push stream information according to the task configuration information, and send the data processing task to the task management service. After receiving the corresponding data processing task, the task management service can first obtain the node operation parameters corresponding to each edge node in the scheduling system, and select the first edge node that meets the first preset condition corresponding to the data processing task from each edge node according to the node operation parameters, and send the data processing task to the first edge node, so as to screen out the edge node matching the data processing task through the operation parameters for executing the data processing task, which can not only ensure the effectiveness of task execution, but also execute the data processing task through the appropriate edge node, which can ensure the balance of the edge node operation performance, and reduce bandwidth consumption and bandwidth cost.

[0125] Different data processing tasks have different requirements for the operating parameters of edge nodes. For example, for screenshot tasks, they mainly consume the CPU and I / O resources of edge nodes, and are required to be processed by edge nodes with low CPU usage and low I / O resource utilization; for video transcoding tasks, they mainly consume the GPU and network I / O resources of edge nodes, and are required to be processed by edge nodes with low CPU usage and low network I / O resource utilization; for recognition tasks, they consume CPU or GPU resources, and are required to be processed by edge nodes with low CPU resource usage or low GPU resource utilization, etc. In one example, for different data processing tasks, the task management service can extract the first operating parameter associated with the data processing task from the node operating parameters corresponding to each edge node, and then calculate the node score of each edge node based on the first operating parameter, and then select the edge node with the highest node score as the edge node to execute the data processing task according to the size of the node score, and then send the data processing task to the corresponding edge node.

[0126] In another example, for different data processing tasks, in addition to extracting the corresponding first operating parameters from the node operating parameters of each edge node and screening the edge nodes, the task management service can also obtain the bandwidth cost information corresponding to each edge node, and use the bandwidth cost information to calculate the first sub-score corresponding to each edge node, and calculate the second sub-score of each edge node based on the first operating parameter, and then sum the two sub-scores corresponding to the edge node to obtain the node score of the edge node, so that the edge node with the highest node score can be used as the edge node for executing the data processing task according to the size of the node score, and then the data processing task is sent to the corresponding edge node, so that by screening out the edge node matching the data processing task to execute the data processing task, not only the effectiveness of the task execution can be guaranteed, but also the execution of the data processing task by the appropriate edge node can ensure the balance of the edge node operating performance and reduce bandwidth consumption and bandwidth cost.

[0127] Among them, for bandwidth cost information, it can be the cost corresponding to the bandwidth required for data transmission between the edge node and the computing node to which the data flow is connected. The smaller the required bandwidth, the lower the cost, and the higher the first sub-score; otherwise, the lower the first sub-score. Similarly, the lower the usage rate corresponding to the operating parameters required to perform data processing tasks in the edge node, the higher the second sub-score; otherwise, the lower the second sub-score. For example, assuming there are two edge nodes A and B, for a screenshot task, the corresponding resource consumption is related to the video stream encoding format, resolution, screenshot cycle, etc. Specifically, when a video stream is in H264 encoding format (coefficient 10), resolution 720p, and screenshot cycle 5s, if 480p is used as the benchmark resource consumption Q = 10*720*5 / 480 = 75, edge nodes A and B, the used resources / total resources are 200 / 8000 and 300 / 8000 respectively, then Score2 can be 8000 / 200 and 8000 / 300 respectively. Assuming that Score1 is the same, the screenshot task can be sent to edge node A for execution.

[0128] It should be noted that, for the screening of edge nodes, a corresponding screening strategy can be set according to actual needs, that is, the embodiments of the present application include but are not limited to the above examples.

[0129] In an alternative embodiment, referring to Figure 3, showing a structural block diagram of the edge node provided in an embodiment of the present application, for an edge node, it may at least include a node management and control service and several execution unit management and control services connected to the node management and control service, and each of the execution unit management and control services corresponds to one of the execution units. Among them, the node management and control service can be used to perform task management on all execution units in the edge node, periodically report the node operating parameters and node heartbeat of the edge node to the central cloud, etc.; the execution unit management and control service can be used to be responsible for the collection of the unit operating parameters and unit heartbeat of the corresponding execution unit, and at the same time manage the data processing tasks of the execution unit and perform health monitoring on the execution unit; the execution unit is responsible for executing specific tasks. Optionally, for both the central cloud and the edge node, they may include corresponding cloud-edge channels for communication between the central cloud and the edge node.

[0130] Specifically, during the scheduling of tasks, each execution unit management service can obtain the unit operation parameters of the execution unit to which it belongs, and send the unit operation parameters to the node management service, so that the node management service periodically summarizes the unit operation parameters corresponding to each execution unit to obtain the node operation parameters, and reports the node operation parameters to the central cloud, so that the task management service of the central cloud can filter out the edge nodes that match the data processing task according to the node operation parameters. When the node management service receives the data processing task issued by the task management service of the central cloud, it can respond to receiving the data processing task and select the first execution unit that meets the second preset condition from each execution unit according to the unit operation parameters of each execution unit, and send the data processing task to the first execution unit management service corresponding to the first execution unit. The first execution unit management service can send the data processing task to the corresponding execution unit so that the first execution unit can execute the data processing service corresponding to the target data flow. At the same time, the first execution unit management service can manage the first execution unit during the process of the first execution unit executing the data processing task to ensure that the data processing task can be effectively executed.

[0131] For the screening of execution units, the node management service in the first edge node can extract the second operating parameters associated with the data processing task from the unit operating parameters corresponding to each execution unit, and then calculate the unit score corresponding to the execution unit according to the second operating parameter, and use the execution unit with the highest unit score as the first execution unit to execute the data processing task. Through scheduling within the edge node, the corresponding processing cost can be further reduced and the efficiency of task execution can be improved. It should be noted that for the screening of execution units, reference can be made to the screening process of the aforementioned edge node, which are the same or similar and will not be repeated here.

[0132] In addition, refer to Figure 4, showing a structural block diagram of the edge node provided in an embodiment of the present application, the edge node may also include an edge cache layer, which can be used to cache the interactive information between the cloud and the edge when the network status is not good. Specifically, in the process of the execution unit executing the corresponding data processing task, if the node management service detects that the communication connection with the central cloud is disconnected, the task execution result of the first execution unit for the data processing task and the reporting information corresponding to the task execution result can be obtained, and then the edge cache layer can store the task execution result and the reporting information. In addition, when the node management service detects that the communication connection with the central cloud is restored, a synchronization instruction can be sent to the edge cache layer so that the edge cache layer will synchronize the task execution result and the reporting information through the cloud-edge channel to the task management service of the central cloud, so that when there is a short-term unavailability between the cloud-edge network, the edge node can temporarily store the task execution result and the information to be reported in the edge cache layer, and perform cloud-edge synchronization after the network is restored, effectively ensuring the high availability of the edge service of the system during short-term network jitter.

[0133] Among them, the task execution result can be a processing result that characterizes the execution unit executing the data processing task, and may include information such as the execution progress of the data processing task and the data files generated during the execution process. The reported information may be a report on the execution status of the data processing task at that time, and may include the execution termination, execution failure, execution completion, etc. of the data processing task at that time. For example, in a screenshot task, the edge node takes a screenshot of an image. In addition to transmitting the image to the central cloud so that the central cloud stores the image in a specified location, it also needs to transmit the record corresponding to the transmitted data to the central cloud, which will be recorded by the central cloud so that data can be queried later. This application does not impose any restrictions on this.

[0134] In addition, for situations where the system is unavailable for a long time, it can be handled by scheduling strategies at different levels of edge node scheduling and execution unit scheduling to ensure the high availability of the system's edge services. Specifically, in the case of edge node scheduling, the task management service of the central cloud can obtain the node heartbeat of the first edge node. If the node heartbeat of the first edge node times out, the second edge node that meets the first preset condition can be selected based on the node operating parameters corresponding to other edge nodes except the first edge node; and the data processing task can be migrated to the second edge node so that the second edge node receives the data processing task, obtains the unit operating parameters of each execution unit, and selects the second execution unit that meets the second preset condition from each execution unit according to the unit operating parameters, and then the second execution unit executes the data processing task for the target data stream, so that by performing cross-node task scheduling between edge nodes, the high availability of edge services can be effectively guaranteed when the edge network is unavailable for a long time, thereby ensuring the overall stability of the system.

[0135] In the case of execution unit scheduling, the node management and control service of the first edge node can obtain the unit heartbeat of the first execution unit. If the unit heartbeat of the first execution unit times out, a third execution unit that meets the second preset condition can be selected based on the node operating parameters corresponding to other execution units except the first execution unit, and the data processing task can be sent to the second execution unit management and control service corresponding to the third execution unit. The second execution unit management and control service sends the data processing task to the third execution unit so that the third execution unit executes the data processing service corresponding to the target data flow. Moreover, when the node management and control service detects that the node operating parameters of each execution unit to which it belongs do not meet the second preset condition, it can send a task migration request to the task management and control service of the central cloud, so that the task management and control service can reselect the second edge node that meets the first preset condition based on the node operating parameters corresponding to other edge nodes except the first edge node, migrate the data processing task to the second edge node, and have the second edge node execute the data processing task, thereby performing heartbeat detection on the execution unit that executes the data processing task, so that when an abnormality occurs in the execution unit, the task migration within the edge node or between edge nodes can be used to improve the availability of the overall operation of the system, so that when the system provides the corresponding edge service, it can perform task migration in an imperceptible manner, which can effectively ensure the stability of the edge service.

[0136] It should be noted that the embodiments of the present application include but are not limited to the above examples. It is understandable that those skilled in the art can also make settings according to actual needs under the guidance of the ideas of the embodiments of the present application, and the present application does not impose any restrictions on this.

[0137] In an embodiment of the present application, for a scheduling system, it may include a central cloud and several edge nodes that are communicatively connected to the central cloud, each edge node may include several execution units, and in the process of scheduling tasks, the central cloud may respond to the push information for the target data stream, obtain the task configuration information corresponding to the target data stream, and generate a data processing task for the push information according to the task configuration information; obtain the node operating parameters corresponding to each edge node, and select a first edge node that meets the first preset condition corresponding to the data processing task from each edge node according to the node operating parameters, and select a suitable node for executing the data processing task through the node operating parameters of the edge node, which can not only ensure the effectiveness of task execution, but also execute the data processing task through a suitable edge node, which can ensure the balance of the edge node operation performance, and reduce bandwidth consumption and bandwidth cost, and, in the process of executing the data processing task through a suitable edge node, the edge node can also obtain the unit operating parameters of each execution unit, and select the first execution unit that meets the second preset condition from each execution unit according to the unit operating parameters, and execute the data processing task for the target data stream through the first execution unit, and through scheduling inside the edge node, the corresponding processing cost can be further reduced, and the efficiency of task execution can be improved.

[0138] In order to enable those skilled in the art to better understand the technical solution of the embodiment of the present application, an example is given below for illustrative explanation:

[0139] Reference Figure 5 , shows a block diagram of the scheduling system provided in an embodiment of the present application, the scheduling system may include a central cloud and several edge nodes, wherein the central cloud may include several service layers and a task control service connected to the service layer via an API (Application Programming Interface); the edge node may include an edge cache layer, a node control service, and several execution unit control services connected to the node control service via an API, each execution unit control service is connected to an execution unit. In addition, cloud-edge channels may be included in both the central cloud and the edge nodes for data transmission between cloud edges, within the central cloud, and within edge nodes.

[0140] During the scheduling of data processing tasks, the task control service in the central cloud can be responsible for the management and control of edge nodes, including obtaining the node operating parameters of edge nodes, bringing edge nodes online and offline, and monitoring the node heartbeat of edge nodes, etc. At the same time, it can also manage and control data processing tasks, including scheduling, starting and stopping data processing tasks, etc. In addition, the task control service can obtain the node operating parameters, node heartbeat, task feedback and other data of edge nodes through the cloud-edge channel, and can also send corresponding data processing tasks to edge nodes.

[0141] For edge nodes, the node management and control service can be responsible for managing and controlling each execution unit management and control service, including obtaining the unit operating parameters and unit heartbeat of the execution unit, issuing data processing tasks, and obtaining task feedback, etc.; for a single execution unit management and control service, it can be responsible for managing and controlling the execution unit to which it belongs, including collecting unit operating parameters (CPU, GPU, disk, memory, bandwidth, etc.), managing data processing tasks (task start, task stop, etc.), managing the cache, and monitoring the heartbeat of the execution unit, etc.; for the execution unit, during the execution of data processing tasks, it can call back the corresponding data to the node management and control service through the cloud-edge channel so that the node management and control service can transmit the data to the central cloud.

[0142] In addition, when the edge node is unavailable for a short time, the edge node can store the task execution results and reporting information corresponding to the currently executed data processing task to the edge cache layer, so that when communication is restored, the corresponding data can be synchronized to the central cloud through the cloud-edge channel.

[0143] Through the above optimization of the scheduling system, a highly available and low-cost task scheduling method for cloud-edge collaboration can be effectively provided, and high-availability solutions such as edge autonomy, intra-node scheduling, and cross-node scheduling can be realized, effectively ensuring the stability of edge services.

[0144] Reference Figure 6 , shows a flowchart of a task scheduling method provided in an embodiment of the present application, which may specifically include the following steps:

[0145] Step 601, in response to streaming information for a target data stream, obtaining task configuration information corresponding to the target data stream, and generating a data processing task for the streaming information according to the task configuration information;

[0146] Step 601, obtain node operating parameters corresponding to each edge node, and select a first edge node that meets a first preset condition corresponding to the data processing task from the various edge nodes according to the node operating parameters, and the first edge node is used to execute the data processing task for the target data flow.

[0147] In an optional embodiment, selecting, from the edge nodes according to the node operation parameter, a first edge node that meets a first preset condition corresponding to the data processing task includes:

[0148] Obtaining bandwidth cost information corresponding to each of the edge nodes;

[0149] Extracting a first operating parameter associated with the data processing task from the node operating parameters corresponding to the edge node;

[0150] The node score corresponding to the edge node is calculated according to the bandwidth cost information and the first operating parameter, and the edge node with the highest node score is used as the first edge node for executing the data processing task.

[0151] In an optional embodiment, obtaining a synchronization instruction sent by the first edge node and a task execution result and reporting information corresponding to the data processing task;

[0152] In response to the synchronization instruction, the task execution result and the reporting information are synchronized.

[0153] In an optional embodiment, obtaining a node heartbeat of the first edge node;

[0154] If the node heartbeat of the first edge node times out, a second edge node that meets the first preset condition is selected based on the node operating parameters corresponding to other edge nodes except the first edge node, and the data processing task is migrated to the second edge node. The second edge node is used to execute the data processing task for the target data flow.

[0155] Reference Figure 7 , shows a flowchart of a task scheduling method provided in an embodiment of the present application, which may specifically include the following steps:

[0156] Step 701, obtaining node operation parameters, and sending the node operation parameters to the central cloud, the central cloud is used to respond to the push stream information for the target data stream, obtain the task configuration information corresponding to the target data stream, and generate a data processing task for the push stream information according to the task configuration information;

[0157] Step 702, receiving the data processing task and obtaining unit operation parameters of each execution unit;

[0158] Step 703: Select a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters, wherein the first execution unit is used to execute the data processing task for the target data stream.

[0159] In an optional embodiment, selecting a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameter includes:

[0160] Extracting a second operating parameter associated with the data processing task from the unit operating parameters corresponding to each of the execution units;

[0161] The unit score corresponding to the execution unit is calculated according to the second operating parameter, and the execution unit with the highest unit score is used as the first execution unit for executing the data processing task.

[0162] In an optional embodiment, it also includes:

[0163] If it is detected that the communication connection with the central cloud is disconnected, obtaining the task execution result of the data processing task by the first execution unit and the reporting information corresponding to the task execution result;

[0164] The task execution result and the reporting information are stored in a preset edge cache layer.

[0165] In an optional embodiment, it also includes:

[0166] If it is detected that the communication connection with the central cloud is restored, a synchronization instruction is sent to the edge cache layer, and the edge cache layer is used to respond to the synchronization instruction and synchronize the task execution result and the reporting information to the central cloud.

[0167] In an optional embodiment, it also includes:

[0168] Obtaining a unit heartbeat of the first execution unit;

[0169] If the unit heartbeat of the first execution unit times out, a second execution unit that meets the second preset condition is selected based on the node operating parameters corresponding to other execution units except the first execution unit, and the data processing task is sent to the second execution unit. The second execution unit is used to execute the data processing task for the target data stream.

[0170] In an optional embodiment, it also includes:

[0171] If the node operating parameters of each of the execution units do not meet the second preset condition, a task migration request is sent to the central cloud. The central cloud is used to select a second edge node that meets the first preset condition based on the node operating parameters corresponding to other edge nodes except the first edge node, and migrate the data processing task to the second edge node.

[0172] Reference Figure 8 , shows a flowchart of the steps of a task scheduling method provided in an embodiment of the present application, which is applied to a scheduling system, wherein the scheduling system includes a central cloud and a plurality of edge nodes connected to the central cloud in communication, each of the edge nodes includes a plurality of execution units, and specifically may include the following steps:

[0173] Step 801: the central cloud obtains task configuration information corresponding to the target data stream in response to the streaming information of the target data stream, and generates a data processing task for the streaming information according to the task configuration information;

[0174] Step 802, the central cloud obtains node operating parameters corresponding to each edge node, and selects a first edge node that meets a first preset condition corresponding to the data processing task from the edge nodes according to the node operating parameters;

[0175] Step 803: The first edge node obtains unit operation parameters of each of the execution units, and selects a first execution unit that meets a second preset condition from the various execution units according to the unit operation parameters;

[0176] Step 804: The first execution unit executes the data processing task for the target data stream.

[0177] In an optional embodiment, the central cloud includes at least several service layers and a task control service connected to the service layer; the central cloud obtains task configuration information corresponding to the target data stream in response to the push stream information of the target data stream, and generates a data processing task for the push stream information according to the task configuration information, including:

[0178] The service layer obtains task configuration information corresponding to the target data stream in response to the streaming information of the target data stream, and generates a data processing task for the streaming information according to the task configuration information;

[0179] The task management service obtains node operating parameters corresponding to each of the edge nodes, and selects a first edge node that meets a first preset condition corresponding to the data processing task from each of the edge nodes according to the node operating parameters, and sends the data processing task to the first edge node.

[0180] In an optional embodiment, the first edge node includes at least a node management service and a plurality of execution unit management services connected to the node management service, each of the execution unit management services corresponds to one of the execution units; the first edge node obtains unit operation parameters of each of the execution units, and selects a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters, including:

[0181] The execution unit management and control service obtains the unit operation parameters of the execution unit to which it belongs, and sends the unit operation parameters to the node management and control service;

[0182] The node management and control service, in response to receiving the data processing task, selects a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters of each of the execution units, and sends the data processing task to a first execution unit management and control service corresponding to the first execution unit;

[0183] The first execution unit executes the data processing task for the target data stream, including:

[0184] The first execution unit obtains the data processing service sent by the first execution unit management service, and executes the data processing service corresponding to the target data flow.

[0185] In an optional embodiment, the first edge node further includes an edge cache layer, and the method further includes:

[0186] If the node management and control service detects that the communication connection with the central cloud is disconnected, then obtaining the task execution result of the data processing task by the first execution unit and the reporting information corresponding to the task execution result;

[0187] The task execution result and the reporting information are stored in the edge cache layer.

[0188] In an optional embodiment, it also includes

[0189] If the node management and control service detects that the communication connection with the central cloud is restored, it sends a synchronization instruction to the edge cache layer;

[0190] The edge cache layer synchronizes the task execution result and the reporting information to the task management and control service of the central cloud.

[0191] In an optional embodiment, it also includes:

[0192] The task management service obtains the node heartbeat of the first edge node; if the node heartbeat of the first edge node times out, selects a second edge node that meets the first preset condition according to the node operation parameters corresponding to other edge nodes except the first edge node; and migrates the data processing task to the second edge node;

[0193] The second edge node receives the data processing task, obtains unit operation parameters of each of the execution units, and selects a second execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters;

[0194] The second execution unit executes the data processing task for the target data stream.

[0195] In an optional embodiment, it also includes:

[0196] The node management and control service obtains the unit heartbeat of the first execution unit; if the unit heartbeat of the first execution unit times out, selects a third execution unit that meets the second preset condition according to the node operation parameters corresponding to other execution units except the first execution unit; and sends the data processing task to the second execution unit management and control service corresponding to the third execution unit;

[0197] The third execution unit obtains the data processing service sent by the second execution unit management service, and executes the data processing service corresponding to the target data flow.

[0198] In an optional embodiment, it also includes:

[0199] If the node control service detects that the node operating parameters of each execution unit to which it belongs do not meet the second preset condition, it sends a task migration request to the task control service of the central cloud;

[0200] The task management service selects a second edge node that meets the first preset condition according to node operating parameters corresponding to other edge nodes except the first edge node; and migrates the data processing task to the second edge node.

[0201] In an optional embodiment, selecting, from the edge nodes according to the node operation parameter, a first edge node that meets a first preset condition corresponding to the data processing task includes:

[0202] Obtaining bandwidth cost information corresponding to each of the edge nodes;

[0203] Extracting a first operating parameter associated with the data processing task from the node operating parameters corresponding to the edge node;

[0204] The node score corresponding to the edge node is calculated according to the bandwidth cost information and the first operating parameter, and the edge node with the highest node score is used as the first edge node for executing the data processing task.

[0205] In an optional embodiment, selecting a first execution unit that meets a second preset condition from the various execution units according to the unit operation parameters includes:

[0206] Extracting a second operating parameter associated with the data processing task from the unit operating parameters corresponding to each of the execution units;

[0207] The unit score corresponding to the execution unit is calculated according to the second operating parameter, and the execution unit with the highest unit score is used as the first execution unit for executing the data processing task.

[0208] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0209] The embodiment of the present invention also provides a schematic diagram of a network device, such as Fig. 9 As shown, the network device includes a memory 920, a transceiver 910, and a processor 900;

[0210] Memory 920, used for storing computer programs;

[0211] The transceiver 910 is used to receive and send data under the control of the processor 900;

[0212] In a first aspect, when the network device acts as a central cloud, the processor 900 is used to read the computer program in the memory 920 and execute the scheduling method of the aforementioned task;

[0213] In a second aspect, when the network device acts as an edge node, the processor 900 is used to read the computer program in the memory 920 and execute the scheduling method of the aforementioned task.

[0214] Among them, Fig. 9In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 900x10 and various circuits of memory represented by memory 920x20 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 910 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, which transmission medium may include a wireless channel, a wired channel, an optical cable, and other transmission media. The processor 900 is responsible for managing the bus architecture and general processing, and the memory 920 may store data used by the processor 900 when performing operations.

[0215] The processor 900 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD). The processor 900 may also adopt a multi-core architecture.

[0216] It should be noted here that the above-mentioned device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0217] An embodiment of the present invention further provides a processor-readable storage medium, characterized in that the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the scheduling method for the above-mentioned task.

[0218] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0219] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0220] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0221] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0222] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0223] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A task scheduling system, characterized in that: The scheduling system at least includes a central cloud and a plurality of edge nodes connected to the central cloud in communication, and each edge node includes a plurality of execution units; wherein, The central cloud is used to respond to the push stream information for the target data stream, obtain the task configuration information corresponding to the target data stream, and generate a data processing task for the push stream information according to the task configuration information; obtain the node operation parameters corresponding to each of the edge nodes, and select a first edge node that meets the first preset condition corresponding to the data processing task from each of the edge nodes according to the node operation parameters; The first edge node is used to receive the data processing task, obtain unit operation parameters of each of the execution units, and select a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters; The first execution unit is used to execute the data processing task for the target data stream; Wherein, the first edge node at least includes a node management service, and the central cloud at least includes a task management service; wherein, The node management and control service is used to send a task migration request to the task management and control service of the central cloud if the unit operation parameters of each execution unit in the first edge node do not meet the second preset condition; The task control service is used to select a second edge node that meets the first preset condition according to node operation parameters corresponding to other edge nodes except the first edge node; and migrate the data processing task to the second edge node; The second edge node is used to execute the data processing task for the target data flow.

2. The dispatching system according to claim 1, characterized in that: The central cloud includes at least several service layers and the task control service connected to the service layers; wherein, The service layer is used to respond to the push stream information for the target data stream, obtain the task configuration information corresponding to the target data stream, and generate the data processing task for the push stream information according to the task configuration information; The task management service is used to obtain the node operating parameters corresponding to each of the edge nodes, and select a first edge node that meets the first preset conditions corresponding to the data processing task from each of the edge nodes according to the node operating parameters, and send the data processing task to the first edge node.

3. The dispatching system according to claim 1, characterized in that: The first edge node at least includes a plurality of execution unit control services connected to the node control service, each of which corresponds to one execution unit; wherein, The execution unit management and control service is used to obtain unit operation parameters of the execution unit to which it belongs, and send the unit operation parameters to the node management and control service; The node management and control service is used to respond to receiving the data processing task and select a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters of each of the execution units, and send the data processing task to a first execution unit management and control service corresponding to the first execution unit; The first execution unit is used to obtain the data processing task sent by the first execution unit management and control service, and execute the data processing task corresponding to the target data flow.

4. The dispatching system according to claim 3, characterized in that: The first edge node also includes an edge cache layer; wherein, The node management and control service is further used to obtain the task execution result of the data processing task by the first execution unit and the reporting information corresponding to the task execution result if it is detected that the communication connection with the central cloud is disconnected; The edge cache layer is used to store the task execution result and the reporting information.

5. The dispatching system according to claim 4, characterized in that: The node management and control service is further configured to send a synchronization instruction to the edge cache layer if it is detected that the communication connection with the central cloud is restored; The edge cache layer is also used to synchronize the task execution result and the reporting information to the task management and control service of the central cloud.

6. The dispatching system according to claim 2, characterized in that: The task control service is used to obtain the node heartbeat of the first edge node; if the node heartbeat of the first edge node times out, select a second edge node that meets the first preset condition according to the node operation parameters corresponding to other edge nodes except the first edge node; and migrating the data processing task to the second edge node; The second edge node is used to receive the data processing task, obtain unit operation parameters of each of the execution units, and select a second execution unit that meets a second preset condition from each of the execution units in the second edge node according to the unit operation parameters; The second execution unit is used to execute the data processing task for the target data stream.

7. The dispatching system according to claim 3, characterized in that: The node management and control service is further used to obtain the unit heartbeat of the first execution unit; if the unit heartbeat of the first execution unit times out, select a third execution unit that meets the second preset condition according to the node operation parameters corresponding to other execution units except the first execution unit; and send the data processing task to the second execution unit management and control service corresponding to the third execution unit; The third execution unit is used to obtain the data processing task sent by the second execution unit management and control service, and execute the data processing task corresponding to the target data flow.

8. The dispatching system according to claim 1, characterized in that: The central cloud is also used to obtain bandwidth cost information corresponding to each of the edge nodes, and to extract a first operating parameter associated with the data processing task from the node operating parameters corresponding to the edge nodes; calculate the node score corresponding to the edge node based on the bandwidth cost information and the first operating parameter, and use the edge node with the highest node score as the first edge node to execute the data processing task.

9. The dispatching system according to claim 1, characterized in that: The first edge node is also used to extract a second operating parameter associated with the data processing task from the unit operating parameters corresponding to each of the execution units; calculate the unit score corresponding to the execution unit according to the second operating parameter, and use the execution unit with the highest unit score as the first execution unit to execute the data processing task.

10. A task scheduling method, characterized in that: include: In response to the push stream information for the target data stream, acquiring task configuration information corresponding to the target data stream, and generating a data processing task for the push stream information according to the task configuration information; Obtaining node operating parameters corresponding to each edge node, and selecting a first edge node that meets a first preset condition corresponding to the data processing task from each edge node according to the node operating parameters, the first edge node being used to receive the data processing task, and obtaining unit operating parameters of each execution unit in the first edge node, and selecting a first execution unit that meets a second preset condition from each execution unit according to the unit operating parameters, the first execution unit being used to execute the data processing task for the target data stream; Wherein, the method further comprises: receiving a task migration request sent by the first edge node, where the task migration request is a request generated by the first edge node when unit operating parameters of each execution unit in the first edge node do not meet the second preset condition; In response to the task migration request, a second edge node that meets the first preset condition is selected based on the node operating parameters corresponding to other edge nodes except the first edge node; and the data processing task is migrated to the second edge node, and the second edge node is used to execute the data processing task for the target data flow.

11. A task scheduling method, characterized in that: Applied to a first edge node, the method includes: Obtain node operating parameters, and send the node operating parameters to the central cloud, the central cloud is used to respond to the push stream information for the target data stream, obtain the task configuration information corresponding to the target data stream, and generate a data processing task for the push stream information according to the task configuration information, and select an edge node that meets the first preset condition corresponding to the data processing task according to the node operating parameters; receiving the data processing task, and obtaining unit operation parameters of each execution unit in the first edge node; Selecting a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameter, wherein the first execution unit is used to execute the data processing task for the target data stream; Wherein, the method further comprises: If the unit operating parameters of each of the execution units in the first edge node do not meet the second preset condition, a task migration request is sent to the central cloud. The central cloud is used to select a second edge node that meets the first preset condition based on the node operating parameters corresponding to other edge nodes except the first edge node, and migrate the data processing task to the second edge node. The second edge node is used to execute the data processing task for the target data flow.

12. A task scheduling method, characterized in that: Applied to a dispatching system, the dispatching system includes a central cloud and a plurality of edge nodes in communication connection with the central cloud, each of the edge nodes includes a plurality of execution units, and the method includes: The central cloud obtains task configuration information corresponding to the target data stream in response to the streaming information of the target data stream, and generates a data processing task for the streaming information according to the task configuration information; The central cloud obtains node operating parameters corresponding to each edge node, and selects a first edge node that meets a first preset condition corresponding to the data processing task from the edge nodes according to the node operating parameters; The first edge node obtains unit operation parameters of each of the execution units, and selects a first execution unit that meets a second preset condition from each of the execution units according to the unit operation parameters; The first execution unit executes the data processing task for the target data stream; Wherein, the first edge node at least includes a node management service, and the central cloud at least includes a task management service; wherein the method further includes: If the node management and control service detects that the node operating parameters of each execution unit in the first edge node do not meet the second preset condition, a task migration request is sent to the task management and control service of the central cloud; The task management service selects a second edge node that meets the first preset condition according to node operation parameters corresponding to other edge nodes except the first edge node; and migrates the data processing task to the second edge node; The second edge node performs the data processing task for the target data flow.

Images