A resource multi-level scheduling method in an edge scene

Abstract

The application discloses a multi-level resource scheduling method in an edge scene, which comprises the following steps: a CDN edge server is connected to a k8s cluster through a cloud edge channel kubeedge mode, and is exposed to a business through a virtual cluster; according to the resource requirement of the business, the CDN resource scheduling platform is applied for resources, and a resource list meeting the condition is returned on the premise of CDN business priority; the business side is mapped to the virtual cluster according to the responded resource list, and is divided into all clusters, CRD deployment arrangement is generated, and is deployed to the CDN edge cluster through an ESK platform; a CRD controller listens to EdgeInfo resources and creates a pod according to the resource content; and the business container reports the actual resource consumption to monitoring through an index, and the resource scheduling platform globally adjusts according to the monitoring data and the resource threshold.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed edge computing, and more particularly to a multi-level resource scheduling method in an edge scenario. BACKGROUND

[0002] With the growth of the content distribution network (CDN) service market, the number of edge node machines of CDN service providers is tens of thousands, and the number of devices is growing. CDN business uses resources with obvious peaks and valleys, and mainly uses uplink bandwidth resources, and other resource utilization is relatively low. With the rapid development of edge computing, edge business is constantly enriched, and if CDN edge nodes can be reasonably reused to improve device resource utilization and reduce business costs to improve product competitiveness.

[0003] Because CDN edge nodes are distributed in different regions and different operator network environments, CDN services are not evenly distributed in each edge node, resulting in unevenly reusable resources of edge node machines. Under the premise of ensuring CDN services, CDN edge nodes are reused through distributed resource scheduling. SUMMARY

[0004] In view of the deficiencies in the prior art, the purpose of the present application is to provide a multi-level resource scheduling method in an edge scenario, which solves the problem of idle time resource reuse of large-scale CDN edge nodes.

[0005] To achieve the above purpose, the present application provides the following technical solutions:

[0006] A multi-level resource scheduling method in an edge scenario, comprising the following steps:

[0007] The CDN edge server is connected to the k8s cluster through the cloud edge channel kubeedge mode, and is exposed to the business through the virtual cluster.

[0008] According to the resource demand of the business, the CDN resource scheduling platform is applied for resources, and under the premise of CDN business priority, a resource list meeting the conditions is returned;

[0009] The business side maps the response resource list to the virtual cluster, and divides it into all virtual clusters to generate CRD deployment arrangement and deploy it to the CDN edge cluster through the ESK platform;

[0010] The CRD controller listens to the EdgeInfo resource and creates a pod according to the resource content, and synchronizes the pod to the CDN edge working cluster through syncer;

[0011] Business containers report actual resource usage to monitoring through indicators, and the resource scheduling platform performs global coordination based on monitoring data and resource thresholds.

[0012] As a further improvement of the present invention, the pod is synchronized to the CDN edge working cluster through a syncer.

[0013] As a further improvement of the present invention, in order to protect the controllable network resources of the edge work node, CNI is enabled and current limiting is performed through the meta plug-in under the containernetworking project.

[0014] As a further improvement of the present invention, the following one or more dimensions are specified when applying for resources:

[0015] Dimension 1: Total upstream and downstream bandwidth, in Mbps;

[0016] Dimension 2: Application health value requirements, expulsion health value requirements;

[0017] Dimension 3: Single-machine CPU core count requirement, load application requirement, load eviction requirement, and load as a percentage;

[0018] Dimension 4: memory size requirement (unit: M), memory load request requirement, memory load eviction requirement, load as a percentage;

[0019] Dimension 5: Disk size requirement (unit: M), disk load application requirement, disk load eviction requirement (load as a percentage);

[0020] Dimension 6: Link load application requirements, link load eviction requirements, load is the number.

[0021] As a further improvement of the present invention, the node dimension defines a resource usage template. After the service is applied to the resource pool, the node dimension resource usage is generated within the resource pool. The resource usage of each node is declared in EdgeInfo.

[0022] EdgeTask is used to define label selection, image version, number of replicas, and EdgeInfo instance information.

[0023] As a further improvement of the present invention, the controller schedules pods to idle Node nodes through node affinity, monitors the status of nodes in the cluster, and deletes the corresponding Pod when the node status is abnormal and an idle Pod is deployed.

[0024] As a further improvement of the present invention, the K8s operator cooperates with the cloud-edge channel to implement edge node load scheduling, including the following steps:

[0025] Operator registers custom resources with the Kubernetes master's API server. The custom resources are registered as EdgeTask and EdgeInfo, which are used to specify node usage rules and workloads.

[0026] The Operator starts a custom controller and maintains the number of EdgeTask replicas through reconcile.

[0027] The K8s node is registered with the K8s master through the cloud-edge channel composed of cloud-manager and edge-agent;

[0028] When users operate businesses through the platform, the system writes the specified machine range information into the custom resource service orchestration of the EdgeInfo type, and writes the image version and number of replicas into the custom resource service orchestration of the EdgeTask type;

[0029] When orchestrating deployment to a virtual cluster, the custom controller's reconcile function creates a pod for each node in the EdgeInfo list.

[0030] The custom controller uses reconcile to perform tuning based on the information in the EdgeInfo orchestration, creates a task service pod, and submits it to the API server.

[0031] Use syncer to synchronize the pod to the CDN edge worker cluster.

[0032] As a further improvement of the present invention, the K8s operator cooperates with the cloud-edge channel to implement edge node load scheduling, further comprising the following steps:

[0033] The cloud-manager in the cloud monitors resource changes on the api-server in real time and identifies pod creation / updates.

[0034] The cloud-manager distributes the pod orchestration modification message to the edge-agent at the edge through the cloud-edge channel;

[0035] The edge-agent at the edge receives the orchestration modification message and calls CRI to manage the pod.

[0036] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the multi-level resource scheduling method in the edge scenario as described above.

[0037] A computer device comprising:

[0038] a memory for storing instructions;

[0039] a processor for executing the instructions, so that the computer device performs operations of implementing the multi-level resource scheduling method in an edge scenario as described above.

[0040] Advantages of the present application:

[0041] 1. Since the original k8s itself does not provide scheduling for bandwidth resource multiplexing, the existing CDN bandwidth scheduling is based on business, and this method can only solve CDN business scheduling, cannot be applied to other business scenarios of edge computing, and cannot solve the resource multiplexing problem of large-scale edge nodes. A distributed resource scheduling method for large-scale edge nodes is implemented by using the cloud native concept, CDN resource scheduling, resource dimension grouping, multi-cluster scheduling and k8s operator, which has automatic reconciliation capability;

[0042] 2. From the perspective of resource scheduling, the scheduling process is divided into two steps of center resource pool scheduling and edge k8s node scheduling. The center resource pool scheduling is based on resource dimension, and the edge scheduling is within the scope of the resource pool scheduling result and is defined and implemented through declarative API and k8s operator;

[0043] 3. Precise control of large-scale edge node resources is achieved through multi-level scheduling of the center and the edge. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a SaaS platform architecture diagram;

[0045] Figure 2 is a SaaS business issuance orchestration flowchart. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the described embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor belong to the scope of protection of the present application.

[0047] Unless otherwise defined, technical terms or scientific terms used in the present application shall have the same meaning as those commonly understood by a person of ordinary skill in the art to which the present application belongs. The terms "first", "second", and similar terms used in the present application do not denote any order, quantity, or importance, but are used to distinguish different components. The terms "include", "contain", and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like are used only to indicate relative positional relationships, and when the absolute positions of the described objects are changed, the relative positional relationships can also be changed accordingly.

[0048] In order to keep the following description of the embodiments of the present application clear and brief, the present application omits the detailed description of known functions and known components.

[0049] Embodiment 1:

[0050] Reference Figure 1 and Figure 2 As shown in FIGS. 1 to 8, the present application is a specific embodiment of a resource multi-level scheduling method in an edge scenario, which includes the following steps:

[0051] The CDN edge server accesses the k8s cluster through the cloud edge channel kubeedge mode, avoiding the hidden dangers caused by the direct exposure of the CDN K8S cluster to the outside, and exposing the virtual cluster to the business for use;

[0052] According to the resource demand of the business, resources are applied to the CDN resource scheduling platform, and under the premise of CDN business priority, a resource list meeting the conditions is returned. The resource demand includes but is not limited to bandwidth granularity, device health value, and distribution area. In this process, the business side needs to report the actual usage through monitoring indicators and has the ability to quickly release resources;

[0053] The business side maps the response resource list to the virtual cluster, and divides it into all virtual clusters, generates CRD deployment arrangement, and deploys it to the CDN edge cluster through the ESK platform;

[0054] The CRD controller listens to the EdgeInfo resource and creates a pod according to the resource content. The pod is synchronized to the CDN edge working cluster through syncer;

[0055] The business container reports the actual resource usage to the monitoring through the index. The resource scheduling platform performs global reconciliation according to the monitoring data and resource threshold. In order to protect the controllable network resources of the edge work node, the CNI needs to be started to limit the flow through the meta plug-in under the containernetworking project.

[0056] The application resource specifies one dimension or multiple dimensions as follows:

[0057] Dimension 1: Total uplink and downlink bandwidth, unit: Mbps;

[0058] Dimension 2: Application health value requirement, eviction health value requirement;

[0059] Dimension 3: Single CPU core requirement, load application requirement, load eviction requirement, load percentage;

[0060] Dimension 4: Memory size requirement, unit: M, memory load application requirement, memory load eviction requirement, load percentage;

[0061] Dimension 5: Disk size requirement, unit: M, disk load application requirement, disk load eviction requirement, load percentage;

[0062] Dimension 6: Link number load application requirement, link number load eviction requirement, load number.

[0063] The node dimension defines the resource usage template. After the business application is applied to the resource pool, the node dimension resource usage is generated within the resource pool range. The resource usage of each node is declared in EdgeInfo, and the arrangement information is as follows:

[0064]

[0065]

[0066] EdgeTask is used to define tag selection, image version, replica number and EdgeInfo instance information. The arrangement information is as follows:

[0067]

[0068] The controller schedules the pod to the idle Node node through node affinity. The state of the node in the cluster is listened to. When the node state is abnormal and the idle pod is deployed, the corresponding Pod will be deleted.

[0069] The K8s operator cooperates with the cloud-edge channel to realize the edge node load scheduling, including the following steps:

[0070] Operator registers custom resources with the Kubernetes master's API server. The custom resources are registered as EdgeTask and EdgeInfo, which are used to specify node usage rules and workloads.

[0071] The Operator starts a custom controller and maintains the number of EdgeTask replicas through reconcile.

[0072] The K8s node is registered with the K8s master through the cloud-edge channel composed of cloud-manager and edge-agent;

[0073] When users operate businesses through the platform, the system writes information such as the specified machine range into the custom resource service orchestration of the EdgeInfo type, and writes information such as the image version and number of replicas into the custom resource service orchestration of the EdgeTask type.

[0074] When orchestrating deployment to a virtual cluster, the custom controller's reconcile function creates a pod for each node in the EdgeInfo list.

[0075] The custom controller uses reconcile to perform tuning based on the information in the EdgeInfo orchestration, creates a task service pod, and submits it to the API server.

[0076] Use syncer to synchronize pods to the CDN edge working cluster;

[0077] The cloud-manager in the cloud monitors resource changes on the api-server in real time and identifies pod creation / updates.

[0078] The cloud-manager distributes the pod orchestration modification message to the edge-agent at the edge through the cloud-edge channel;

[0079] The edge-agent at the edge receives the orchestration modification message and calls CRI to manage the pod.

[0080] Performance Testing Service (PTS) stresses the SaaS platform, simulating real-world business scenarios with massive user volumes to comprehensively verify the performance, capacity, and stability of business sites. This requires a large number of edge nodes to initiate stress testing. Analysis shows that PTS stress testing generally consumes a large amount of total bandwidth, primarily downstream bandwidth and network connections, while CDN acceleration primarily consumes upstream bandwidth.

[0081] PTS runs the business to the resource scheduling platform to apply for the downlink bandwidth, the resource requirement of two dimensions of the area, generates the business CRD resource and deploys to the edge node.

[0082] Since the original k8s itself does not provide scheduling for bandwidth resource multiplexing, the existing CDN bandwidth scheduling is based on business, and this method can only solve the CDN service scheduling, and cannot be applied to other service scenarios of edge computing, and cannot solve the resource multiplexing problem of large-scale edge nodes. A distributed resource scheduling method for large-scale edge nodes is implemented by using the cloud native concept, CDN resource scheduling, resource dimension grouping, multi-cluster scheduling and k8s operator, which has automatic adjustment capability; from the perspective of resource scheduling, the scheduling process is divided into two steps of center resource pool scheduling and edge k8s node scheduling, the center resource pool scheduling is based on resource dimension, and the edge scheduling is within the scope of the resource pool scheduling result, and is defined and implemented through a declarative API and a k8s operator; through the multi-level scheduling of the center and the edge, the precise control of the resources of the large-scale edge nodes is realized.

[0083] Embodiment 2:

[0084] In this embodiment, a computer device is provided, which includes a memory and a processor, the memory is used to store instructions, and the processor is used to execute the instructions, so that the computer device executes the resource multi-level scheduling method in the edge scenario as described above.

[0085] Embodiment 3:

[0086] In this embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed to implement the operation of the resource multi-level scheduling method in the edge scenario as described above.

[0087] The computer readable storage medium includes: a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various storage program code media.

[0088] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0089] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0090] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0091] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0092] Furthermore, although exemplary embodiments have been described in the present disclosure, it is to be understood that the scope of the present application includes any and all modifications, omissions, combinations (e.g., of aspects across various embodiments), adaptations and / or alterations as

[0093] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of their solutions) can be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the present invention. This should not be interpreted as an intention that a disclosed feature that is not required to be protected is necessary for any claim. On the contrary, the subject matter of the present invention may be less than all the features of a specific disclosed embodiment. Thus, the following claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the appended claims and the full scope of equivalents to which these claims are entitled.

[0094] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the scope of the present invention. The scope of protection of the present invention is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the spirit and scope of protection of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present invention.

Claims

1. A multi-level resource scheduling method in an edge scenario, characterized in that: The following steps are involved: The CDN edge server is connected to the Kubernetes cluster through the cloud-edge channel kubeedge and is exposed to business use through the virtual cluster virtual cluster; Apply for resources from the CDN resource scheduling platform based on the business's resource needs. With CDN business priority, a list of qualified resources is returned. The business side maps the response resource list to the virtual cluster and evenly distributes it to all virtual clusters. It then generates a CRD deployment orchestration and deploys it to the CDN edge cluster through the ESK platform. The CRD controller listens to the EdgeInfo resource and creates a pod based on the resource content, and synchronizes the pod to the CDN edge working cluster through the syncer; Business containers report actual resource usage to monitoring through indicators, and the resource scheduling platform performs global coordination based on monitoring data and resource thresholds.

2. The multi-level resource scheduling method in an edge scenario according to claim 1, characterized in that: The pod is synchronized to the CDN edge working cluster through syncer.

3. The multi-level resource scheduling method in an edge scenario according to claim 2, characterized in that: To protect the controllable network resources of edge worker nodes, enable CNI and implement rate limiting through the meta plug-in under the containernetworking project.

4. The multi-level resource scheduling method in an edge scenario according to claim 3, characterized in that: When applying for resources, specify one or more of the following dimensions: Dimension 1: Total upstream and downstream bandwidth, in Mbps; Dimension 2: Application health value requirements, expulsion health value requirements; Dimension 3: Single-machine CPU core count requirement, load application requirement, load eviction requirement, and load as a percentage; Dimension 4: memory size requirement (unit: M), memory load request requirement, memory load eviction requirement, load as a percentage; Dimension 5: Disk size requirement (unit: M), disk load application requirement, disk load eviction requirement (load as a percentage); Dimension 6: Link load application requirements, link load eviction requirements, load is the number.

5. The multi-level resource scheduling method in an edge scenario according to claim 4, characterized in that: The node dimension defines a resource usage template. After the business is applied to the resource pool, the node dimension resource usage is generated within the resource pool. The resource usage of each node is declared in EdgeInfo. EdgeTask is used to define label selection, image version, number of replicas, and EdgeInfo instance information.

6. The multi-level resource scheduling method in an edge scenario according to claim 5, characterized in that: The controller schedules pods to idle nodes based on node affinity and monitors the status of nodes in the cluster. When the node status is abnormal and idle pods are deployed, the corresponding pods will be deleted.

7. The multi-level resource scheduling method in an edge scenario according to claim 6, characterized in that: K8soperator cooperates with cloud-edge channel to implement edge node load scheduling, which includes the following steps: Operator registers custom resources with the Kubernetes master's API server. The custom resources are registered as EdgeTask and EdgeInfo, which are used to specify node usage rules and workloads. The Operator starts a custom controller and maintains the number of EdgeTask replicas through reconcile. The K8s node is registered with the K8s master through the cloud-edge channel composed of cloud-manager and edge-agent; When users operate businesses through the platform, the system writes the specified machine range information into the custom resource service orchestration of the EdgeInfo type, and writes the image version and number of replicas into the custom resource service orchestration of the EdgeTask type. When orchestrating deployment to a virtual cluster, the custom controller's reconcile function creates a pod for each node in the EdgeInfo list. The custom controller uses reconcile to tune according to the information in the EdgeInfo orchestration, creates a task service pod, and submits it to the API server; Use syncer to synchronize the pod to the CDN edge worker cluster.

8. The multi-level resource scheduling method in an edge scenario according to claim 7, characterized in that: The K8soperator cooperates with the cloud-edge channel to implement edge node load scheduling, which also includes the following steps: The cloud-manager in the cloud monitors resource changes on the api-server in real time and identifies pod creation / updates. The cloud-manager distributes the pod orchestration modification message to the edge-agent at the edge through the cloud-edge channel; The edge-agent at the edge receives the orchestration modification message and calls CRI to manage the pod.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-level resource scheduling method in the edge scenario as described in any one of claims 1 to 8 is implemented.

10. A computer device, characterized in that: include: a memory for storing instructions; A processor is used to execute the instructions so that the computer device performs operations to implement the multi-level resource scheduling method in the edge scenario as described in any one of claims 1-8.

Images