A cluster scheduling method based on ubiquitous computing architecture

Abstract

The present invention discloses a cluster scheduling method based on a pan-compatible computing architecture, which can be scheduled according to the characteristic labels of containers in the pan-compatible computing architecture, each label represents the different functions that each container is mainly responsible for, such as computing function, storage function, network function, etc. According to the difference in labels, different containers are scheduled to nodes with different resource advantages. If two or more containers frequently work together, the scheduling algorithm will schedule them to the same node, reducing the additional overhead caused by the use of the network. The main difference from other scheduling algorithms is that the scheduling algorithm prioritizes scheduling according to the characteristics of the Docker container, rather than the characteristics of the node, and tries to make each container run on a node with better resources for the function it is responsible for.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing, and in particular to a cluster scheduling method based on a pan-capable computing architecture. Background Art

[0002] In recent years, cloud computing technology has gradually entered people's lives. Cloud computing is a type of distributed computing technology. It integrates many existing technologies such as storage, computing, virtualization, etc. Through the network cloud, it decomposes the huge data computing and processing programs that users need to run into countless small programs, and then sends them to a unified computing center for calculation and analysis, and then returns the results to the user. Cloud computing technology provides users with pay-as-you-go services. Users can purchase services according to their own needs; extensive network access, as long as there is a network, it can be used; it uses resource pooling technology, which can add or subtract resources at will to shield hardware differences; fast elastic scaling, which can quickly increase or decrease services according to demand.

[0003] The pan-content computing architecture is a new type of fine-grained, loosely coupled distributed cloud computing architecture based on Docker that is improved on the basis of cloud computing. It establishes a cloud computing architecture that is Docker-oriented, available as needed, and meets inherent security. Compared with the traditional cloud computing architecture, the main difference is that the multi-process collaboration in traditional cloud computing has been transformed into the collaboration between multiple Dockers, achieving a detailed application granularity division and effectively decoupling the association between Dockers with different functions, realizing a loosely coupled service architecture, and on the other hand, it will also have a good effect in ensuring the security of data information.

[0004] Kubernetes is a container cluster management platform that was open-sourced by Google in 2014. It can not only manage and orchestrate containers conveniently and efficiently, but also provides monitoring plug-ins for various container indicators. There are two types of roles in the cluster nodes, namely management nodes and working nodes. As the name implies, the management node is responsible for the management, scheduling, storage, etc. of the entire cluster architecture, and the working node is the node where the actual application container works.

[0005] In a Kubernetes cluster based on a pan-inclusive computing architecture, due to the characteristics of the pan-inclusive computing architecture, the services in the cluster will be subdivided into multiple detailed independent services, and each service will be completed by an independent container. Therefore, to complete a service initiated by a user, multiple containers responsible for different functions will work together to complete it. In this collaborative work of multiple independent containers, it is inevitable to bring additional network overhead to the cluster, which also involves cluster management issues, especially scheduling algorithms. How to select the right container to run on the right node has become a hot research issue. The resources of a node in a Kubernetes cluster must be limited. A suitable scheduling algorithm can not only improve the utilization of system resources, but also reduce system consumption caused by frequent scheduling of containers, saving costs and time.

[0006] In summary, a scheduling algorithm is designed based on cluster management scheduling based on Kubernetes to achieve a purpose of container scheduling. This scheduling algorithm labels Docker containers according to the different functions of each Docker container in the pan-comprehensive computing, and schedules them to more suitable nodes according to the types of container labels. This method of implementing Kubernetes cluster scheduling and deployment based on the pan-comprehensive computing architecture is a container-oriented solution. It is not like the node-oriented scheduling solution we see in daily life, which scores nodes according to various indicators and then selects the highest-scoring node for scheduling. In pan-comprehensive computing, this new scheduling solution is undoubtedly a high-resource utilization and efficient solution.

[0007] This scheduling algorithm is mainly designed to improve the utilization of system resources in Kubernetes clusters based on the pan-compatibility computing architecture. Compared with the traditional Kubernetes cluster, a business is only responsible for an independent container, and there is no data interaction between containers. In pan-compatibility computing, multiple containers need to work together, so data interaction between containers occurs. If the original scheduling scheme is used to deploy containers only based on the load balancing of the nodes, if two containers that need to communicate and interact frequently are on different working nodes, the network overhead problem will be caused. In addition, the node attributes required by containers with different functions are different. For example, the storage function container requires more storage capacity of the node, and the computing function container requires higher CPU. This new container-oriented scheduling algorithm is designed to solve this type of problem. Summary of the invention

[0008] To solve the above problems, the present invention proposes a cluster scheduling method based on a pan-compatible computing architecture, which solves the problem of system resource utilization in a Kubernetes cluster based on a pan-compatible computing architecture. Compared with the traditional Kubernetes cluster, a business is only responsible for an independent container, and does not involve data interaction between containers. In pan-compatible computing, because multiple containers need to work together, data interaction between containers occurs. If based on the original scheduling scheme, containers are deployed only according to the load balancing of nodes, if two containers that need frequent communication and interaction are involved, the network overhead problem is caused in different working nodes. And the node attributes responsible for the requirements of containers with different functions are different, such as the container responsible for the storage function has a greater demand for the storage capacity of the node, and the container responsible for the computing function has a higher requirement for the CPU. In the Kubernetes cluster, at least three nodes focusing on different characteristics and multiple containers with different functions are included. The three nodes focusing on different characteristics, one is responsible for running a container that requires a large amount of computing, called a computing node, and has better computing resources. One is responsible for running a container that needs to store a large amount of data, called a storage node, and has better storage resources. One is responsible for running a container that is not sensitive to other resources, called a basic node, and has balanced system resources. In addition, the Kubernetes-based cluster can also monitor and manage the data and status of all containers and nodes in the entire architecture. There can be multiple nodes in the architecture, such as nodes responsible for storage functions, to form a distributed data storage cluster, which is mainly responsible for distributed storage of data in containers responsible for storage functions, improving the overall backup storage capabilities and scalability of the architecture. The scheduling algorithm will schedule according to the labels of the nodes and the labels of the containers to be scheduled, so that each container can run on a node with better resources for the functions it is responsible for.

[0009] To achieve the above object, the present invention provides the following technical solution: a cluster scheduling method based on a ubiquitous computing architecture, specifically comprising the following steps:

[0010] Step 1: Configure the Kubernetes cluster and define the management node and various working nodes of the Kubernetes cluster, including establishing various Docker images on the management node and working nodes and installing related tools;

[0011] Step 2: Divide the services into multiple containers with different functions according to different services of users. The containers include storage functions and computing functions. These containers are then deployed on the Kubernetes cluster.

[0012] Step 3: At the beginning of the scheduling algorithm, first query all available working nodes and record the label information of these working nodes;

[0013] Step 4: Query the labels of all containers to be scheduled;

[0014] Step 5: Compare the labels of the containers to be scheduled with the labels of the nodes in turn. If there is a matching node, proceed to step 6. If not, schedule the container to the basic node.

[0015] Step 6: According to the label information, check whether the container to be scheduled has any container that needs to communicate frequently. If so, schedule one or more containers corresponding to the container to the same working node. If not, directly schedule the container to be scheduled to the working node corresponding to the label of the container.

[0016] Step 7: Deploy the visualization plug-in of the Kubernetes cluster to visualize the status of all nodes and containers in the Kubernetes cluster.

[0017] Preferably, in step 2, when a business requires a container with storage or computing functions, multiple containers with the same functions may be generated; when multiple containers with storage functions are generated, user data and database data in a business may not be stored in only one container responsible for storage function, and there may be one-to-many, many-to-one, or many-to-many situations, forming a distributed storage architecture; multiple containers with computing functions can be generated to improve the parallel capability of the architecture.

[0018] Preferably, in step 6, the container is scheduled to different nodes according to the container's own label. If two or more containers need to communicate frequently, the scheduling algorithm will schedule the containers to the same node, thereby reducing network consumption between the containers, reducing resource consumption, and improving system resource utilization;

[0019] The data interaction and communication between multiple containers uses the Flask framework under Python. It is a microframework that does not rely on external libraries. It is very lightweight, has fewer dependencies, and has good performance in terms of security. The Flask framework uses the HTTP protocol in data interaction, which is simple, fast, flexible and efficient. It supports breakpoint continuation and retransmission mechanisms and has high security.

[0020] Preferably, scheduling is performed according to the characteristic labels of the containers in the Kubernetes cluster. Each label represents a different function that each container is mainly responsible for, including computing function, storage function, network function, etc. Different containers are scheduled to nodes with different resource advantages according to different labels. Containers that provide computing services will be scheduled to nodes with better computing power, and containers that provide storage services will be scheduled to nodes with better storage resources. The main difference from other scheduling algorithms is that this scheduling algorithm prioritizes scheduling according to the characteristics of the container rather than the characteristics of the node, and tries to make each container run on a node with better resources for the function it is responsible for.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] The scheduling algorithm is designed for the new cloud computing architecture, the pan-inclusive architecture. This scheduling algorithm is a scheduling method for containers in the architecture. It can be scheduled according to the characteristic labels of the containers in the pan-inclusive computing architecture. Each label represents the different functions that each container is responsible for, such as computing function, storage function, network function, etc. According to the different labels, different containers are scheduled to nodes with different resource advantages. If two or more containers frequently work together, the scheduling algorithm will schedule them to the same node to reduce the additional overhead caused by the use of the network. The main difference from other scheduling algorithms is that this scheduling algorithm prioritizes scheduling according to the characteristics of the Docker container, rather than the characteristics of the node, and tries to make each container run on a node with better resources for the function it is responsible for.

[0023] The main technical problem solved is that compared with the traditional Kubernetes cluster, a business is only responsible for an independent container, and there is no data interaction between containers. In pan-capable computing, because multiple containers need to work together, the problem of data interaction between containers arises, and the problem of which node different functional containers in the Kubernetes cluster should be scheduled to. A scheduling algorithm for container characteristics in a Kubernetes cluster based on a pan-capable computing architecture adopts a container-oriented scheduling method, classifies different functional containers according to different labels, and tends to schedule each type of container with the same label to a node that can provide better resources of this type. For example, containers that provide computing services will be scheduled to nodes with better computing capabilities, and containers that provide storage services will be scheduled to nodes with better storage resources. In addition, when two or more containers work together, if there is frequent communication between them, the scheduling algorithm will schedule these containers to the same node to reduce the network overhead between them. This scheduling algorithm oriented to container characteristics plays a vital role in the performance of containers in a Kubernetes cluster based on a pan-capable computing architecture.

[0024] In addition, the visualization module of Kubernetes in the present invention can visualize multiple indicators of nodes and containers in the cluster, detect abnormal conditions in the cluster, and improve the reliability of the Kubernetes cluster in responding to emergencies. In the Kubernetes cluster, if the container is shut down due to an unexpected situation, the working node will automatically restart the container, which also improves the overall anti-attack and security of the system.

[0025] The solution to the scheduling problem of the present invention is a scheduling algorithm for containers, which determines which node the container will be scheduled to run on according to the characteristics of the Docker container itself. For example, containers that provide computing services will be scheduled to nodes with better computing capabilities, and containers that provide storage services will be scheduled to nodes with better storage capabilities. In addition, when two or more containers work together, if they communicate frequently, the scheduling algorithm will uniformly schedule these containers to the same node, reducing the network overhead between them. The collaborative communication and interaction between Docker containers adopts a lightweight Flask framework, and tries to achieve a lightweight architecture, thereby improving the efficiency of resource utilization of the system. In addition, a visualization module of kubernetes is used to visualize the various indicators of nodes and containers, which provides a guarantee for the reliability of the architecture and improves the security of the architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a scheduling flow chart of a cluster scheduling method based on a ubiquitous computing architecture of the present invention;

[0027] Figure 2 This is a deployment architecture diagram of a text editor in a Kubernetes cluster after scheduling based on a pan-capable computing architecture. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings of the present invention in the embodiments of the present invention to clearly and completely describe the technical solutions of the present invention in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] like Figure 1 As shown,

[0030] Step 1: At the beginning of the scheduling algorithm, all available working nodes are queried and the label information of these working nodes is recorded.

[0031] Step 2: Query the labels of all containers to be scheduled.

[0032] Step 3: Compare the labels of the containers to be scheduled with the labels of the nodes in turn. If there is a matching node, the container is bound to the corresponding node. If not, the container is scheduled to the basic node.

[0033] Step 4: Based on the label information, check whether the container to be scheduled has containers that need to communicate frequently. If so, schedule one or more corresponding containers to the same working node. If there are no such containers, schedule the container directly to the working node corresponding to its label.

[0034] like Figure 2 Figure 2 shows a deployment architecture diagram of a text editor in a Kubernetes cluster based on a pan-capable computing architecture after scheduling.

[0035] In this text editor business architecture, four functional Docker containers are refined, including editor container, storage container, database container and a login management container. The system architecture has a master management node and three working nodes, which are labeled as computing node, storage node and basic node.

[0036] Among them, the editor container is a container that is subdivided from the text editing business in the pan-content computing architecture to realize text functions. It can realize various basic text editing functions. The storage container is responsible for storing the text edited by the text editor container, including plain text files, pictures or audio and video data and other files. The database container is responsible for storing database files such as database user names and user passwords, which are used for initial user login and provide identity authentication services when using the editor container. In addition, the login management container is responsible for the user's initial login function, realizes multi-user access to the text editor business, and interacts with the database container for identity authentication and login. The two types of storage containers can also send the stored data back to the containers responsible for other functions to realize various functions.

[0037] For the three working nodes, Node 1 as a computing node has better computing resources, Node 2 as a storage node has better storage capacity, and Node 3 is a basic node with relatively balanced resources that can meet the needs of containers of various functions.

[0038] During the scheduling process, the editor container needs better computing resources to achieve timely response to users, so it is scheduled to the computing node according to the label information. The storage container needs to frequently exchange data with the editor container, so in order to avoid the large amount of overhead caused by network communication, it is scheduled to the computing node together with the editor container. The database container is a container with data storage function and there is no container that needs to communicate with it frequently, so it is scheduled to run on the storage node alone. The login management container is a container that is not sensitive to computing and storage resources. There is no corresponding label node in the system, so it is scheduled to run on the basic node where various resources are relatively balanced.

[0039] Compared with the traditional scheduling algorithm, the container feature-oriented scheduling algorithm using the Kubernetes cluster based on the pan-inclusive computing architecture has the following innovations: First, it is a container-oriented scheduling method in concept, which can be scheduled according to the feature labels of the containers in the pan-inclusive computing architecture. Each label represents the different functions that each container is mainly responsible for, such as computing function, storage function, network function, etc. According to the different labels, different containers are scheduled to nodes with different resource advantages. For example, containers that provide computing services will be scheduled to nodes with better computing power, and containers that provide storage services will be scheduled to nodes with better storage resources. This scheduling algorithm prioritizes scheduling based on the characteristics of the container rather than the characteristics of the node, and tries to make each container run on a node with better resources for the function it is responsible for.

[0040] Second, because in the Kubernetes cluster based on pan-capable computing, a multi-container collaboration method is used to complete business, it inevitably leads to the consumption of network resources. In the traditional scheduling algorithm, two or more containers that need frequent communication and collaboration may be scheduled to different nodes, which greatly increases the network overhead. This scheduling method schedules these containers that need frequent communication and interaction to the same node in a unified manner, reducing the network overhead caused by their communication. In addition, the collaborative communication and interaction between Docker containers in the cluster adopts the lightweight Flask framework, trying to achieve a lightweight architecture and improve the efficiency of system resource utilization.

[0041] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A cluster scheduling method based on a pan-capable computing architecture, characterized in that: The specific steps include: Step 1: Configure the Kubernetes cluster and define the management node and various working nodes of the Kubernetes cluster, including establishing various Docker images on the management node and working nodes and installing related tools; Step 2: Divide the services into multiple containers with different functions according to different services of users, including containers that provide storage functions and containers that provide computing functions; and deploy these containers on the Kubernetes cluster; Step 3: At the beginning of the scheduling algorithm, first query all available working nodes and record the label information of these working nodes, where the label information of the working nodes includes: computing nodes, storage nodes, and basic nodes; Step 4: Query the labels of all containers to be scheduled, where the labels of the containers include: storage container and computing container; Step 5: Compare the labels of the containers to be scheduled with the labels of the nodes in turn. If there is a matching node, proceed to step 6. If not, schedule the container to the basic node. Step 6: According to the label information, check whether the container to be scheduled has any container that needs to communicate frequently with it. If so, schedule one or more containers corresponding to the container to be scheduled that need to communicate frequently with it to the same working node. If not, directly schedule the container to be scheduled to the working node corresponding to the label of the container. Step 7: Deploy the visualization plug-in of the Kubernetes cluster to visualize the status of all nodes and containers in the Kubernetes cluster.

Images