Scheduling system and method for Redis instance crossing K8s cluster and electronic equipment

Abstract

The invention provides a scheduling system and method for Redis instances of a cross-K8s cluster and electronic equipment. The system comprises a cluster management module, a state monitoring module, a decision engine module, a federal scheduling module and an execution module. And when a user needs to deploy the Redis instance in each K8s cluster, the user initiates a user Redis deployment request to the federated scheduling module and responds to the user Redis cluster deployment request, and the execution module executes the determined target scheduling strategy to deploy the target Redis instance requested to deploy by the user to the target K8s cluster in the target scheduling strategy. And determining an available Redis instance scheduling strategy based on the monitored state of each K8s cluster and the state of the Redis instance deployed in each K8s cluster, and executing the Redis instance scheduling strategy to deploy the target Redis instance to the corresponding target K8s cluster, thereby realizing cross-K8s cluster scheduling and deployment of the Redis instance. The bottleneck problem that the Redis instance cannot be deployed across the K8s cluster in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a scheduling system, method and electronic device for Redis instances across K8s clusters. Background Art

[0002] Redis cluster is a common non-relational storage system that can be deployed in a master-slave architecture, that is, a data storage system is built using an architecture of one master node and multiple slave nodes. Kubernetes (abbreviated as K8s in the industry) is a commonly used tool for managing container clusters. K8s provides a set of unified API data interfaces, which can facilitate unified management and scheduling of services deployed in K8s. Therefore, the industry often uses this feature of K8s to deploy Redis to the K8s cluster to ensure data persistence and high availability.

[0003] However, due to factors such as network latency and resource consumption, the solution for deploying Redis clusters across K8s clusters often has problems such as high network latency and high resource consumption. Therefore, the existing deployment solution for deploying Redis clusters in K8s clusters is often to deploy a Redis cluster in a single K8s cluster, and it is impossible to deploy and schedule Redis instances across K8s clusters. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a scheduling system, method and electronic device for Redis instances across K8s clusters to implement the deployment of Redis clusters across K8s clusters.

[0005] In a first aspect, an embodiment of the present application provides a scheduling system for Redis instances across K8s clusters, the system comprising:

[0006] A cluster management module, a state monitoring module, a decision engine module, a federated scheduling module, and an execution module, wherein the state monitoring module, the cluster management module, and the execution module are all connected to each K8s cluster, wherein each K8s cluster is mixedly deployed with various Redis nodes that conform to the master-slave structure, and the Redis instance is deployed on the master node and the slave node in each of the Redis nodes;

[0007] The cluster management module is used to obtain and manage the status information of each K8s cluster.

[0008] The status monitoring module is used to monitor the cluster performance indicator information of each K8s cluster and the instance performance indicator information of the Redis instance deployed in each K8s cluster in real time;

[0009] The decision engine module is used to determine a target scheduling strategy based on the state information, the cluster performance indicator information, and the instance performance indicator information;

[0010] The federated scheduling module is used to receive a user Redis cluster deployment request and forward the user Redis cluster deployment request to the execution module, wherein the user Redis cluster deployment request carries a target Redis instance for processing the user request;

[0011] The execution module is used to execute the target scheduling strategy and deploy the target Redis instance to the target K8s cluster in the target scheduling strategy.

[0012] Optionally, in some embodiments, the user Redis cluster deployment request carries the total load QPS and memory size information required by the user, and the federated scheduling module is specifically used to:

[0013] Receive the total bearer QPS and memory size information carried by the user Redis cluster deployment request, and determine the priority of the user Redis cluster deployment request.

[0014] Optionally, in some embodiments, the federated scheduling module is specifically used to:

[0015] If the priority of the user's Redis cluster deployment request is high priority, a target exclusive K8s cluster is determined for each target Redis instance of the user's Redis cluster deployment request, and only Redis instances are deployed in the target exclusive K8s cluster;

[0016] If the priority of the user's Redis cluster deployment request is medium priority, the target dedicated K8s cluster is determined for the Redis master instance in the user's Redis cluster deployment request, and the target colocation cluster is determined for the Redis slave instance in the user's Redis cluster deployment request, wherein the target colocation cluster runs Redis instances, databases, and big data tasks;

[0017] If the priority of the user Redis cluster deployment request is a low priority, the target colocation cluster is determined for each target Redis instance in the user Redis cluster deployment request.

[0018] Optionally, in some embodiments, the federated scheduling module is further used to:

[0019] If the priority of the user Redis cluster deployment request is zero, a temporary cluster is determined for each target Redis instance of the user Redis cluster deployment request, wherein when the running time of the temporary cluster meets the preset expiration time condition, the temporary cluster will automatically expire.

[0020] Optionally, in some embodiments, the federated scheduling module is further used to:

[0021] Initiate a Pod query command to each of the K8s clusters to obtain the resource usage of each Pod container deployed in each of the K8s;

[0022] A global resource view is generated according to the resource usage of each Pod container, wherein the global resource view includes: the CPU usage and memory usage of each node in each K8s cluster, and the CPU usage and memory usage of each Pod.

[0023] Optionally, in some embodiments, in response to the user Redis cluster deployment request with a high priority, the federated scheduling module is further configured to:

[0024] Deploy Redis proxy middleware in each computer room, wherein the Redis proxy middleware deployed in each computer room is distributed in different K8s clusters;

[0025] According to the Redis server executor package instance carried in the user's Redis cluster deployment request, determine the normalized CPU and normalized memory that meet the requirements of the Redis server executor package instance;

[0026] Among them, the normalized CPU is the CPU resource calculated according to the available CPU resources of each machine in each target exclusive K8s cluster according to the preset CPU normalization algorithm; the normalized memory is the memory resource calculated according to the available memory resources in each target exclusive K8s cluster according to the preset memory normalization algorithm.

[0027] Optionally, in some embodiments, in response to the user Redis cluster deployment request with a medium priority, the federated scheduling module is specifically configured to:

[0028] Determine the target Node node according to the CPU utilization and remaining memory space of each Node node in each of the target dedicated K8s clusters and each of the target colocation clusters;

[0029] Deploy a target Redis proxy middleware for the user Redis cluster deployment request, and allocate the number of CPU cores occupied by the target Redis server execution program for the user Redis cluster deployment request, wherein the number of CPU cores occupied by the target Redis proxy middleware and the target Redis server execution program is less than a preset CPU core number threshold;

[0030] Control each of the target Node nodes to run offline big data tasks during a preset non-emergency time period.

[0031] Optionally, in some embodiments, if a Redis master database in the target dedicated K8s cluster fails, the execution module is specifically used to:

[0032] Control the Redis sentinel node in the target exclusive K8s cluster where the Redis master database has a fault to perform a master-slave switching operation. If the remaining resources of the target exclusive K8s cluster with a fault are sufficient to run a Redis slave node, a new slave node is generated in the target exclusive K8s cluster with a fault. If the remaining resources of the target exclusive K8s cluster with a fault cannot meet the needs of running a Redis slave node, a new slave node is generated from other target exclusive K8s clusters with sufficient remaining resources; if the remaining resources of each target exclusive K8s cluster do not meet the needs of running a Redis slave node, a new slave node is successfully generated from the target hybrid cluster with sufficient remaining resources;

[0033] The Redis instance on the failed Redis master node is migrated to a new slave node, and the new slave node is determined as a new Redis master node.

[0034] In a second aspect, an embodiment of the present application provides a scheduling method for Redis instances across K8s clusters, the method being applied to a K8s cluster system, wherein the K8s cluster system includes multiple K8s clusters, each of the K8s clusters is mixedly deployed with Redis nodes that conform to a master-slave structure, and the Redis instance is deployed on a master node and a slave node in each of the Redis nodes, the method comprising:

[0035] Obtain and manage the status information of each K8s cluster, and monitor the cluster performance indicator information of each K8s cluster and the instance performance indicator information of the Redis instance deployed in each K8s cluster in real time;

[0036] Determine a target scheduling strategy based on the state information, the cluster performance indicator information, and the instance performance indicator information;

[0037] Receive a user Redis cluster deployment request and forward the user Redis cluster deployment request to the execution module, wherein the user Redis cluster deployment request carries a target Redis instance for processing the user request;

[0038] Execute the target scheduling strategy and deploy the target Redis instance to the target K8s cluster in the target scheduling strategy.

[0039] The user Redis cluster deployment request carries the total load QPS and memory size information required by the user. The federated scheduling module is specifically used to:

[0040] Receive the total bearer QPS and memory size information carried by the user Redis cluster deployment request, and determine the priority of the user Redis cluster deployment request.

[0041] Optionally, in some embodiments, the method further comprises:

[0042] If the priority of the user's Redis cluster deployment request is high priority, a target exclusive K8s cluster is determined for each target Redis instance of the user's Redis cluster deployment request, and only Redis instances are deployed in the target exclusive K8s cluster;

[0043] If the priority of the user's Redis cluster deployment request is medium priority, the target dedicated K8s cluster is determined for the Redis master instance in the user's Redis cluster deployment request, and the target colocation cluster is determined for the Redis slave instance in the user's Redis cluster deployment request, wherein the target colocation cluster runs Redis instances, databases, and big data tasks;

[0044] If the priority of the user Redis cluster deployment request is low priority, determining the target colocation cluster for each target Redis instance in the user Redis cluster deployment request;

[0045] Optionally, in some embodiments, the method further comprises:

[0046] If the priority of the user Redis cluster deployment request is zero, a temporary cluster is determined for each target Redis instance of the user Redis cluster deployment request, wherein when the running time of the temporary cluster meets the preset expiration time condition, the temporary cluster will automatically expire.

[0047] Optionally, in some embodiments, the method further comprises:

[0048] Initiate a Pod query command to each of the K8s clusters to obtain the resource usage of each Pod container deployed in each of the K8s;

[0049] A global resource view is generated according to the resource usage of each Pod container, wherein the global resource view includes: the CPU usage and memory usage of each node in each K8s cluster, and the CPU usage and memory usage of each Pod.

[0050] Optionally, in some embodiments, in response to the user Redis cluster deployment request with a high priority, the method further includes:

[0051] Deploy Redis proxy middleware in each computer room, wherein the Redis proxy middleware deployed in each computer room is distributed in different K8s clusters;

[0052] According to the Redis server executor package instance carried in the user's Redis cluster deployment request, determine the normalized CPU and normalized memory that meet the requirements of the Redis server executor package instance;

[0053] Among them, the normalized CPU is the CPU resource calculated according to the available CPU resources of each machine in each target exclusive K8s cluster according to the preset CPU normalization algorithm; the normalized memory is the memory resource calculated according to the available memory resources in each target exclusive K8s cluster according to the preset memory normalization algorithm.

[0054] Optionally, in some embodiments, in response to the user Redis cluster deployment request with a medium priority, the method further includes:

[0055] Determine the target Node node according to the CPU utilization and remaining memory space of each Node node in each of the target dedicated K8s clusters and each of the target colocation clusters;

[0056] Deploy a target Redis proxy middleware for the user Redis cluster deployment request, and allocate the number of CPU cores occupied by the target Redis server execution program for the user Redis cluster deployment request, wherein the number of CPU cores occupied by the target Redis proxy middleware and the target Redis server execution program is less than a preset CPU core number threshold;

[0057] Control each of the target Node nodes to run offline big data tasks during a preset non-emergency time period.

[0058] Optionally, in some embodiments, if a Redis master database in the target dedicated K8s cluster fails, the method further includes:

[0059] Control the Redis sentinel node in the target exclusive K8s cluster where the Redis master database has a fault to perform a master-slave switching operation. If the remaining resources of the target exclusive K8s cluster with a fault are sufficient to run a Redis slave node, a new slave node is generated in the target exclusive K8s cluster with a fault. If the remaining resources of the target exclusive K8s cluster with a fault cannot meet the needs of running a Redis slave node, a new slave node is generated from other target exclusive K8s clusters with sufficient remaining resources; if the remaining resources of each target exclusive K8s cluster do not meet the needs of running a Redis slave node, a new slave node is successfully generated from the target hybrid cluster with sufficient remaining resources;

[0060] The Redis instance on the failed Redis master node is migrated to a new slave node, and the new slave node is determined as a new Redis master node.

[0061] In a third aspect, an embodiment of the present application provides an electronic device, wherein the electronic device includes:

[0062] Processor; and

[0063] Memory for storing programs,

[0064] Among them, the program includes instructions, which, when executed by the processor, enable the processor to execute the scheduling method for Redis instances across K8s clusters described in the second aspect.

[0065] In a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, characterized in that the computer instructions are used to enable a computer to execute the scheduling method for Redis instances across K8s clusters described in the second aspect.

[0066] Beneficial effects of this application:

[0067] The present application provides a scheduling system, method and electronic device for Redis instances across K8s clusters, wherein the system includes a cluster management module, a state monitoring module, a decision engine module, a federated scheduling module and an execution module, wherein the state monitoring module, the cluster management module and the execution module are all connected to each K8s cluster, and each K8s cluster is mixedly deployed with each Redis node that conforms to the master-slave structure, and the Redis instance is deployed in the master node and the slave node in the Redis node. The cluster management module is used to obtain and manage the status information of each K8s cluster, and the state monitoring module is used to monitor the cluster performance indicator information of each K8s cluster and the instance performance indicator information of the Redis instance deployed in each K8s cluster in real time.

[0068] When the user needs to deploy Redis instances in each K8s cluster, the user initiates a user Redis deployment request to the federated scheduling module. The federated scheduling module responds to the user Redis cluster deployment request and forwards the request to the execution module. The execution module executes the determined target scheduling strategy to deploy the target Redis instance requested by the user to the target K8s cluster in the target scheduling strategy.

[0069] The scheduling system for Redis instances across K8s clusters provided in the embodiment of the present application can monitor the status of each K8s cluster and the Redis instances deployed in each K8s cluster, and then determine the available Redis instance scheduling strategy based on the monitoring results, and execute the Redis instance scheduling strategy to deploy the target Redis instance to the corresponding target K8s cluster, thereby realizing the scheduling and deployment of Redis instances across K8s clusters, solving the bottleneck problem of the prior art that Redis instances cannot be deployed across K8s clusters. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Further details, features and advantages of the present application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0071] Figure 1 A schematic diagram of deploying a Redis instance across a K8s cluster provided in an embodiment of the present application is shown;

[0072] Figure 2 A schematic diagram of the logical structure of a scheduling system for Redis instances across K8s clusters provided in an embodiment of the present application is shown;

[0073] Figure 3 A schematic diagram of a process for scheduling a Redis instance across a K8s cluster provided in an embodiment of the present application is shown;

[0074] Figure 4A logical structure diagram of an electronic device for implementing a scheduling method for Redis instances across K8s clusters provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0075] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0076] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0077] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0078] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0079] Before describing the scheduling system, method, and electronic device for Redis instances across K8s clusters provided by the present application, the professional terms involved in this article are explained here:

[0080] Redis (Remote Dictionary Server) storage system is a popular key-value storage system in the industry. It belongs to the category of non-relational database (NoSQL database). The Redis storage system supports multiple data structures, including string, hash, set, zset, list and other data structures.

[0081] Redis cluster: a large data storage system obtained by connecting multiple Redis nodes together. However, in the embodiment of the present application, the Redis cluster is not the traditional open source Redis-cluster mode. In the embodiment of the present application, the Redis cluster refers to the Redis intermediate proxy layer Router-server (routing service module) and multiple Redis shards. Each Redis shard corresponds to a one-master-and-multiple-slave (at least one master and one slave) architecture, where one master refers to one master node and multiple slaves refer to multiple slave nodes.

[0082] Router-server: The middleware proxy of the Redis cluster is used to receive the user's Redis request and forward the Redis request to the backend Redis cluster according to specific routing rules. The Redis cluster responds to the Redis request and returns the processing result to the user.

[0083] Redis-master: Redis master node, usually used to handle the read and write traffic of upstream Internet services.

[0084] Redis-slave: Redis slave node, generally used as a cold device or to undertake the read traffic of some users. When an abnormality occurs in the master node, a role switch will occur, from the slave node role to the master node role, and execute the corresponding actions of the master node.

[0085] Redis-sentinel: Redis sentinel node, used to detect the health status of the Redis cluster. For example, if one of the master nodes fails or is abnormal, you can try to initiate and execute master-slave switching to reduce the risks caused by failures or abnormalities.

[0086] Container: An independent runtime environment that isolates applications and their dependencies. A container is essentially a group of processes that are resource constrained and isolated from each other.

[0087] Kubernetes: Often referred to as K8s in the industry, it is used to manage container clusters. Kubernetes is divided into Master nodes and Node nodes. Master is the manager role, and Node is the managed role (scheduled). The smallest scheduling management unit of Kubernetes is Pod.

[0088] K8s cluster: an open source container orchestration platform for automating the deployment, expansion, and management of containerized applications. K8s cluster provides a declarative configuration method that allows applications to run on multiple hosts and automatically adjust resource allocation based on load. K8s cluster can manage various types of applications, including web applications, databases, message queues, etc.

[0089] Pod: The smallest deployable unit in Kubernetes, usually containing one or more closely cooperating containers. Usually, multiple Pods can be deployed on a Node, and a Pod can contain multiple containers.

[0090] Containerized Redis: A Redis service instance deployed on Kubernetes.

[0091] Scheduler: Kubernetes scheduler, which can schedule and deploy Pods.

[0092] Karmada: An open source cloud container orchestration platform that can be used to manage multi-cluster applications.

[0093] Redis cluster hosted in K8s cluster: Deploy the Redis cluster as a containerized application in the Kubernetes environment, and manage and schedule it uniformly by the K8s cluster, which can help ensure data persistence and high availability. However, the current industry's scheduling capabilities for Redis clusters hosted in K8s are limited to a single K8s cluster, that is, it only supports Redis cluster scheduling within a single K8s cluster, and cannot support the ability to schedule Redis across multiple K8s clusters. At this time, the entire Redis cluster can only be deployed in a K8s cluster and scheduled in a single K8s cluster.

[0094] In order to overcome the problem of not being able to schedule Redis across multiple K8s clusters, the industry uses an open source cloud container orchestration platform such as Karmada to schedule Redis clusters across K8s clusters. However, the Karmada orchestration platform only supports stateless services and cannot support scheduling for stateful services. Stateless service means that a single service request is processed independently and does not depend on other requests. To process a stateless service request in this way, the stateless service request is required to contain all the information required to process the stateless service request, and there is no need to obtain the information required to process the stateless service request from an external database. Stateful services are completely opposite to stateless services. There is a correlation between stateful service requests. Although they will store some of the data required to process the stateful service request, there is still a need to obtain the data required to process the service request from an external database.

[0095] In view of this, the present application provides a scheduling system, method and electronic device for Redis instances across K8s clusters, wherein the scheduling method for Redis instances across K8s clusters is applied to any electronic device capable of scheduling Redis instances across K8s clusters, and the types of the electronic devices include but are not limited to: personal mobile terminals, computers, servers, server cluster management devices, etc. Among them, the scheduling method for Redis instances across K8s clusters can be specifically applied to the scheduling system for Redis instances across K8s clusters. Whether it is a scheduling system for Redis instances across K8s clusters or a scheduling method for Redis instances across K8s clusters, the implementation basis is that the Redis cluster is deployed across K8s clusters in the K8s cluster.

[0096] In this application, deploying a Redis cluster across K8s clusters means that the same Redis cluster is divided into different Redis nodes, and each Redis node is deployed in a different K8s cluster. Figure 1 As shown, there are three K8s clusters in total: K8s-1 cluster, K8s-2 cluster and K8s-3 cluster, and then the Redis cluster: the Redis-master master node and the slave nodes (Redis-slave-1 to Redis-slave-3) belonging to the Redis master node are deployed in different K8s clusters. Specifically, the master node Redis-master and the slave node Redis-slave-3 of the Redis cluster are deployed in the K8s-1 cluster, Redis-slave-1 is deployed in the K8s-2 cluster, and Redis-slave-2 is deployed in the K8s-3 cluster.

[0097] In addition, each K8s cluster also deploys a Router-sever proxy middleware and a Redis-sentinel sentinel node. The Router-sever proxy middleware is responsible for forwarding Redis instances, and the Redis-sentinel sentinel node monitors the status of Redis nodes deployed in the same K8s cluster.

[0098] In the embodiment of the present application, Redis instance refers to: a service instance deployed on a Redis cluster, wherein a service instance refers to a virtual server deployed in a server or cloud environment and run by the server or cloud environment, each service instance has independent computing, storage and network resources, and can independently run operating systems and applications. In the embodiment of the present application, Redis service instance specifically refers to a Redis database instance running in a server or cloud environment, each Redis service instance provides high-performance Key-Value key-value pair storage, and can independently perform data operations and management. Since the Redis service instance is deployed in a Redis cluster, and the Redis cluster is deployed in different K8s clusters in a cross-K8s cluster manner, the Redis service instance is naturally deployed and runs in a K8s cluster in a cross-K8s cluster manner.

[0099] Based on this, in the first aspect, the embodiment of the present application provides a scheduling system for Redis instances across K8s clusters, which can be Figure 2 As shown, it includes the following parts:

[0100] Cluster management module 21, status monitoring module 22, decision engine module 23, federated scheduling module 24, and execution module 25.

[0101] The status monitoring module, the cluster management module, and the execution module are all connected to each K8s cluster, wherein each K8s cluster is mixedly deployed with various Redis nodes that conform to the master-slave structure, and the Redis instance is deployed on the master node and the slave node in each of the Redis nodes;

[0102] The cluster management module is used to obtain and manage the status information of each K8s cluster.

[0103] The status monitoring module is used to monitor the cluster performance indicator information of each K8s cluster and the instance performance indicator information of the Redis instance deployed in each K8s cluster in real time;

[0104] The decision engine module is used to determine a target scheduling strategy based on the state information, the cluster performance indicator information, and the instance performance indicator information;

[0105] The federated scheduling module is used to receive a user Redis cluster deployment request and forward the user Redis cluster deployment request to the execution module, wherein the user Redis cluster deployment request carries a target Redis instance for processing the user request;

[0106] The execution module is used to execute the target scheduling strategy and deploy the target Redis instance to the target K8s cluster in the target scheduling strategy.

[0107] When the user needs to deploy Redis instances in each K8s cluster, the user initiates a user Redis deployment request to the federated scheduling module. The federated scheduling module responds to the user Redis cluster deployment request and forwards the request to the execution module. The execution module executes the determined target scheduling strategy to deploy the target Redis instance requested by the user to the target K8s cluster in the target scheduling strategy.

[0108] The scheduling system for Redis instances across K8s clusters provided in the embodiment of the present application can monitor the status of each K8s cluster and the Redis instances deployed in each K8s cluster, and then determine the available Redis instance scheduling strategy based on the monitoring results, and execute the Redis instance scheduling strategy to deploy the target Redis instance to the corresponding target K8s cluster, thereby realizing the scheduling and deployment of Redis instances across K8s clusters, solving the bottleneck problem of the prior art that Redis instances cannot be deployed across K8s clusters.

[0109] The following is a detailed description of each module of the scheduling system for Redis instances across K8s clusters provided in an embodiment of the present application:

[0110] In an embodiment of the present application, each Redis node that conforms to the master-slave structure is a database architecture with one master and multiple slaves, wherein the Redis node types as described above include: a Redis master node and each Redis slave node with the Redis master node, in addition, a Redis sentinel node is also included, wherein the nodes that respond to data requests (including read requests and write requests) are Redis master nodes and Redis slave nodes, and the specific execution actions of the master node and the slave node are determined by the scheduling strategy. Among them, the Redis master node and the Redis slave node are essentially databases or execution devices. Based on this, in an embodiment of the present application, the Redis master node can be regarded as a Redis master database, and the Redis slave node can be regarded as a Redis slave database. The Redis cluster across the K8s cluster can be understood as using the Kubernetes cluster management technology to schedule the processing resources of each database in the Redis cluster so that each Redis instance can operate normally. In order to ensure the high availability of the entire Redis cluster, resources will be scheduled for a master node under a single Redis shard and multiple slave nodes corresponding to the master node, and the master and multiple slave nodes under the single Redis shard will be scattered to different K8s clusters. Each K8s cluster is stored in a physical computer room, and each physical computer room must ensure that there are slave nodes corresponding to the single Redis shard, that is, slave Redis instances, so that when a slave Redis instance fails, the Redis instance can be replaced.

[0111] In the embodiment of the present application, the association between the Redis node and the Redis instance can be understood as the Redis node being the physical cornerstone for the operation of the Redis instance. The Redis instance is at the software level, and the Redis node is at the hardware level. The Redis instance requires the processing resources and storage resources of the Redis node to operate normally. From the perspective of the data room, the Redis master node and the Redis slave node should be distributed in different data rooms. Among them, one Redis node can be deployed to run a Redis instance, or multiple Redis nodes can cooperate with each other to run a Redis instance. The specific number of Redis nodes running the Redis instance needs to be flexibly set according to the actual data reading and writing and computing requirements of the Redis instance, and this application does not make strict restrictions.

[0112] As an implementation method, the scheduling system for Redis instances across K8s clusters provided in the embodiment of the present application can be a software system, which is installed in the management device of the database system of the K8s cluster. There is a communication connection between the management device and each database of the K8s cluster at the back end, and the back end database is monitored in real time and status information is collected based on the communication connection. Specifically, the cluster management module in the scheduling system for Redis instances across K8s clusters is connected to each database at the back end, and the cluster management module obtains and manages the status information of each K8s cluster. Among them, the status information of each K8s cluster includes: CPU usage, memory usage, network IO (Input Ouput, input and output) usage, and disk IO usage.

[0113] As an implementation method, the status information of each K8s cluster can be obtained in real time. As another implementation method, the status information of each K8s cluster can be obtained periodically through regular polling. Similarly, the status monitoring module can also monitor the cluster performance indicator information of each K8s cluster and the instance performance indicator information of the Redis instance deployed in each K8s cluster in real time, or it can obtain the cluster performance indicator information of each K8s cluster and the instance performance indicator information of the Redis instance deployed in each K8s cluster periodically through regular polling. Among them, the regular polling method specifically refers to the cluster management module and the status monitoring module regularly sending query requests to each K8s cluster to obtain the status information or performance indicator information returned by the K8s cluster.

[0114] In an embodiment of the present application, the cluster performance indicator information of each K8s cluster may be the same as the status information of each K8s cluster, and the performance of each K8s cluster is determined by status information such as CPU usage and memory usage. As an implementation method, the cluster performance indicator information of the K8s cluster may include indicator information such as communication packet loss rate, communication QPS (Query per second), load avg average load, etc. on the basis of the status information of the K8s cluster. As an implementation method, the instance performance indicator information of the Redis instance includes: the process level of the Router-sever proxy middleware, the CPU utilization of the Router-sever proxy middleware, the process level of Redis-sever (Redis server executor), the CPU utilization of Redis-sever, and the memory utilization of Redis-sever.

[0115] In the embodiment of the present application, the status information and performance indicator information obtained by the cluster management module and the status monitoring module will be uploaded to a pre-specified time series database, and the subsequent decision engine module can determine the target scheduling strategy by calling the data in the time series database. Among them, the pre-specified time series database can be an influxdb database.

[0116] In addition to the cluster management module and the status monitoring module being able to obtain the status information, cluster performance indicator information, and instance performance indicator information of the backend K8s cluster in real time, the federated scheduling module provided in the embodiment of the present application can also establish a communication connection with the backend K8s cluster to obtain the resource usage of each container in the K8s cluster. In some embodiments, the federated scheduling module is specifically used to:

[0117] Initiate a Pod query command to each of the K8s clusters to obtain the resource usage of each Pod container deployed in each of the K8s;

[0118] A global resource view is generated according to the resource usage of each Pod container, wherein the global resource view includes: the CPU usage and memory usage of each node in each K8s cluster, and the CPU usage and memory usage of each Pod.

[0119] Specifically, the federal scheduling module can initiate a Pod query request to the execution module, and the execution module generates a Pod query instruction based on the Pod query request and forwards the Pod query instruction to each connected K8s cluster. After receiving the Pod query instruction, each K8s cluster will summarize the resources it occupies and return the summarized resource status to the execution module.

[0120] In the embodiment of the present application, after the cluster is deployed, it can initiate hosting to the execution module, and the execution module will aggregate the description information of each hosted Pod into the same Namespace space, and the execution module will forward the corresponding operation instructions to each Pod in the hosted cluster according to the information in the Namespace space. At this time, after receiving the Pod query instruction initiated by the federal scheduling module, the execution module can initiate a GetPods operation to all hosted clusters, collect and aggregate the Pod information in each Namespace space under all K8s clusters, and then transmit it back to the federal scheduling module.

[0121] Based on this, the federated scheduling module can generate a global resource view based on the information returned by the execution module. The global resource view clearly and intuitively displays the specific resource usage of each node and each Pod in the current K8s cluster, which can be specifically reflected in the CPU usage and memory usage, so that resources can be allocated according to the available resources and Redis instances can be scheduled so that each Redis instance can be scheduled to a node or Pod with sufficient resources.

[0122] In the embodiment of the present application, the federated scheduling module opens a data interface for interaction with the user. The user can upload the user Redis cluster deployment request to the federated scheduling module through the data interface according to the deployment requirements of the Redis service instance. The user Redis cluster deployment request indicates the CPU resources, memory resources and QPS required for deploying the required Redis instance. The federated scheduling module receives the user Redis cluster deployment request uploaded by the user and parses it to obtain the required CPU resources, memory resources and QPS required to be carried. As an implementation method, the user Redis cluster deployment request submitted by the user is not limited to adding a new Redis instance, but can also be a request for migration, deletion, expansion and other operations on an existing deployed Redis instance. For operations such as migration, deletion, and expansion of an existing Redis instance, the user Redis cluster deployment request uploaded by the user should indicate the operation type of the Redis instance and the Redis instance object to be migrated, deleted, and expanded, so that the federated scheduling module can parse the user Redis cluster deployment request to clarify the user's specific operation intention.

[0123] In an embodiment of the present application, there is data interaction between the federal scheduling module and the execution module. The federal scheduling module forwards the specific content that the execution module needs to execute to the execution module, and then the execution module performs the corresponding operations. For example, the actual operations such as migration, deletion, and expansion are actually performed by the federal scheduling module instructing the execution module, and the specific execution operations are implemented by the execution module.

[0124] In some embodiments, the user's Redis cluster deployment request carries the total load QPS and memory size information required by the user. In this case, the federated scheduling module is specifically used to:

[0125] Receive the total bearer QPS and memory size information carried by the user Redis cluster deployment request, and determine the priority of the user Redis cluster deployment request.

[0126] In the embodiment of the present application, the user Redis cluster deployment request passed in by the user can be simply understood as the Redis package submitted by the user. The user can clearly write the required operations, operation objects, processing resources and storage resources required to operate the operation objects, QPS required by the system, etc. in the Redis package. Then, the federal scheduling module determines the priority corresponding to the user Redis cluster deployment request based on the Redis package, especially determines the priority of the Redis cluster requested by the user.

[0127] In the embodiment of the present application, the Redis cluster will be prioritized in advance, and the missions undertaken by the Redis clusters corresponding to different priorities are different. As a preferred implementation, the priorities of the Redis cluster can be divided from high to low into: high priority, medium priority, low priority and zero level. In the embodiment of the present application, the basis for the priority division is flexibly divided by the user according to the software operation requirements required by his own business, and this application does not make strict restrictions. Among them, the high-priority Redis cluster serves the high-priority Redis instance, that is to say, only the high-priority Redis instance is deployed in the high-priority Redis cluster. In addition to deploying Redis instances, the medium-priority Redis cluster can also deploy other contents such as databases and large database tasks according to actual needs.

[0128] In the embodiments of the present application, the priority of the Redis instance can also be divided according to factors such as the urgency of the business and the time period required to run, which is not strictly limited in the present application. As an example, a high-priority Redis instance may refer to a Redis instance corresponding to a business service that needs to run 24 hours a day and requires sufficient running resources. For example, taking a business platform such as a music player as an example, the stability of the database storing music is extremely important, and the database storing music needs to respond to the user's playback request at any time. At this time, the Redis instance storing music can be determined as a high-priority Redis. For advertising materials on the music player, there is no need to respond 24 hours a day, so the Redis instance storing advertising materials on the music player is classified as a medium-priority Redis instance.

[0129] Based on this, in some embodiments, the federated scheduling module is specifically used to:

[0130] If the priority of the user's Redis cluster deployment request is high priority, a target exclusive K8s cluster is determined for each target Redis instance of the user's Redis cluster deployment request, and only Redis instances are deployed in the target exclusive K8s cluster;

[0131] If the priority of the user's Redis cluster deployment request is medium priority, the target dedicated K8s cluster is determined for the Redis master instance in the user's Redis cluster deployment request, and the target colocation cluster is determined for the Redis slave instance in the user's Redis cluster deployment request, wherein the target colocation cluster runs Redis instances, databases, and big data tasks;

[0132] If the priority of the user Redis cluster deployment request is a low priority, the target colocation cluster is determined for each target Redis instance in the user Redis cluster deployment request.

[0133] In an embodiment of the present application, the Redis cluster that responds to the Redis instance with high service priority is determined as the target exclusive K8s cluster. Only Redis instances are deployed in the target exclusive K8s cluster, and only the user's Redis service request is responded to. However, the operating fee of the target exclusive K8s cluster is relatively high. Since the mixed cluster supports the deployment of Redis instances in addition to the deployment of Redis instances, it also supports the deployment of instances of other types of databases, and supports the operation of big data processing tasks. Therefore, compared with the exclusive K8s cluster, the fee is cheaper. At this time, the Redis master instance in the medium-priority user Redis cluster deployment request is deployed in the target exclusive K8s cluster, and the Redis slave instance is deployed in the target mixed cluster, which can save deployment fees and make the entire medium-priority Redis instance deployment cost medium.

[0134] In addition, even if the priority of a Redis instance is low, the low-priority Redis instance, whether it is a master instance or a slave instance, will be deployed to the target co-location cluster. No additional cluster will be opened to handle the low-priority Redis instance separately, which helps to improve the resource utilization of the cluster and save deployment fees.

[0135] In addition, zero-level priority is a special priority, usually for the temporary needs of development or testing users during the R&D and testing process. This type of zero-level priority Redis deployment request is often short-lived and only runs for a period of time. After the expiration, it will be automatically destroyed or released. Based on this, in some embodiments, the federated scheduling module is also used to:

[0136] If the priority of the user Redis cluster deployment request is zero, a temporary cluster is determined for each target Redis instance of the user Redis cluster deployment request, wherein when the running time of the temporary cluster meets the preset expiration time condition, the temporary cluster will automatically expire.

[0137] In an embodiment of the present application, if the user's Redis cluster deployment request is a deployment request with a zero priority, a temporary processing cluster can be temporarily allocated for the deployment request. The user's Redis cluster deployment request can carry the time when the user applies to use the temporary cluster, and then generate a preset expiration time condition for the temporary cluster. When the preset expiration time condition is reached, the temporary cluster can automatically expire, and the processing resources and memory resources of the temporary cluster can be automatically released to avoid the temporary cluster being occupied for a long time, which is conducive to improving the resource utilization of the K8s cluster.

[0138] In an embodiment of the present application, the scheduling of Redis instances can occur after the user inputs a user Redis cluster deployment request to deploy, migrate, delete, etc. the Redis instances already deployed in the K8s cluster. That is to say, when the scheduling system receives the user Redis cluster deployment request passed in by the user, it will automatically trigger the federal scheduling module, and the federal scheduling module will process the response of the Redis instance according to the specific resource conditions of each K8s cluster.

[0139] As an implementation method, when the user has not input a user Redis cluster deployment request, adaptive adjustments can be made based on the operating status of the Redis instances in the K8s cluster. For example, if a processing resource shortage warning occurs during the operation of one of the Redis instances, the Redis instance with the resource shortage warning can be automatically adjusted based on the available processing resources.

[0140] In the embodiment of the present application, the target scheduling strategy is different according to the priority of the Redis cluster, and the scheduling strategy of the corresponding Redis instance is also different. However, before determining the target scheduling strategy, it is necessary to comprehensively evaluate the resources of the Redis cluster. When comprehensively evaluating the resources of the Redis cluster, the corresponding evaluation methods will also be different for Redis clusters with different priorities.

[0141] Among them, for high-priority Redis clusters, the two conditions of replica redundancy and capacity redundancy need to be met. Among them, replica redundancy refers to the sufficient number of slave nodes in each target deployment K8s cluster, and capacity redundancy refers to the sufficient remaining processing resource capacity and memory capacity in each target deployment K8s cluster. The specific sufficient conditions can be flexibly set according to actual business needs, and this application does not strictly limit them.

[0142] Specifically, in some embodiments, in response to a high-priority user Redis cluster deployment request, the federated scheduling module schedules and deploys the high-priority Redis cluster in the following manner:

[0143] Deploy Redis proxy middleware in each computer room, wherein the Redis proxy middleware deployed in each computer room is distributed in different K8s clusters;

[0144] According to the Redis server executor package instance carried in the user's Redis cluster deployment request, determine the normalized CPU and normalized memory that meet the requirements of the Redis server executor package instance;

[0145] Among them, the normalized CPU is the CPU resource calculated according to the available CPU resources of each machine in each target exclusive K8s cluster according to the preset CPU normalization algorithm; the normalized memory is the memory resource calculated according to the available memory resources in each target exclusive K8s cluster according to the preset memory normalization algorithm.

[0146] In the embodiment of the present application, Redis proxy middleware should be deployed in each K8s cluster. If each computer room corresponds to a Redis proxy middleware, it is necessary to ensure that each Redis proxy middleware should be distributed in different K8s clusters to ensure that each K8s cluster has a proxy middleware responsible for routing forwarding. Among them, in the embodiment of the present application, the Redis proxy middleware is the aforementioned Router-server. Since the role of Router-server is to route and distribute requests, the carrying capacity of Router-server determines the carrying capacity of the entire Redis cluster. Based on this, it is necessary to give priority to the carrying capacity of Router-server. Before ensuring the carrying capacity of Router-server, it is necessary to evaluate the Router-server instances that may be needed. Specifically, it can be allocated through the user Redis cluster deployment request package submitted by the user.

[0147] As a preferred implementation, it can be allocated according to the package of 1000QPS corresponding to 0.1 yuan / 100MB. And it is necessary to ensure that each computer room has a corresponding Router-server instance, and the Router-servers in each computer room are distributed in different K8s clusters, and even distributed on different machine racks.

[0148] In an embodiment of the present application, the Redis server execution program may specifically be Redis-server. For the scheduling and deployment of the Redis-server submitted by the user, the main scheduling object is the exclusive K8s cluster where only the Redis instance is deployed. Due to the different service life of different machines and equipment and the different remaining memory, the corresponding performance is also different. As an example, for example, the user needs a total of 2 cores of computing power, and needs to use a 1.2-core CPU on machine A (machine A is an old machine), and needs to use DDR (Double Data Rate) 4 memory, and use 0.8 cores of computing power on machine B (machine B is a new machine) and DDR5 memory. At this time, the 2-core computing power of the old model can only be equivalent to the 1-core computing power of the new model. At this time, the simple 1.2 cores + 0.8 cores to obtain the computing power result cannot support the user's 2-core computing needs.

[0149] In this way, a normalized evaluation should be performed on the CPU and memory in each cluster to determine the normalized CPU and memory required by the Redis-server. The normalized CPU specifically refers to the CPU computing power obtained by performing a normalized algorithm calculation based on the CPU of each machine device, and the normalized memory specifically refers to the memory running speed obtained by performing a normalized algorithm calculation based on the memory of each machine device.

[0150] As an implementation method, the preset CPU normalization algorithm is as follows:

[0151] Step 1-1, obtain the CPU model, number of cores, whether it supports hyperthreading, main frequency, CPU architecture and CPU performance score, and define basic parameters based on the information obtained. Among them, the CPU performance score includes: single-core performance score SPS, multi-core performance score MPS. The single-core performance score SPS is specifically a score obtained by a single-core benchmark test method, and the multi-core performance score is a score obtained by a multi-core benchmark test method. As a preferred implementation, the single-core benchmark test method can use the Geekbench5 test method, and the multi-core benchmark test method can use the Cinebench R23 test method.

[0152] Step 1-2, calculate the adjusted multi-core performance score. Specifically, the adjusted multi-core performance score AMPS (Adjusted MPS) is calculated using the following formula:

[0153]

[0154] MPS is the multi-core performance score of the machine obtained by the multi-core benchmark test method, K is the correction factor, and the correction factor K is a number less than 1, which is used to simulate the nonlinear growth of multi-core processor performance. C is the number of CPU cores, and as the number of cores increases, gradually decreases, and It gradually increases, but the growth rate slows down gradually, thus simulating the nonlinear growth of multi-core performance.

[0155] Step 1-3: Calculate the normalized computing power score. Specifically, the normalized computing power score NCP can be calculated using the following formula:

[0156] NCP=α×SPS+β×AMPS

[0157] Among them, α and β are weight coefficients, which are used to balance the importance of the single-core performance score and the corrected multi-core performance score in the normalized computing power score. The values ​​of the two can be flexibly adjusted according to specific application scenarios and needs, and this application does not impose strict limitations.

[0158] As an implementation method, the preset memory normalization algorithm is specifically as follows:

[0159] Step 2-1. Get the memory parameters: memory model, total memory capacity, memory operating speed, memory CAS (Compare-And-Swap) latency, and memory bandwidth. Memory model (Type): a categorical variable representing different types of memory (such as DDR4, DDR5, etc.). Total memory capacity (Capacity): in GB, indicating the total storage capacity of the memory. Memory operating speed (Speed): in MHz, indicating the transfer rate of the memory. Memory CAS latency (CAS Latency): in the number of clock cycles, indicating the latency of the memory read operation. Memory bandwidth (Bandwidth): in GB / s, indicating the amount of data that the memory can transmit per second.

[0160] Step 2-2: Standardize the memory parameters. Since the dimensions and value ranges of the above memory parameters are different, it is necessary to standardize the above memory parameters first so that they can be fairly compared and combined in subsequent calculations. Specifically:

[0161] Total memory capacity (Capacity): You can use the original value directly, or perform a logarithmic transformation as needed to better reflect the impact of capacity growth on performance.

[0162] For memory operating speed (Speed): it can be standardized by dividing it by a base frequency (such as 2400MHz).

[0163] For memory CAS latency (CAS Latency): the reciprocal of the latency can be converted into a quantity representing the response speed and appropriately scaled.

[0164] For memory bandwidth, it can be normalized by dividing it by a benchmark bandwidth (such as 19.2GB / s).

[0165] Step 2-3: Calculate the normalized memory performance score. The memory performance score NMPS is calculated according to the following formula:

[0166]

[0167] Among them, ω1, ω2, ω3, and ω4 are weight coefficients, which are used to adjust the contribution of total memory capacity, memory speed, memory CAS latency, and memory bandwidth to NMPS. ref It is a reference value of the total memory capacity. You can select a typical capacity value. ref It is a reference value of memory operating speed, and you can select a typical frequency value. scaling Bandwidth is a scaling factor used to adjust the impact of CAS latency in the normalized score. ref It is a reference value of memory bandwidth. You can select a typical bandwidth value.

[0168] In some embodiments, the normalized CPU and the normalized memory may be further weighted and summed, wherein the normalized CPU and the normalized memory each account for 50% of the weight, and finally the normalized resource value of each machine is obtained, so that the federated scheduling module can determine the appropriate machine device for Redis instance deployment according to the normalized resource value.

[0169] Among them, considering that Redis-server involves replica backup redundancy, the federal scheduling module needs to ensure that the Redis slave nodes and the master nodes are deployed in different K8s clusters in a staggered manner during the Redis instance scheduling process. Specifically, the federal scheduling module can set a mutually exclusive condition, which stipulates that the scheduling deployment of the slave nodes in the same computer room must ensure that they do not belong to the same machine device as the master node. Specifically, it is also necessary to ensure that the master node and the slave node do not belong to the same interactive machine, nor are they located in the same rack position. In addition, a separate backup database must be allocated across computer rooms for cold backup to cope with computer room switching in the event of a single computer room failure.

[0170] In some embodiments, in response to a medium priority user Redis cluster deployment request, the federated scheduling module is specifically configured to:

[0171] Determine the target Node node according to the CPU utilization and remaining memory space of each Node node in each of the target dedicated K8s clusters and each of the target colocation clusters;

[0172] Deploy a target Redis proxy middleware for the user Redis cluster deployment request, and allocate the number of CPU cores occupied by the target Redis server execution program for the user Redis cluster deployment request, wherein the number of CPU cores occupied by the target Redis proxy middleware and the target Redis server execution program is less than a preset CPU core number threshold;

[0173] Control each of the target Node nodes to run offline big data tasks during a preset non-emergency time period.

[0174] As an implementation method, for the medium-priority user Redis cluster deployment request, the Redis instance is scheduled specifically through the following steps:

[0175] First, evaluate the CPU utilization and remaining memory space (i.e., available memory space) on each Node in each dedicated K8s cluster and colocation cluster. Then determine the average load of each Node in the period at a fixed period, such as the average load avg of each Node every 15 minutes. In this process, two types of Nodes need to be filtered out: 1: Nodes that cannot support Redis package insertion, that is, no resources respond to Redis package services, or 2: Nodes whose average load in the past 15 minutes exceeds the CPU core.

[0176] Then, the number of CPU cores is allocated to the medium-priority target Redis proxy middleware and the target Redis server executor Redis-server. However, the number of CPU cores of both the target Redis proxy middleware and the target Redis server executor Redis-sever should not be too large, and needs to be limited within the preset CPU core number threshold. The preset CPU core number threshold can be flexibly set according to actual experience, preferably 1, and then the allocated CPU core number is bound to the corresponding Redis proxy middleware and Redis-server.

[0177] Finally, after the Redis-server is deployed, the working hours of the Node node corresponding to the medium-priority Redis instance should be controlled, and other resources should be prevented from running during this time period, so as to ensure the operation of the medium-priority Redis instance. Based on this, a preset non-emergency time period can be set according to the peak business time, and then the target Node node can be controlled to run offline big data tasks during the preset non-emergency time period. As a preferred implementation method, for example, offline big data computing tasks are not allowed to be scheduled to the Node node from 8 am to 8 pm, so as to avoid offline big data computing tasks occupying the resources of the medium-priority Redis instance.

[0178] As an implementation method for low-priority Redis deployment requests and level-0 Redis deployment requests, the Redis memory can be divided into 100MB to run an instance. For example, if a user applies to use 2GB of Redis memory, the 2GB will be divided into nearly 20 100MB packages. This can make full use of the cluster's resource fragments, and then each package will be evenly deployed in various non-exclusive clusters.

[0179] Specifically, for low-priority Redis scheduling deployment: when the remaining memory space of the Node node is sufficient (not less than 30% of the entire machine), only CPU over-allocation is performed. For zero-level Redis scheduling deployment, the specific strategy is: only deployment is allowed on test machines / offline environments, and the remaining CPU / memory fragments are used to fill in the gaps. Generally, random selection of Node nodes is selected. In addition, zero-level Redis uses the persistent memory area on the Node, not the DRAM flash area.

[0180] The issuance and execution of scheduling strategies are ultimately completed by the executor. In the embodiment of the present application, the executor is actually connected to multiple K8s clusters. The scheduling strategy will be accompanied by relevant information of the K8s cluster (such as routing path). The executor then sends the relevant information to the corresponding K8s cluster based on the relevant information. The specific execution component in the corresponding K8s cluster executes the corresponding scheduling strategy. For example, the Kube-APIserver component executes the specific scheduling strategy.

[0181] Due to various potential network attacks, physical failures and other factors, the cluster may fail. Based on this, in some embodiments, if the Redis master database in the target dedicated K8s cluster fails, the execution module is specifically used to:

[0182] Control the Redis sentinel node in the target exclusive K8s cluster where the Redis master database has a fault to perform a master-slave switching operation. If the remaining resources of the target exclusive K8s cluster with a fault are sufficient to run a Redis slave node, a new slave node is generated in the target exclusive K8s cluster with a fault. If the remaining resources of the target exclusive K8s cluster with a fault cannot meet the needs of running a Redis slave node, a new slave node is generated from other target exclusive K8s clusters with sufficient remaining resources; if the remaining resources of each target exclusive K8s cluster do not meet the needs of running a Redis slave node, a new slave node is successfully generated from the target hybrid cluster with sufficient remaining resources;

[0183] The Redis instance on the failed Redis master node is migrated to a new slave node, and the new slave node is determined as a new Redis master node.

[0184] In an embodiment of the present application, for a high-priority Redis master node failure, the sentinel node performs master-slave switching, determines a new slave node that can be used as the master node based on the specific remaining resource situation, and then performs data migration. Among them, priority is given to trying to determine a new slave node in this cluster. If the resources in this cluster cannot meet the needs of generating a new slave node, the federal scheduler will separately deploy and schedule a new slave node from another k8s exclusive cluster of the same type. If the remaining space of other equivalent K8s exclusive clusters still cannot meet the needs, it will try to evict some Pod instances of the Node node in the target colocation cluster, free up CPU and memory space for the slave node to run, and regularly check the remaining resources of the exclusive cluster, and migrate back once there is space.

[0185] In addition, in response to a medium-priority Redis master node failure, the embodiment of the present application will also give priority to deriving a new slave node in the cluster. If it cannot be met, the federated scheduler will deploy and schedule a new slave node from another k8s exclusive cluster of the same type. If it still cannot be met, it will try to evict offline big data tasks in the node to free up space for the redis slave node.

[0186] To deal with low-priority or zero-level Redis master node failures, specifically:

[0187] For low priority: When the master node fails, master-slave switching will be implemented first, and then a new slave node will be added to the remaining space in the same cluster. If the resources are insufficient, the existing Pod instances on any Node node will not be evicted. Instead, new Pod instance resources will be scheduled to other co-located k8s clusters to meet the needs. If the global space is insufficient, the federated scheduling will retry scheduling within the same cluster, and the frequency will be maintained at 1 time / 1 minute.

[0188] For zero priority: failure of the master node, if the Node node is not faulty, the Pod instance will be tried to be directly restarted (zero-level redis will not configure the corresponding slave node resources). If the Node node fails, a new Pod instance resource will be allocated in the remaining k8s cluster to deploy the redis master node to replace the previously hung master node (at this time, the newly created redis instance data is empty, zero-level redis will not reserve slave nodes, only the master node layer is retained).

[0189] In the second aspect, the present application provides a scheduling method for Redis instances across K8s clusters, and the method is applied to a K8s cluster system, wherein the K8s cluster system includes multiple K8s clusters, and each of the K8s clusters is mixedly deployed with Redis nodes that conform to the master-slave structure, and the Redis instance is deployed in the master node and the slave node in each of the Redis nodes. Figure 3 As shown, the method comprises the following steps:

[0190] S31. Obtain and manage the status information of each K8s cluster, and monitor the cluster performance indicator information of each K8s cluster and the instance performance indicator information of the Redis instance deployed in each K8s cluster in real time;

[0191] S32, determining a target scheduling strategy based on the state information, the cluster performance indicator information, and the instance performance indicator information;

[0192] S33, receiving a user Redis cluster deployment request and forwarding the user Redis cluster deployment request to the execution module, wherein the user Redis cluster deployment request carries a target Redis instance for processing the user request;

[0193] S34. Execute the target scheduling strategy and deploy the target Redis instance to the target K8s cluster in the target scheduling strategy.

[0194] The user Redis cluster deployment request carries the total load QPS and memory size information required by the user. The federated scheduling module is specifically used to:

[0195] Receive the total bearer QPS and memory size information carried by the user Redis cluster deployment request, and determine the priority of the user Redis cluster deployment request.

[0196] Optionally, in some embodiments, the method further comprises:

[0197] If the priority of the user's Redis cluster deployment request is high priority, a target exclusive K8s cluster is determined for each target Redis instance of the user's Redis cluster deployment request, and only Redis instances are deployed in the target exclusive K8s cluster;

[0198] If the priority of the user's Redis cluster deployment request is medium priority, the target dedicated K8s cluster is determined for the Redis master instance in the user's Redis cluster deployment request, and the target colocation cluster is determined for the Redis slave instance in the user's Redis cluster deployment request, wherein the target colocation cluster runs Redis instances, databases, and big data tasks;

[0199] If the priority of the user Redis cluster deployment request is low priority, determining the target colocation cluster for each target Redis instance in the user Redis cluster deployment request;

[0200] Optionally, in some embodiments, the method further comprises:

[0201] If the priority of the user Redis cluster deployment request is zero, a temporary cluster is determined for each target Redis instance of the user Redis cluster deployment request, wherein when the running time of the temporary cluster meets the preset expiration time condition, the temporary cluster will automatically expire.

[0202] Optionally, in some embodiments, the method further comprises:

[0203] Initiate a Pod query command to each of the K8s clusters to obtain the resource usage of each Pod container deployed in each of the K8s;

[0204] A global resource view is generated according to the resource usage of each Pod container, wherein the global resource view includes: the CPU usage and memory usage of each node in each K8s cluster, and the CPU usage and memory usage of each Pod.

[0205] Optionally, in some embodiments, in response to the user Redis cluster deployment request with a high priority, the method further includes:

[0206] Deploy Redis proxy middleware in each computer room, wherein the Redis proxy middleware deployed in each computer room is distributed in different K8s clusters;

[0207] According to the Redis server executor package instance carried in the user's Redis cluster deployment request, determine the normalized CPU and normalized memory that meet the requirements of the Redis server executor package instance;

[0208] Among them, the normalized CPU is the CPU resource calculated according to the available CPU resources of each machine in each target exclusive K8s cluster according to the preset CPU normalization algorithm; the normalized memory is the memory resource calculated according to the available memory resources in each target exclusive K8s cluster according to the preset memory normalization algorithm.

[0209] Optionally, in some embodiments, in response to the user Redis cluster deployment request with a medium priority, the method further includes:

[0210] Determine the target Node node according to the CPU utilization and remaining memory space of each Node node in each of the target dedicated K8s clusters and each of the target colocation clusters;

[0211] Deploy a target Redis proxy middleware for the user Redis cluster deployment request, and allocate the number of CPU cores occupied by the target Redis server execution program for the user Redis cluster deployment request, wherein the number of CPU cores occupied by the target Redis proxy middleware and the target Redis server execution program is less than a preset CPU core number threshold;

[0212] Control each of the target Node nodes to run offline big data tasks during a preset non-emergency time period.

[0213] Optionally, in some embodiments, if a Redis master database in the target dedicated K8s cluster fails, the method further includes:

[0214] Control the Redis sentinel node in the target exclusive K8s cluster where the Redis master database has a fault to perform a master-slave switching operation. If the remaining resources of the target exclusive K8s cluster with a fault are sufficient to run a Redis slave node, a new slave node is generated in the target exclusive K8s cluster with a fault. If the remaining resources of the target exclusive K8s cluster with a fault cannot meet the needs of running a Redis slave node, a new slave node is generated from other target exclusive K8s clusters with sufficient remaining resources; if the remaining resources of each target exclusive K8s cluster do not meet the needs of running a Redis slave node, a new slave node is successfully generated from the target hybrid cluster with sufficient remaining resources;

[0215] The Redis instance on the failed Redis master node is migrated to a new slave node, and the new slave node is determined as a new Redis master node.

[0216] Among them, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this application are in compliance with the relevant laws and regulations and do not violate public order and good morals.

[0217] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0218] In a third aspect, the exemplary embodiments of the present application further provide an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform a method according to an embodiment of the present application when executed by the at least one processor.

[0219] In a fourth aspect, an exemplary embodiment of the present application further provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform a method according to an embodiment of the present application.

[0220] In a fifth aspect, an exemplary embodiment of the present application further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform a method according to an embodiment of the present application.

[0221] refer to Figure 4 , the structural block diagram of the electronic device 400 that can be used as the server or client of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0222] like Figure 4As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0223] A plurality of components in the electronic device 400 are connected to the I / O interface 405, including: an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 may be any type of device capable of inputting information to the electronic device 400, and the input unit 406 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 407 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 408 may include, but is not limited to, a disk, an optical disk. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0224] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above. For example, in some embodiments, the aforementioned scheduling method for Redis instances across K8s clusters may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. In some embodiments, the computing unit 401 may be configured to execute the aforementioned scheduling method for Redis instances across K8s clusters in any other appropriate manner (e.g., by means of firmware).

[0225] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0226] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0227] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0228] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0229] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0230] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

Claims

1. A scheduling system for Redis instances across K8s clusters, characterized in that: The system comprises: A cluster management module, a state monitoring module, a decision engine module, a federated scheduling module, and an execution module, wherein the state monitoring module, the cluster management module, and the execution module are all connected to each K8s cluster, wherein each K8s cluster is mixedly deployed with various Redis nodes that conform to the master-slave structure, and the Redis instance is deployed on the master node and the slave node in each of the Redis nodes; The cluster management module is used to obtain and manage the status information of each K8s cluster. The status monitoring module is used to monitor the cluster performance indicator information of each K8s cluster and the instance performance indicator information of the Redis instance deployed in each K8s cluster in real time; The decision engine module is used to determine a target scheduling strategy based on the state information, the cluster performance indicator information, and the instance performance indicator information; The federated scheduling module is used to receive a user Redis cluster deployment request and forward the user Redis cluster deployment request to the execution module, wherein the user Redis cluster deployment request carries a target Redis instance for processing the user request; The execution module is used to execute the target scheduling strategy and deploy the target Redis instance to the target K8s cluster in the target scheduling strategy.

2. The dispatching system according to claim 1, characterized in that: The user Redis cluster deployment request carries the total load QPS and memory size information required by the user. The federated scheduling module is specifically used to: Receive the total bearer QPS and memory size information carried by the user Redis cluster deployment request, and determine the priority of the user Redis cluster deployment request.

3. The dispatching system according to claim 2, characterized in that: The federal scheduling module is specifically used for: If the priority of the user's Redis cluster deployment request is high priority, a target exclusive K8s cluster is determined for each target Redis instance of the user's Redis cluster deployment request, and only Redis instances are deployed in the target exclusive K8s cluster; If the priority of the user's Redis cluster deployment request is medium priority, the target dedicated K8s cluster is determined for the Redis master instance in the user's Redis cluster deployment request, and the target colocation cluster is determined for the Redis slave instance in the user's Redis cluster deployment request, wherein the target colocation cluster runs Redis instances, databases, and big data tasks; If the priority of the user Redis cluster deployment request is a low priority, the target colocation cluster is determined for each target Redis instance in the user Redis cluster deployment request.

4. The dispatching system according to claim 2, characterized in that: The federal scheduling module is also used for: If the priority of the user Redis cluster deployment request is zero, a temporary cluster is determined for each target Redis instance of the user Redis cluster deployment request, wherein when the running time of the temporary cluster meets the preset expiration time condition, the temporary cluster will automatically expire.

5. The dispatching system according to claim 1, characterized in that: The federal scheduling module is also used for: Initiate a Pod query command to each of the K8s clusters to obtain the resource usage of each Pod container deployed in each of the K8s; A global resource view is generated according to the resource usage of each Pod container, wherein the global resource view includes: the CPU usage and memory usage of each node in each K8s cluster, and the CPU usage and memory usage of each Pod.

6. The dispatching system according to claim 3, characterized in that: In response to the user Redis cluster deployment request with a high priority, the federated scheduling module is further used to: Deploy Redis proxy middleware in each computer room, wherein the Redis proxy middleware deployed in each computer room is distributed in different K8s clusters; According to the Redis server executor package instance carried in the user's Redis cluster deployment request, determine the normalized CPU and normalized memory that meet the requirements of the Redis server executor package instance; Among them, the normalized CPU is the CPU resource calculated according to the available CPU resources of each machine in each target exclusive K8s cluster according to the preset CPU normalization algorithm; the normalized memory is the memory resource calculated according to the available memory resources in each target exclusive K8s cluster according to the preset memory normalization algorithm.

7. The dispatching system according to claim 3, characterized in that: In response to the user Redis cluster deployment request of medium priority, the federated scheduling module is specifically used to: Determine the target Node node according to the CPU utilization and remaining memory space of each Node node in each of the target dedicated K8s clusters and each of the target colocation clusters; Deploy a target Redis proxy middleware for the user Redis cluster deployment request, and allocate the number of CPU cores occupied by the target Redis server execution program for the user Redis cluster deployment request, wherein the number of CPU cores occupied by the target Redis proxy middleware and the target Redis server execution program is less than a preset CPU core number threshold; Control each of the target Node nodes to run offline big data tasks during a preset non-emergency time period.

8. The dispatching system according to claim 3, characterized in that: If a failure occurs in the Redis master database in the target dedicated K8s cluster, the execution module is specifically used to: Control the Redis sentinel node in the target exclusive K8s cluster where the Redis master database has a fault to perform a master-slave switching operation. If the remaining resources of the target exclusive K8s cluster with a fault are sufficient to run a Redis slave node, a new slave node is generated in the target exclusive K8s cluster with a fault. If the remaining resources of the target exclusive K8s cluster with a fault cannot meet the needs of running a Redis slave node, a new slave node is generated from other target exclusive K8s clusters with sufficient remaining resources; if the remaining resources of each target exclusive K8s cluster do not meet the needs of running a Redis slave node, a new slave node is successfully generated from the target hybrid cluster with sufficient remaining resources; The Redis instance on the failed Redis master node is migrated to a new slave node, and the new slave node is determined as a new Redis master node.

9. A method for scheduling Redis instances across K8s clusters, characterized in that: The method is applied to a K8s cluster system, wherein the K8s cluster system includes multiple K8s clusters, each of the K8s clusters is mixedly deployed with Redis nodes that conform to the master-slave structure, and the Redis instance is deployed on the master node and the slave node in each of the Redis nodes. The method includes: Obtain and manage the status information of each K8s cluster, and monitor the cluster performance indicator information of each K8s cluster and the instance performance indicator information of the Redis instance deployed in each K8s cluster in real time; Determine a target scheduling strategy based on the state information, the cluster performance indicator information, and the instance performance indicator information; Receive a user Redis cluster deployment request and forward the user Redis cluster deployment request to the execution module, wherein the user Redis cluster deployment request carries a target Redis instance for processing the user request; Execute the target scheduling strategy and deploy the target Redis instance to the target K8s cluster in the target scheduling strategy.

10. An electronic device, characterized in that: The electronic device comprises: Processor; and Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to claim 9.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to claim 9.

Images