Resource scheduling method and device, equipment, storage medium and product

Abstract

The invention discloses a resource scheduling method, device and equipment, a storage medium and a product, and is applied to a container arrangement platform K8s cluster, the method comprises the following steps: receiving a service request sent by a service requester, the service request being used for requesting to access an upper layer service; determining a node where a service requester is located based on a pre-acquired cluster resource mapping table and the service request; based on the cluster resource mapping table and the distance between the nodes in the K8s cluster, determining a first service provider corresponding to the upper-layer service, the first service provider being a service provider closest to the node where the service requester is located in the service providers corresponding to the upper-layer service, the distance between the nodes is determined based on the network coordinates of the nodes in the K8s cluster; and instructing the service requester to access the first service provider. According to the method, the distance between the nodes in the K8s cluster is predicted based on the network coordinate system, the service request is sent to the service provider closest to the node where the service requester is located, and headless service is compatible.

Description

Technical Field

[0001] The present application relates to resource scheduling processing technology, and in particular to a resource scheduling method, device, equipment, storage medium and product. Background Art

[0002] The container orchestration platform (Kubernetes, K8s) simplifies the deployment, expansion and management of containerized applications, and supports functions such as automatic load balancing, service discovery, horizontal expansion and rolling updates. It is widely used in cloud-native application development and deployment. K8s Domain Name System (DNS) is a service discovery mechanism. K8sDNS assigns a unique domain name to each upper-level service in the K8s cluster. When a Pod in the cluster communicates with other upper-level services, K8s DNS obtains the corresponding IP address based on the accessed service name, and establishes communication with the upper-level service based on the service name.

[0003] The service types within the K8s cluster include Cluster IP Service and Headless Service. The Cluster IP Service is the default type of service. The Cluster IP Service will assign a virtual IP address. The Pod corresponding to this service is usually not directly accessed by other Pods within the K8s cluster. Instead, the Pod corresponding to this service is accessed by accessing the virtual IP address. Specifically, when the service requests the Pod to access the Cluster IP Service based on the virtual IP address, the Cluster IP Service determines the corresponding Pod to be accessed based on the set load balancing strategy. Headless services do not assign virtual IP addresses, but directly point K8s DNS resolution to the IP address of the corresponding Pod. Headless services are usually used for applications that require direct communication with each corresponding Pod or require complex service discovery. In the related technology, when a cluster address service is requested, based on the load balancing strategy, the service request may be sent to a service instance on a node that is far away from the service request Pod, which obviously increases network latency and bandwidth loss. In addition, the process of allocating traffic to the Pod corresponding to the upper-layer service based on the load balancing strategy involves network layer forwarding and packet processing, which will cause performance loss. When a headless service is requested, the headless service does not support the load balancing strategy and has no corresponding resource scheduling method. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a resource scheduling method, apparatus, device, storage medium and product, which aim to reduce the network traffic generated during service resource scheduling within a K8s cluster and achieve compatibility with headless services.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a resource scheduling method, which is applied to a K8s cluster, and the method includes:

[0007] Receiving a service request sent by a service requester, wherein the service request is used to request access to an upper layer service;

[0008] Determine the node where the service requester is located based on the pre-acquired cluster resource mapping table and the service request;

[0009] Based on the cluster resource mapping table and the distances between the nodes in the K8s cluster, determine the first service provider corresponding to the upper-layer service, the first service provider being the service provider closest to the node where the service requester is located among the service providers corresponding to the upper-layer service, and the distances between the nodes are determined based on the network coordinates of the nodes in the K8s cluster;

[0010] The service requester is instructed to access the first service provider.

[0011] In the above scheme, the method further comprises:

[0012] Acquire the cluster resource mapping table, where the cluster resource mapping table includes a first mapping table, a second mapping table, and a third mapping table;

[0013] Among them, the first mapping table indicates the correspondence between Pod information and Pod address, and the Pod information includes the node name; the second mapping table indicates the correspondence between service information and service name of the upper-layer service; the third mapping table indicates the relationship between the service name and Pod address of the upper-layer service; the Pod is the resource unit of the service requester and the service provider.

[0014] In the above solution, determining the node where the service requester is located based on the pre-acquired cluster resource mapping table and the service request includes:

[0015] Based on the Pod address corresponding to the service requester carried in the service request, the first mapping table is queried to determine the node name of the node where the service requester is located.

[0016] In the above solution, determining the first service provider corresponding to the upper-layer service based on the cluster resource mapping table and the distance between each node in the K8s cluster includes:

[0017] Based on the service name of the upper-layer service carried in the service request, query the third mapping table to obtain the Pod addresses of each service provider corresponding to the upper-layer service, and generate a first sub-list;

[0018] Based on the first sub-list, query the first mapping table to obtain the Pod address of each service provider corresponding to the upper-layer service and the node name of the node where the Pod address is located, and generate a second sub-list;

[0019] Based on the second sub-list, query the distances between the nodes, obtain the Pod addresses of the service providers closest to the node where the service requester is located, and generate a third sub-list;

[0020] A first Pod address is selected from the third sublist, and a service provider corresponding to the first Pod address is determined to be the first service provider, and the first Pod address is any Pod address in the third sublist.

[0021] In the above solution, before determining the node where the service requester is located based on the pre-acquired cluster resource mapping table and the service request, the method further includes:

[0022] Based on the service name of the upper layer service carried in the service request, query the second mapping table;

[0023] Based on the corresponding service information queried, if it is determined that the upper-layer service exists in the K8s cluster, the node where the service requester is located is obtained;

[0024] If it is determined that the upper-layer service does not exist in the K8s cluster, an indication indicating that the service name cannot be resolved is returned to the service requester.

[0025] In the above solution, the step of instructing the service requester to access the first service provider includes:

[0026] Instruct the service requester to access the first service provider based on the first Pod address.

[0027] In the above scheme, the method further includes:

[0028] A coordinate instance is created for each node in the K8s cluster, and the coordinate instance is used to obtain the distance between each node in the K8s cluster.

[0029] In the above scheme, the method further includes:

[0030] Setting initial network coordinates for the coordinate instances of each node;

[0031] Based on the round trip time of communication between the coordinate instances, updating the network coordinates of the coordinate instances;

[0032] Based on the updated network coordinates of each coordinate instance, the distances between the nodes are obtained.

[0033] In the above scheme, the method further includes:

[0034] Create one or more headless services in the K8s cluster;

[0035] The headless service is an upper layer service to which a corresponding service address is not allocated; and the service request is specifically used to request access to the headless service.

[0036] In a second aspect, an embodiment of the present application provides a resource scheduling device, which is applied to a K8s cluster, and the device includes:

[0037] A receiving module, used to receive a service request sent by a service requester, wherein the service request is used to request access to an upper layer service;

[0038] A first determination module, configured to determine the node where the service requester is located based on a pre-acquired cluster resource mapping table and the service request;

[0039] A second determination module is used to determine a first service provider corresponding to the upper-layer service based on the cluster resource mapping table and the distance between each node in the K8s cluster, wherein the first service provider is the service provider closest to the node where the service requester is located among the service providers corresponding to the upper-layer service, and the distance between each node is determined based on the network coordinates of each node in the K8s cluster;

[0040] The instruction module is used to instruct the service requester to access the first service provider.

[0041] In a third aspect, an embodiment of the present application provides a resource scheduling device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor, when running the computer program, executes the steps of the method described in the first aspect.

[0042] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0043] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0044] The embodiment of the present application provides a resource scheduling method applied to a K8s cluster, the method comprising: receiving a service request sent by a service requester, the service request being used to request access to an upper-layer service; determining the node where the service requester is located based on a pre-acquired cluster resource mapping table and a service request; determining the first service provider corresponding to the upper-layer service based on the cluster resource mapping table and the distance between each node in the K8s cluster, the first service provider being the service provider closest to the node where the service requester is located among the service providers corresponding to the upper-layer service, the distance between each node being determined based on the network coordinates of each node in the K8s cluster; instructing the service requester to access the first service provider. In this way, the embodiment of the present application predicts the distance between each node in the K8s cluster based on the network coordinate system, and when an upper-layer service in the K8s cluster is requested, the service request is sent to the service provider closest to the node where the service requester is located based on the cluster resource mapping table, thereby reducing network delay and bandwidth consumption, and the service request of the embodiment of the present application directly points to the service provider, without obtaining the virtual address of the upper-layer service, and is compatible with headless services. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a schematic diagram of the resource objects of the K8s cluster;

[0046] Figure 2 It is a schematic diagram of a K8s resource scheduling method in related technologies;

[0047] Figure 3 A schematic diagram of a resource scheduling method according to an embodiment of the present application;

[0048] Figure 4 A schematic diagram of a resource scheduling method in an application example of the present application;

[0049] Figure 5 A method diagram of a service request processing method applied to a K8s cluster in an application example of this application;

[0050] Figure 6 This is a schematic diagram of the structure of the resource scheduling device according to an embodiment of the present application;

[0051] Figure 7 This is a schematic diagram of the structure of the resource scheduling device according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0054] Before describing the resource scheduling method of the embodiment of the present application, a brief description of the K8s cluster is first given. Pod is the smallest resource unit that can be scheduled in the K8s cluster, running on the worker node Worker. A Worker contains multiple Pods, and each Pod encapsulates one or more containers; the upper-level service Service is a resource object of the K8s cluster, which is used to expose the network application running on one or a group of Pods as a network service, and each Pod corresponds to a service instance; K8s DNS is a service discovery mechanism of the K8s cluster. When a Service is created in the K8s cluster, the K8s DNS service will assign a unique domain name to the Service. When accessing within the K8s cluster, other Pods can establish communication with the Service based on the service name. Specifically, K8s DNS will resolve the corresponding IP address based on the service name to instruct other Pods to access the Service.

[0055] Specifically, Figure 1 As shown, in one example, the K8s cluster includes a first worker node Worker1 and a second worker node Worker2, and the Pods corresponding to the Service in the K8s cluster include Pod1, Pod2, and Pod3, wherein Pod1 runs on Worker1, and Pod2 and Pod3 run on Worker2, and Pod1, Pod2, and Pod3 serve as service instances of the Service. When the Service is requested to be accessed, at least one of Pod1, Pod2, and Pod3 responds to the service request.

[0056] Here, in the K8s cluster, Pod is a temporary resource that changes dynamically when running an application. The Service cannot grasp the corresponding Pod quantity, Pod address, and Pod availability in real time. Therefore, a resource object with the same name as the Service is created in the K8s cluster: Endpoint, to record the changes in the Pod corresponding to the Service. The endpoint Endpoint is the interface for accessing the Pod corresponding to the Service. The Endpoint information includes information such as the Endpoint address and the node name NodeName. Among them, the Endpoint corresponds to the Pod one-to-one, and the Endpoint address is the Pod address of the Pod corresponding to the service with the same name. When the Pod corresponding to the Service changes, the Endpoint Endpoint corresponding to the Service updates the Endpoint information based on the Pod change. When the Service is requested, the access request can be sent to the Pod corresponding to the Service based on the Endpoint information to provide the corresponding service. It should be noted that the K8s cluster supports multiple types of services, and different types of services can be selected according to different usage scenarios. For example, the Cluster IP Service is used for internal access to the K8s cluster, where the Cluster IP Service provides a unified virtual IP address for the corresponding Pods. When the service request Pod requests to access the Cluster IP Service, the service request Pod can access the Pod corresponding to the Cluster IP Service based on the virtual IP address returned by the K8s DNS service. Specifically, after the Cluster IP Service receives the service request, it determines the corresponding one or more Pods to respond to the request based on the set scheduling strategy (such as load balancing strategy). In the internal access scenario of the K8s cluster, it also supports requesting a Headless Service. The Headless Service will not allocate a virtual IP address (that is, the IP address is None). When the Headless Service is requested, the IP address of the Pod corresponding to the Headless Service is directly returned after K8s DNS resolution. The Headless Service is usually used for applications that require direct communication with each corresponding Pod or require complex service discovery.

[0057] It should be noted that in the related technology, the service request Pod accesses the Service based on the IP address returned after K8s DNS resolution. If the requested Service is a cluster address service, the returned IP address is the virtual IP address of the cluster address service; if the requested service is a headless service, the returned IP address is the IP address of the Pod corresponding to the headless service. When the cluster address service is requested, based on the load balancing strategy, the resource usage status of each corresponding service instance is given priority when scheduling resources. The service request of the service request Pod may be sent to an idle service instance far away from the service request Pod, which increases the network delay and bandwidth loss of the service. In addition, the process of allocating traffic to the Pod based on the load balancing strategy by the cluster address service involves forwarding and data processing at the network layer, resulting in performance consumption; and when the headless service is requested, the headless service does not support the load balancing strategy and has no responsive resource scheduling method.

[0058] Specifically, Figure 2 As shown, in one example, the Service is a cluster address service. When Pod4 running on Worker1 requests to access the Service, the K8s DNS service resolves the service name of the Service carried in the access request and returns the virtual IP address corresponding to the Service. Pod4 accesses the Service based on the returned virtual IP address. The Service determines that Pod2 running on Worker2 is currently idle based on the Endpoint information according to the load balancing strategy, and sends the access request to Pod2 based on the Endpoint2 information, and Pod2 provides services for Pod4.

[0059] Based on this, the present application provides a resource scheduling method for applying a K8s cluster, aiming to improve the response speed and performance of the service and reduce the generation of network traffic in the internal access scenario of the K8s cluster, and this method is compatible with headless services.

[0060] The present application embodiment provides a resource scheduling method, which is applied to a K8s cluster. Figure 3 As shown, the method includes:

[0061] Step 301: Receive a service request sent by a service requester, where the service requester is used to request access to an upper-layer service. Here, Pod is the resource unit of the service requester, and the service requester is specifically the service requester Pod. The service request carries the Pod address of the service requester Pod and the service name of the service requested to be accessed. The service name of the service is used to obtain the corresponding IP address after K8s DNS resolution.

[0062] Step 302: Determine the node where the service requester is located based on the pre-acquired cluster resource mapping table and the service request.

[0063] Here, the service request Pod runs on the working node, which can be a virtual machine node, a physical machine node, or a cloud server node.

[0064] Exemplarily, the method further includes: obtaining a cluster resource mapping table, the cluster resource mapping table including a first mapping table, a second mapping table and a third mapping table. The first mapping table indicates the correspondence between Pod information and Pod addresses, the Pod information including the node name; the second mapping table indicates the correspondence between service information and service names of upper-layer services; and the third mapping table indicates the relationship between service names of upper-layer services and Pod addresses.

[0065] It should be noted that K8s supports a list connection mechanism (List-Watch). List-Watch is a mechanism provided by the K8s API Server for monitoring changes in resource objects. The specific mechanism is as follows: the client sends a list request to the K8s API server, requesting a list of objects of a specific resource type; the K8s API server returns a list of resource objects and maintains a connection with the client; when the resource object requested by the list changes (such as creation, update, or deletion), the K8s API server sends a change notification to the client; after receiving the change notification, the client can process it as needed, such as updating the local cache or triggering an operation.

[0066] It can be understood that the embodiment of the present application is based on the List-Watch mechanism, which can obtain the full object list of a specific resource type from the K8s application interface server in real time and obtain the cluster resource mapping table without frequently sending requests to poll the K8s application interface server.

[0067] Here, the cluster resource mapping table obtained by the embodiment of the present application includes a first mapping table, a second mapping table and a third mapping table, wherein the first mapping table is a Pods mapping table, indicating the correspondence between Pod information and Pod address, and the Pod information includes the Pod name and the node name (Node Name) of the node where the Pod is located, and other information. Based on the first mapping table, the Pod information of each Pod in the K8s cluster can be quickly queried.

[0068] Exemplarily, based on the pre-acquired cluster resource mapping table and the service request, determining the node where the service requester is located includes: querying the first mapping table based on the Pod address corresponding to the service requester carried in the service request, and determining the node name of the node where the service requester is located.

[0069] It can be understood that when the K8s DNS service in the K8s cluster resolves the service name carried in the service request, it can obtain the node name of the node where the service request Pod is located based on the first mapping table and the Pod address of the service request Pod also carried in the service request.

[0070] Here, the second mapping table in the cluster resource mapping table is the Services mapping table, which indicates the correspondence between the service information of the Service and the service name, wherein the service information may include at least one of the following: service type, service address and port information. Based on the second mapping table, the service information of the Service in the K8s cluster can be quickly queried.

[0071] In an application example of the present application, the service name is named in the form of "service identifier.namespace name" to isolate service resources under different namespaces.

[0072] Here, the third mapping table in the cluster resource mapping table is the Endpoints mapping table, which indicates the relationship between the service name and the endpoint address of the Service. Based on the third mapping table, the Endpoint information corresponding to the Service in K8s can be quickly queried.

[0073] It can be understood that the K8s DNS service in the embodiment of the present application obtains the full object list of Pod, Service and Endpoint based on the List-Watch mechanism, and then obtains the cluster resource mapping table.

[0074] Exemplarily, before determining the node where the service requester is located based on the pre-acquired cluster resource mapping table and service request, the method also includes: querying the second mapping table based on the service name of the upper-layer service carried in the service request; based on the corresponding service information queried, if it is determined that the upper-layer service exists in the K8s cluster, obtaining the node where the service requester is located; if it is determined that the upper-layer service does not exist in the K8s cluster, returning an indication to the service requester indicating that the service name cannot be resolved.

[0075] It should be noted that before performing resource scheduling, the embodiment of the present application first determines whether an available Service has been created in the K8s cluster by querying the second mapping table based on the service name of the Service requested by the service request Pod. If the corresponding service information is queried, the subsequent resource scheduling method is executed; if the corresponding service information is not queried, the K8s DNS service returns an indication that the service name cannot be resolved to the service request Pod, and the subsequent resource scheduling method is no longer executed.

[0076] Step 303, based on the cluster resource mapping table and the distance between each node in the K8s cluster, determine the first service provider corresponding to the upper-layer service. The first service provider is the service provider closest to the node where the service requester is located among the service providers corresponding to the upper-layer service. The distance between each node is determined based on the network coordinates of each node in the K8s cluster. Here, the service provider is the service instance of the requested Service, and the Pod is the resource unit of the service provider. The service provider is specifically the Pod corresponding to the Service, and the Pod corresponding to the Service runs on the working node.

[0077] Exemplarily, based on the cluster resource mapping table and the distances between nodes in the K8s cluster, determining the first service provider corresponding to the upper-layer service includes:

[0078] Based on the service name of the upper-layer service carried in the service request, query the third mapping table to obtain the Pod addresses of each service provider corresponding to the upper-layer service, and generate a first sub-list; based on the first sub-list, query the first mapping table to obtain the Pod addresses of each service provider corresponding to the upper-layer service and the node name of the node where the service provider is located, and generate a second sub-list; based on the second sub-list, query the distance between each node to obtain the Pod addresses of each service provider closest to the node where the service requester is located, and generate a third sub-list; select the first Pod address from the third sub-list, and determine that the service provider corresponding to the first Pod address is the first service provider, and the first Pod address is any Pod address in the third sub-list.

[0079] Here, the network coordinate system is a scalable Internet distance prediction solution. The nodes participating in the network coordinate system can obtain the network coordinates of the node in the network coordinate system through a small amount of measurement. Based on the network coordinates of any two nodes, the distance between the two nodes can be predicted.

[0080] It can be understood that the network coordinate system is suitable for K8s clusters including a large number of distributed nodes. For a K8s cluster including N nodes, the network distance measurement complexity is O(N). The network coordinate system can predict the network distance of N*(N-1) links through measurements with a complexity of O(N), effectively avoiding a large number of end-to-end measurement processes.

[0081] It can be understood that the embodiment of the present application executes the shortest path routing strategy based on the network coordinate system, that is, after receiving the service request sent by the service request Pod, the Pod address of each service instance of the Service and the node name of the node where the service instance is located are obtained by querying the first mapping table and the third mapping table, and the network coordinates of the node where the service instance is located are determined based on the node name of the node where the service instance is located, and the service instance closest to the service request Pod among the service instances corresponding to the Service is selected based on the network coordinates to respond to the request, which can improve the response speed and reliability of the service.

[0082] It can be understood that the service instance closest to the service request Pod is the service instance on the same node, that is, if it is determined that there is an available service instance of the Service on the service request Pod, then the service instance is determined to be used to respond to the request.

[0083] It can be understood that this application calculates the distance based on the network coordinates of each node in the K8s cluster. If multiple service instances are running simultaneously on the node closest to the service request Pod, any service instance is selected to respond to the request.

[0084] It should be noted that the embodiment of the present application executes the shortest path routing strategy based on the network coordinate system. Before executing resource scheduling, it is necessary to obtain the network coordinates of each node in the K8s cluster and the distance between each node calculated based on the network coordinates of each node. Here, the embodiment of the present application provides a method for obtaining the network coordinates of each node in the K8s cluster. Exemplarily, the method also includes: creating a coordinate instance for each node in the K8s cluster, and the coordinate instance is used to obtain the distance between each node in the K8s cluster.

[0085] It should be noted that the coordinate instance is created by the daemon controller (DaemonSet) of the K8s cluster. DaemonSet is a controller object of the K8s cluster, which is used to ensure that each node in the cluster runs a copy of the Pod. When a new node joins the K8s cluster, DaemonSet will create a new Pod on the new node; when a node is removed from the K8s cluster, DaemonSet will delete the Pod created by DaemonSet on the removed node; when DaemonSet is deleted, DaemonSet will delete all Pods created by DaemonSet. DaemonSet is usually used to run specific system-level services on each node in the K8s cluster, such as log collection, cluster storage, and monitoring agents. The embodiment of the present application uses DaemonSet to ensure that all nodes in the K8s cluster run a coordinate instance to calculate the distance between each node in the cluster. Here, the coordinate instance can be a Pod with a worker agent deployed. The worker agent is used to update the network coordinates of this node. Each coordinate instance joins the same communication network, and based on the mutual communication between each coordinate instance under the same communication network, the information consistency propagation between all coordinate instances is realized.

[0086] Here, the distance between each node is the network distance between each node. In the network coordinate system, the network distance between each node is positively correlated with the geographical distance of each node.

[0087] In an application example of the present application, the coordinate instances of each node in the K8s cluster communicate with each other based on the Gossip protocol. The Gossip protocol is a protocol for information interaction between nodes based on the epidemic propagation method. Each coordinate instance randomly selects other coordinate instances at a set frequency and shares information, ultimately achieving information consistency between all coordinate instances.

[0088] It can be understood that the coordinate instances of each node in the K8s cluster share information based on the Gossip protocol, and each coordinate instance can obtain the network coordinates of all nodes in the K8s cluster.

[0089] It should be noted that using the Gossip protocol to achieve network coordinate sharing of each node is only an application example provided by this application. The network coordinate sharing of each node can also be achieved using the Kademlia protocol and the Chord protocol. This application does not specifically limit the communication method between the coordinate instances of each node.

[0090] Exemplarily, the method also includes: setting initial network coordinates for the coordinate instances of each node; updating the network coordinates of each coordinate instance based on the round-trip time of communication between each coordinate instance; and obtaining the distance between each node based on the updated network coordinates of each coordinate instance.

[0091] It should be noted that the embodiment of the present application implements the calculation of the network distance between all nodes in the K8s cluster based on the distributed network coordinate Vivaldi algorithm. The Vivaldi algorithm represents the network distance between nodes based on measuring the round-trip time (Round-Trip Time) of communication between nodes, and by assigning a network coordinate to each node in the K8s cluster, each node can calculate the network distance with other nodes based on the network coordinate.

[0092] Here, round-trip time specifically refers to the time from sending a data packet to receiving the corresponding confirmation information or response information, including the time the data packet spends in the transmission process and the time it takes to process and respond at the receiving end. Round-trip time is usually used to measure network latency.

[0093] It can be understood that the embodiment of the present application is based on the Vivaldi algorithm, which can represent the distance between nodes based on the measured round-trip time without obtaining the global geographic address information of each node in the K8s cluster.

[0094] Specifically, based on the Vivaldi algorithm, the embodiment of the present application first sets the initial network coordinates for the worker agent of each coordinate instance as the network coordinates of the corresponding node. The initial network coordinates can be a multi-dimensional origin. The dimension of the network coordinates can be determined based on the cluster requirements. The higher the dimension of the network coordinates, the more accurately the calculated network coordinates can reflect the physical address information of the node. Secondly, each coordinate instance sends a message to some coordinate instances in the K8s cluster at a set frequency based on the Gossip protocol, and the round-trip time of the message is used as the network delay of the node; thereafter, the coordinate instance of each node calculates an error vector based on the measured network delay and the current network coordinates, and applies the obtained error vector to the current network coordinates to update its own network coordinates; finally, each coordinate instance converges the network coordinates by periodically measuring the network delay and updating the network coordinates to meet the geographic address relationship between the nodes.

[0095] In an application example of the present application, a method for calculating the network distance between two nodes using two-bit network coordinates is provided. The K8s cluster includes nodes a and b, the network coordinates of node a are (aX, aY, a.Height), and the network coordinates of node b are (bX, bY, b.Height), where Height represents the fixed delay overhead when accessing the node, and is used to model the access link for node access. The network distance between node a and node b is the round-trip time between node a and node b, specifically:

[0096]

[0097] It can be understood that the embodiment of the present application is based on creating a first mapping table indicating the correspondence between Pod information and Pod addresses and a third mapping table indicating the correspondence between Endpoint information and service names, so that after the service name of the Service carried by the service request is resolved, the Pod information of each service instance of the Service can be directly obtained, and based on the network coordinate system, the service instance closest to the service request Pod is selected from each service instance of the Service to respond to the request, effectively avoiding the problem that the Service may dispatch a service instance that is far away from the service request Pod to respond to the request.

[0098] Step 304: instruct the service requester to access the first service provider.

[0099] Exemplarily, instructing the service requester to access the first service provider includes: instructing the service requester to access the first service provider based on the first Pod address.

[0100] Exemplarily, the method also includes: creating one or more headless services in the K8s cluster; wherein the headless service is an upper-layer service that does not allocate a corresponding service address; and the service request is specifically used to request access to the headless service.

[0101] Here, in the embodiment of the present application, the service request may be used to request access to the headless service in addition to requesting access to the cluster address service.

[0102] It can be understood that in the embodiment of the present application, regardless of whether the service request Pod requests access to the cluster address service or the headless service, the K8s DNS service will return the first Pod address of the first service provider for access, thereby achieving compatibility with the headless service; and the service request Pod directly accesses the service instance based on the first Pod address, eliminating the step of determining the service instance that responds to the request based on the load balancing strategy after the Service is requested, thereby avoiding the problem of performance loss caused by the above steps.

[0103] The present application also provides a K8s cluster architecture for implementing the above method, such as Figure 2 As shown. The K8s cluster architecture includes a first working node 1, a second working node 2, a third working node 3, a fourth working node 4 and a master node 5. The first working node 1 runs a DNS endpoint 11 and a first coordinate endpoint instance 12, the second working node 2 runs a service usage endpoint 21 and a second coordinate endpoint instance 22, the third working node 3 runs a first service endpoint 31 and a third coordinate endpoint instance 32, the fourth working node 4 runs a second service endpoint 41 and a fourth coordinate endpoint instance 42, and the master node 5 runs a K8s application interface server 51.

[0104] In an application example of the present application, Figure 4 As shown, the K8s cluster includes 4 working nodes and the master node Master. The 4 working nodes are Worker1, Worker2, Worker3 and Worker4. The Master is used to manage the resources of each working node. Worker1 runs the K8s DNS service and the first coordinate instance, Worker2 runs Pod3 and the second coordinate instance, Worker3 runs Pod1 and the third coordinate instance, and Worker4 runs Pod2 and the fourth coordinate instance. Among them, Pod1 and Pod2 correspond to the same Service, and the Master runs the K8s application interface server.

[0105] Here, the first coordinate instance, the second coordinate instance, the third coordinate instance, and the fourth coordinate instance are replica Pods created by DaemonSet. A worker agent process runs on each coordinate instance. Each coordinate instance communicates periodically in the same communication network based on the Gossip protocol. Each coordinate instance stores the network coordinates of each working node in the K8s cluster and the distance between each working node.

[0106] Here, the open source DNS server CoreDNS runs on the K8s DNS service. CoreDNS has a modular architecture and can extend functions through plug-ins as needed. The K8s DNS service is based on the List-Watch mechanism, requests a connection from the K8s application interface server, obtains a full list of cluster resource objects from the K8s application interface server based on the connection interface, generates an initialized cluster resource mapping table based on the obtained full list, and subscribes to change notifications of cluster resource objects from the K8s application interface server based on the connection interface to update the cluster resource mapping table. It can be understood that the adaptive adjustment of CoreDNS in the embodiment of the present application can enable the K8s DNS service to obtain a cluster resource mapping table including service information, Pod information, and endpoint information.

[0107] When Pod3 requests to access the Service, the K8s DNS service parses the service request sent by Pod3, and based on the cluster resource mapping table, determines that Pod3 is running on Worker2, and obtains the Pod addresses of Pod1 and Pod2 corresponding to the Service, and determines that Pod1 is running on Worker3 and Pod2 is running on Worker4; the K8s DNS service accesses the first coordinate instance to query the distance between Worker2 and Worker3, and the distance between Worker2 and Worker4. For example, if it is determined that Worker3 is closer to Worker2, the Pod address of Pod1 is returned to Pod3. Pod3 directly accesses Pod1 based on the received Pod address, and Pod1 responds to the service request.

[0108] It is understandable that the K8s DNS service can also access the coordinate instances on other working nodes to query the distance between each node. The coordinate instances in the K8s cluster have information consistency, and the K8s DNS service gives priority to accessing the coordinate instances of the local node.

[0109] It can be understood that if the Service requested by Pod3 runs an available service instance on Worker2, the K8sDNS service instructs Pod3 to access the service instance running on Worker2.

[0110] It can be understood that when the K8s cluster architecture in this application example executes the resource scheduling method of the embodiment of the present application, the service requester Pod3 directly accesses the first service provider Pod1, and does not access the Service based on the service address, thereby reducing the performance loss within the cluster.

[0111] In an application example of the present application, a service request processing method applied to a K8s cluster is provided, such as Figure 5 As shown, the method includes:

[0112] Step 501, the K8s DNS service receives a service request from a service request Pod.

[0113] Here, the service request carries the Pod address of the service request Pod and the service name of the requested Service.

[0114] Here, the K8s DNS service pre-acquires a cluster resource mapping table based on the List-Watch mechanism. The cluster resource mapping table includes a first mapping table indicating Pod information, a second mapping table indicating Service information, and a third mapping table indicating Endpoint information.

[0115] Step 502, the K8s DNS service resolves the service request and obtains the namespace name and service identifier of the requested Service.

[0116] Step 503, based on the second mapping table, determine whether there is a corresponding Service in the K8s cluster. If so, execute step 504; if not, execute step 512.

[0117] Here, the second mapping table records the correspondence between the service name and the service information. Based on the parsed namespace name and service identifier, it is queried whether the corresponding service information is recorded in the second mapping table to determine whether the corresponding Service exists in the K8s cluster.

[0118] Step 504, the K8s DNS service resolves the service request and obtains the Pod address of the service request Pod.

[0119] Step 505, the K8s DNS service queries the Pod information of the service request Pod based on the first mapping table and the Pod address of the service request Pod, including the node name of the node where the service request Pod is located.

[0120] Step 506, the K8s DNS service queries the Pod address of each Pod corresponding to the Service and the node name of the node where the Pod is located based on the first mapping table, the third mapping table, the node name of the node where the service request Pod is located, the namespace name of the Service, and the service identifier.

[0121] Here, the third mapping table records the Endpoint information corresponding to the Service. Based on the service name including the namespace name and the service identifier, the Pod addresses of each Pod corresponding to the Service can be queried to generate the first sub-list; based on the queried Pod addresses of each Pod and the first mapping table, the node name corresponding to the Pod address is obtained, and the second sub-list is generated.

[0122] Step 507, the K8s DNS service accesses the coordinate instance, and based on the node name of the node where each Pod corresponding to the Service is located, queries the distance between each node supporting the running of the service instance of the Service and the node where the service request Pod is located, and obtains a distance result set.

[0123] Here, the distance between each node supporting the operation of the service instance of the Service and the first node may be determined by the network coordinates of the nodes, and the network coordinates of each node are determined by measuring the round trip time of information between the nodes.

[0124] Step 508, based on the distance results, determine the node closest to the node where the service request Pod is located among the nodes supporting the running of the service instance of the Service, and obtain the Pod address of the service instance running on the node to generate a third sub-list.

[0125] Step 509, the K8s DNS service randomly selects a Pod address from the third sublist and returns the selected Pod address to the service request Pod.

[0126] In step 510, the K8s DNS service returns an error message indicating that the resolution cannot be performed to the service requesting Pod.

[0127] Here, if the service information corresponding to the namespace name and service identifier carried in the service request is not found based on the first mapping table, it means that the corresponding Service is not created in the K8s cluster, and an error prompt is returned.

[0128] It can be understood that the service request processing method of this application example, combined with the method for obtaining the cluster resource mapping table and the method for calculating the network coordinates, can realize the resource scheduling method applied to the K8s cluster of the embodiment of the present application.

[0129] In order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a resource scheduling device for a K8s cluster. The resource scheduling device corresponds to the above-mentioned resource scheduling method, and each step in the above-mentioned resource scheduling method embodiment is also fully applicable to the embodiment of the present device.

[0130] like Figure 6 As shown, an embodiment of the present application provides a resource scheduling device, which is applied to a K8s cluster, and the device includes: a receiving module 601, a first determination module 602, a second determination module 603 and an indication module 604. The receiving module 601 is used to receive a service request sent by a service requester, and the service request is used to request access to an upper-layer service. The first determination module 602 is used to determine the node where the service requester is located based on a pre-acquired cluster resource mapping table and a service request. The second determination module 603 is used to determine the first service provider corresponding to the upper-layer service based on the cluster resource mapping table and the distance between each node in the K8s cluster. The first service provider is the service provider closest to the node where the service requester is located among the service providers corresponding to the upper-layer service, and the distance between each node is determined based on the network coordinates of each node in the K8s cluster. The indication module 604 is used to instruct the service requester to access the first service provider.

[0131] In some embodiments, the resource scheduling device further includes an acquisition module 605. The acquisition module 605 is used to acquire a cluster resource mapping table, which includes a first mapping table, a second mapping table, and a third mapping table. The first mapping table indicates the correspondence between Pod information and Pod addresses, and the Pod information includes a node name; the second mapping table indicates the correspondence between service information and service names of upper-layer services; the third mapping table indicates the relationship between service names and Pod addresses of upper-layer services; and Pod is a resource unit of a service requester and a service provider.

[0132] In some embodiments, the first determination module 602 is specifically used to: query the first mapping table based on the Pod address corresponding to the service requester carried in the service request, and determine the node name of the node where the service requester is located.

[0133] In some embodiments, the second determination module 603 is specifically used to: query the third mapping table based on the service name of the upper-layer service carried by the service request, obtain the Pod addresses of each service provider corresponding to the upper-layer service, and generate a first sub-list; query the first mapping table based on the first sub-list, obtain the Pod addresses of each service provider corresponding to the upper-layer service and the node name of the node where the service provider is located, and generate a second sub-list; query the distance between each node based on the second sub-list, obtain the Pod addresses of each service provider closest to the node where the service requester is located, and generate a third sub-list; select the first Pod address from the third sub-list, and determine that the service provider corresponding to the first Pod address is the first service provider, and the first Pod address is any Pod address in the third sub-list.

[0134] In some embodiments, the first determination module 602 is also used to: query the second mapping table based on the service name of the upper-layer service carried in the service request; based on the corresponding service information queried, if it is determined that the upper-layer service exists in the K8s cluster, obtain the node where the service requester is located; if it is determined that the upper-layer service does not exist in the K8s cluster, return an indication information indicating that the service name cannot be resolved to the service requester.

[0135] In some embodiments, the instructing module 604 is specifically used to instruct the service requester to access the first service provider based on the first Pod address.

[0136] In some embodiments, the resource scheduling device further includes a creation module 606. The creation module 606 is used to create a coordinate instance in each node in the K8s cluster, and the coordinate instance is used to obtain the distance between each node in the K8s cluster.

[0137] In some embodiments, the creation module 606 is also used to: set the initial network coordinates for the coordinate instance of each node; update the network coordinates of each coordinate instance based on the round-trip time of communication between each coordinate instance; and obtain the distance between each node based on the updated network coordinates of each coordinate instance.

[0138] In some embodiments, the creation module 606 is further used to: create one or more headless services in the K8s cluster. Wherein, the headless service is an upper layer service that does not allocate a corresponding service address; the service request is specifically used to request access to the headless service.

[0139] It should be noted that the resource scheduling device provided in the above embodiment only uses the division of the above program modules as an example when performing resource scheduling. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the resource scheduling device provided in the above embodiment and the resource scheduling method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0140] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a resource scheduling device. Figure 7 Only an exemplary structure of the resource scheduling device is shown, not the entire structure, and can be implemented as needed. Figure 7 Partial or complete structure shown.

[0141] like Figure 7 As shown, the resource scheduling device 700 provided in the embodiment of the present application includes: at least one processor 701, a memory 702, a user interface 703 and at least one network interface 704. The various components in the resource scheduling device 700 are coupled together through a bus system 705. It can be understood that the bus system 705 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 705 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Figure 7 Various buses are labeled as bus system 705.

[0142] The user interface 703 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen.

[0143] The memory 702 in the embodiment of the present application is used to store various types of data to support the operation of the resource scheduling device 700. Examples of such data include: any computer program used to operate on the resource scheduling device 700.

[0144] The resource scheduling method disclosed in the embodiment of the present application can be applied to the processor 701, or implemented by the processor 701. The processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the resource scheduling method can be completed by the hardware integrated logic circuit or software instructions in the processor 701. The above-mentioned processor 701 can be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 701 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 702. The processor 701 reads the information in the memory 702 and completes the steps of the resource scheduling method provided in the embodiment of the present application in combination with its hardware.

[0145] In an exemplary embodiment, the resource scheduling device 700 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), FPGA, general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned resource scheduling method.

[0146] It can be understood that the memory 702 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an EEPROM, a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), direct memory bus random access memory (DRRAM). The memory described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memory.

[0147] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, which can be a computer-readable storage medium, for example, a memory 702 storing a computer program, and the computer program can be executed by a processor 701 of a resource scheduling device 700 to complete the steps described in the resource scheduling method of the present application embodiment. The computer-readable storage medium can be a memory such as a ROM, a PROM, an EPROM, an EEPROM, a Flash Memory, a magnetic surface memory, an optical disk, or a CD-ROM.

[0148] In an exemplary embodiment, the embodiment of the present application further provides a computer program product, including a computer program, which can be executed by the processor 701 of the resource scheduling device 700 to complete the steps described in the method of the embodiment of the present application.

[0149] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0150] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0151] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A resource scheduling method, characterized in that: Applied to the K8s cluster of the container orchestration platform, the method includes: Receiving a service request sent by a service requester, wherein the service request is used to request access to an upper layer service; Determine the node where the service requester is located based on the pre-acquired cluster resource mapping table and the service request; Based on the cluster resource mapping table and the distances between the nodes in the K8s cluster, determine the first service provider corresponding to the upper-layer service, the first service provider being the service provider closest to the node where the service requester is located among the service providers corresponding to the upper-layer service, and the distances between the nodes are determined based on the network coordinates of the nodes in the K8s cluster; The service requester is instructed to access the first service provider.

2. The method according to claim 1, characterized in that The method further comprises: Acquire the cluster resource mapping table, where the cluster resource mapping table includes a first mapping table, a second mapping table, and a third mapping table; Among them, the first mapping table indicates the correspondence between Pod information and Pod address, and the Pod information includes the node name; the second mapping table indicates the correspondence between service information and service name of the upper-layer service; the third mapping table indicates the relationship between the service name and Pod address of the upper-layer service; the Pod is the resource unit of the service requester and the service provider.

3. The method according to claim 2, characterized in that The determining, based on the pre-acquired cluster resource mapping table and the service request, the node where the service requester is located, comprises: Based on the Pod address corresponding to the service requester carried in the service request, the first mapping table is queried to determine the node name of the node where the service requester is located.

4. The method according to claim 2, characterized in that: The determining, based on the cluster resource mapping table and the distances between the nodes in the K8s cluster, a first service provider corresponding to the upper-layer service includes: Based on the service name of the upper-layer service carried in the service request, query the third mapping table to obtain the Pod address of each service provider corresponding to the upper-layer service, and generate a first sub-list; Based on the first sub-list, query the first mapping table to obtain the Pod address of each service provider corresponding to the upper-layer service and the node name of the node where the Pod address is located, and generate a second sub-list; Based on the second sub-list, query the distances between the nodes, obtain the Pod addresses of the service providers closest to the node where the service requester is located, and generate a third sub-list; A first Pod address is selected from the third sublist, and a service provider corresponding to the first Pod address is determined to be the first service provider, and the first Pod address is any Pod address in the third sublist.

5. The method according to claim 4, characterized in that Before determining the node where the service requester is located based on the pre-acquired cluster resource mapping table and the service request, the method further includes: Based on the service name of the upper layer service carried in the service request, query the second mapping table; Based on the corresponding service information queried, if it is determined that the upper-layer service exists in the K8s cluster, the node where the service requester is located is obtained; If it is determined that the upper-layer service does not exist in the K8s cluster, an indication indicating that the service name cannot be resolved is returned to the service requester.

6. The method according to claim 4, characterized in that The instructing the service requester to access the first service provider includes: Instruct the service requester to access the first service provider based on the first Pod address.

7. The method according to claim 1, characterized in that The method further comprises: A coordinate instance is created for each node in the K8s cluster, and the coordinate instance is used to obtain the distance between each node in the K8s cluster.

8. The method according to claim 7, characterized in that The method further comprises: Setting initial network coordinates for the coordinate instances of each node; Based on the round trip time of communication between the coordinate instances, updating the network coordinates of the coordinate instances; Based on the updated network coordinates of each coordinate instance, the distances between the nodes are obtained.

9. The method according to claim 1, characterized in that: The method further comprises: Create one or more headless services in the K8s cluster; The headless service is an upper layer service to which a corresponding service address is not allocated; and the service request is specifically used to request access to the headless service.

10. A resource scheduling device, characterized in that: Applied to K8s cluster, the device includes: A receiving module, used to receive a service request sent by a service requester, wherein the service request is used to request access to an upper layer service; A first determination module, configured to determine the node where the service requester is located based on a pre-acquired cluster resource mapping table and the service request; A second determination module is used to determine a first service provider corresponding to the upper-layer service based on the cluster resource mapping table and the distance between each node in the K8s cluster, wherein the first service provider is the service provider closest to the node where the service requester is located among the service providers corresponding to the upper-layer service, and the distance between each node is determined based on the network coordinates of each node in the K8s cluster; The instruction module is used to instruct the service requester to access the first service provider.

11. A resource scheduling device, characterized in that: include: A processor and a memory for storing a computer program that can be run on the processor, wherein the processor, when running the computer program, executes the steps of the method according to any one of claims 1 to 9.

12. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images