A container load balancing method, device, equipment and storage medium

Abstract

The present application provides a container load balancing method, apparatus, device and storage medium, wherein the method comprises: receiving, through a load balancing server between clusters, a processing request sent by a service caller using a cluster virtual Internet Protocol address VIP and a host port corresponding to the cluster VIP; determining, through the load balancing server between clusters, the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster; forwarding, through the load balancing server between clusters, the processing request to the target Pod corresponding to the PodIP and the Pod port for processing according to the cluster VIP and the host port.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology of financial technology (Fintech), and relate to but are not limited to a container load balancing method, device, equipment and storage medium. Background Art

[0002] With the development of computer technology, more and more technologies are being applied in the financial field. The traditional financial industry is gradually transforming into financial technology (Fintech). However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology.

[0003] In the field of financial technology, the container orchestration and scheduling system (Kubernetes, K8s) supports access to the running unit (pod) of the container through the service (Service), and creates filtering system rules such as iptables rules through a network proxy (kube-proxy) component on the K8s working node to forward the proxy virtual Internet Protocol (Vip) address. When accessing K8s outside the cluster, the entry point for accessing the Service through the external client of the cluster, such as NodePort, provides external access. For cross-node access to services, if the accessed node does not match the backend Pod, the node to be accessed needs to forward to the node where the Pod is located, and the processing performance is poor. Summary of the invention

[0004] The embodiments of the present application provide a container load balancing method, apparatus, device and storage medium to solve the problem in the related art that the accessed node and the backend Pod do not match, the node to be accessed needs to be forwarded to the node where the Pod is located, and the processing performance is poor.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a container load balancing method, including:

[0007] Receiving, through the inter-cluster load balancing server, a processing request sent by a service caller using a cluster virtual Internet Protocol address VIP and a host port corresponding to the cluster VIP;

[0008] The load balancing server between the clusters determines the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster;

[0009] The processing request is forwarded to the target Pod corresponding to the Pod IP and the Pod port for processing through the load balancing server between the clusters according to the cluster VIP and the host port.

[0010] The present application embodiment provides a container load balancing device, including:

[0011] The load balancing server between clusters is used to receive a processing request sent by a service caller using a cluster virtual Internet Protocol address VIP and a host port corresponding to the cluster VIP;

[0012] The inter-cluster load balancing server is used to determine the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster;

[0013] The inter-cluster load balancing server is used to forward the processing request to the target Pod corresponding to the Pod IP and the Pod port for processing according to the cluster VIP and the host port.

[0014] An embodiment of the present application provides a container load balancing device, including: a memory, used to store executable instructions; and a processor, used to implement the above method when executing the executable instructions stored in the memory.

[0015] An embodiment of the present application provides a storage medium storing executable instructions for causing a processor to execute and implement the above method.

[0016] The embodiments of the present application have the following beneficial effects: the processing request sent by the service caller using the cluster virtual Internet Protocol address VIP and the host port corresponding to the cluster VIP is received by the load balancing server between clusters; the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port are determined by the load balancing server between clusters according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster; it can be seen that the present application establishes the load balancing strategy on the host, and the Pod and the host establish port mapping to realize access to the Pod outside the cluster and expose the cluster VIP to the outside; further, the load balancing server between clusters forwards the processing request to the target Pod corresponding to the Pod IP and the Pod port for processing according to the cluster VIP and the host port. That is to say, when accessing the Pod outside the cluster through the cluster VIP and the host port, the uniqueness of the Pod IP pointed to while using the host network can be ensured, thereby avoiding the node forwarding to the node where the Pod is located, thereby improving the processing performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is an optional architecture diagram of a server provided in an embodiment of the present application;

[0018] Figure 2 This is a flow diagram of the container load balancing method provided in the embodiment of the present application. Figure 1 ;

[0019] Figure 3 It is a schematic diagram of the server cluster architecture provided in an embodiment of the present application;

[0020] Figure 4 This is a flow diagram of the container load balancing method provided in the embodiment of the present application. Figure 2 ;

[0021] Figure 5 This is a flow diagram of the container load balancing method provided in the embodiment of the present application. Figure 3 ;

[0022] Figure 6 It is a schematic diagram of the load balancing scenario provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0024] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art of the technical field of the embodiments of the present application. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0025] The following describes an exemplary application of the container load balancing device provided in the embodiment of the present application. The container load balancing device provided in the embodiment of the present application can be implemented as any computer terminal with a screen display function, such as a laptop, a tablet computer, a desktop computer, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device), an intelligent robot, etc., and can also be implemented as a server. The following describes an exemplary application of the container load balancing device when it is implemented as a server.

[0026] See also Figure 1 , Figure 1 is a schematic diagram of the structure of a server cluster 100 provided in an embodiment of the present application, Figure 1 The server cluster 100 shown includes: at least one processor 110, at least one network interface 120, a user interface 130 and a memory 150. The various components in the server cluster 100 are coupled together via a bus system 140. It is understood that the bus system 140 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 140 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 140 is not described in detail. Figure 1 Various buses are labeled as bus system 140 .

[0027] The processor 110 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0028] The user interface 130 includes one or more output devices 131 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 130 also includes one or more input devices 132, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0029] The memory 150 may be removable, non-removable or a combination thereof. Exemplarily, hardware devices include solid-state memory, hard disk drive, optical disk drive, etc. The memory 150 may optionally include one or more storage devices physically away from the processor 110. The memory 150 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 150 described in the embodiments of the present application is intended to include any suitable type of memory. In some embodiments, the memory 150 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as exemplarily described below.

[0030] Operating system 151, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0031] A network communication module 152, for reaching other computing devices via one or more (wired or wireless) network interfaces 120, exemplary network interfaces 120 include: Bluetooth, Wireless Fidelity (Wi-Fi), and Universal Serial Bus (USB);

[0032] The input processing module 153 is used to detect one or more user inputs or interactions from one of the one or more input devices 132 and translate the detected inputs or interactions.

[0033] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 1 A container load balancing device 154 stored in the memory 150 is shown. The container load balancing device 154 may be a container load balancing device in the server cluster 100. It may be software in the form of a program or a plug-in, and includes the following software modules: a resource management component 1541 and a load balancing server 1541. These modules are logical, and therefore may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0034] In other embodiments, the device provided in the embodiments of the present application may be implemented in hardware. As an example, the device provided in the embodiments of the present application may be a processor in the form of a hardware decoding processor, which is programmed to execute the container load balancing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor may adopt one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), DSP, programmable logic device (Programmable Logic Device, PLD), complex programmable logic device (Complex Programmable Logic Device, CPLD), field programmable gate array (Field-Programmable Gate Array, FPGA) or other electronic components.

[0035] The following will describe the container load balancing method provided by the embodiment of the present application in conjunction with the exemplary application and implementation of the server cluster 100 provided by the embodiment of the present application. Figure 2 , Figure 2 This is an optional flow chart of the container load balancing method provided in the embodiment of the present application, which will be combined with Figure 2 The steps shown are explained,

[0036] Step S201, receiving, through the inter-cluster load balancing server, a processing request sent by a service caller using a cluster virtual Internet Protocol address VIP and a host port corresponding to the cluster VIP.

[0037] In an embodiment of the present application, a server cluster includes multiple container orchestration engine clusters such as multiple K8s clusters, load balancing servers between clusters, and resource management components between clusters. Among them, K8s is an open source container orchestration engine that supports automated deployment, large-scale scalability, and application container management. In K8s, multiple containers can be created, each of which runs an application instance, and then the management, discovery, and access to this group of application instances are achieved through the built-in load balancing strategy. The load balancing server between clusters is at least responsible for managing the load balancing configuration parameters of the nodes of the real service (Real Server), that is, the Pod in the cluster, and creating network tunnels such as IPIP tunnels. The resource management component between clusters is at least used to manage the binding relationship between Service, Service, and pod.

[0038] In the embodiment of the present application, Service is a core resource object in K8s, and a Service is provided by a container group (Pod Group) as the backend Real Server, that is, it is provided by multiple Pod instances. Exemplarily, the Service mentioned here refers to a 4-layer network service, and the 4 layers include the application layer, the transport layer, the network layer, and the data link layer. A Pod Group can span multiple clusters and be associated through the cluster identification number (cluster Id) stored in the resource management component between clusters. The original Service of K8s is not used, that is, the Service is defined independently outside the K8s cluster. Define the Service outside the cluster, do not use the Service inside K8s, and realize common load balancing across multiple K8s clusters. The present application can support Pod common proxying of multiple K8s clusters. The capacity of each K8s cluster is limited, and most Internet technology (IT) systems can deploy multiple clusters to combine services.

[0039] In the embodiment of the present application, the Pod Group data structure is as follows:

[0040] pod_group_name vip_port Others… A 10.0.0.4:8080

[0041] The Pod Group data structure shows that for the Pod Group with the container group name (pod_group_name) as A, its vip_port is 10.0.0.4:8080.

[0042] pod_name pod_group_name Others… P1 A P2 A

[0043] Furthermore, the Pod Group whose container name (pod_name) is A includes two containers, namely P1 and P2.

[0044] In actual applications, Service can be provided to the service caller in the form of VIP+port. Service defines an access entry address for the service, namely the cluster VIP. The front-end application accesses the IP of its back-end Pod through the cluster VIP. Here, the front-end application is the application provided by the service caller.

[0045] In some embodiments, the service caller uses the cluster VIP and the host port corresponding to the cluster VIP to send a processing request, and the inter-cluster load balancing server can receive the processing request sent by the service caller using the cluster VIP and the host port corresponding to the cluster VIP. Here, when creating a Service, the inter-cluster resource management component allocates a cluster VIP and the host port corresponding to the cluster VIP to the Service.

[0046] Step S202, the load balancing server between clusters determines the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster.

[0047] In the embodiment of the present application, the destination address conversion relationship includes the mapping relationship between the cluster VIP and the Pod IP, and the mapping relationship between the host port and the Pod port. The load balancing configuration parameters of the Pod in the cluster are maintained by the load balancing server between clusters.

[0048] In some embodiments, when the load balancing server between clusters receives a processing request sent by a service caller using a cluster VIP and a host port corresponding to the cluster VIP, the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port are determined according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster, that is, the IPIP tunnel is determined, and the cluster VIP corresponding to the processing request is pointed to the real IP and port of the Pod to ensure that the backend Pod can be accessed when accessing the host IP: port. Here, in the process of determining the IPIP tunnel, the load balancing configuration parameters of the Pod in the cluster are also introduced, that is, when the service caller initiates a processing request to access the service, the processing request is forwarded to the backend Pod through load balancing methods such as polling, forwarding according to weight, etc., so that the backend Pod load is equal.

[0049] Step S203, the load balancing server between clusters forwards the processing request to the target Pod corresponding to the Pod IP and Pod port for processing according to the cluster VIP and the host port.

[0050] In the embodiment of the present application, the K8s cluster includes a cluster node (Node), which is a working host in the K8s cluster relative to the master component (Master), also known as a container host or a container mother machine, and is a resource pool for the operation of business containers. Node can be a physical host or a virtual machine (Virtual Machine, VM). In practical applications, multiple containers are combined in a Node through the concept of Pod. It should be noted that the IP of the Pod is private, and the load balancing server between clusters, that is, the load balancer or service caller outside the cluster cannot be directly connected. However, in the embodiment of the present application, when the load balancing server between clusters determines the IPIP tunnel, the processing request is forwarded to the target Pod for processing. It can be seen that the present application realizes that the service caller and the Node can communicate with each other through physical devices, and the IPs of both parties are routable, thereby realizing the access of the host agent container to achieve external communication.

[0051] In some feasible scenarios, the host and the Pod are in two network namespaces (NetworkNamespace), which are not interoperable. In the process of forwarding the request to access the host to the Pod, the configuration of the kernel's packet processing module Netfilter is realized by calling an IP packet filtering system such as iptables, and the rule, i.e., the above-mentioned destination address conversion relationship, is created during the configuration process. At the same time, in order to avoid the influence of the iptables rules that come with K8s and the application container engine (docker), this application puts the created rule, i.e., the above-mentioned destination address conversion relationship, at the forefront of all rules and gives priority to matching. The creation rules are as follows: Create a destination address translation (Destination Network Address Translation, DNAT) chain, and point the target address, i.e., the cluster VIP: port adopted by the service caller, to the real IP and port of the Pod. For example, transfer port 8080 of the host, i.e., the mother machine, to 10.244.0.29:80 (the real IP of the Pod). At this time, accessing the mother machine IP: 8080 can access the container.

[0052] The creation example is as follows:

[0053] $ iptables -t nat-nL

[0054] DNAT tcp--0.0.0.0 / 0 0.0.0.0 / 0tcp dpt:8080to:10.244.0.29:80

[0055] From the above, it can be seen that in the embodiment of the present application, the IPIP tunnel is determined by the load balancing server between clusters, and the network policy and access endpoints are converged to a limited host IP, without wasting the resources of irrelevant nodes (such as ports, memory, etc.). It is safe and controllable, does not require forwarding between nodes, and has better performance.

[0056] The container load balancing method provided by the present application receives the processing request sent by the service caller using the cluster virtual Internet Protocol address VIP and the host port corresponding to the cluster VIP through the load balancing server between clusters; the load balancing server between clusters determines the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster; it can be seen that the present application establishes the load balancing strategy on the host, and the Pod and the host establish port mapping to realize access to the Pod outside the cluster and expose the cluster VIP to the outside; further, the load balancing server between clusters forwards the processing request to the target Pod corresponding to the Pod IP and the Pod port for processing according to the cluster VIP and the host port. That is to say, when accessing the Pod outside the cluster through the cluster VIP and the host port, the uniqueness of the Pod IP pointed to while using the host network can be ensured, thereby avoiding the node forwarding to the node where the Pod is located, thereby improving the processing performance.

[0057] In an achievable server cluster architecture, for example, see Figure 3 As shown, the server cluster includes multiple K8s clusters including but not limited to K8s-1 and K8s-2, inter-cluster load balancing servers such as webank loadbalancer Admin Service, referred to as WELB Admin Service, and inter-cluster resource management components such as Resource Manager, referred to as RM. The K8s cluster includes API servers, load balancing controllers such as WELB WebhookController and Node.

[0058] Among them, RM is used to manage the binding relationship between Service and Pod, which is shared by multiple K8s clusters. RM obtains container-related change information through clients such as the core toolkit Informer in client-go. Through K8sinformer, RM obtains information such as Pod IP and Pod's host IP. This information is used as the key to iptables configuration and welbadmin Service delivery strategy.

[0059] The load balancing controller, WELB Webhook Controller, is deployed in K8s in the form of deployment. Configure webhook in K8s, listen to the Pod creation event, query RM by Pod name for each Pod to be created, and call WELB Admin Service to update the configuration if the Pod has a bound Service. Exclusive to the K8s cluster. Using the event callback capability of Webhook, the load balancing network on the Node can be properly configured when the container is created and destroyed. WELBAdmin Service is responsible for managing the configuration of the Real Server nodes, such as creating IPIP tunnels, etc., which are shared by multiple K8s clusters.

[0060] In other embodiments of the present application, in step S202, before determining the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port through the inter-cluster load balancing server according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster, the load balancing configuration parameters of the Pod in the cluster can be updated by the following method:

[0061] Step 11. Through the API server in the cluster, call the webhook configured on the load balancing server to listen to the Pod creation event.

[0062] Step 12. Use the inter-cluster resource management component to query whether the Pod name carried in the Pod creation event is bound to a service;

[0063] Step 13. If a service is found to be bound, call the inter-cluster load balancing server to update the load balancing configuration parameters of the Pod in the cluster corresponding to the Pod name.

[0064] For example, see Figure 4 As shown in the figure, when the application is released or restarted, the application management software (APPManger) sends a query request to RM to query the cluster ID where the Pod is located.

[0065] Through the API server in the cluster, the webhook configured by the load balancing server is called to listen to the Pod creation event, and the Pod creation event is sent to the RM. The RM queries whether the Pod name carried in the Pod creation event is bound to a service. If it is found that there is a bound service, the load balancing server between clusters is called to update the load balancing configuration parameters of the Pod in the cluster corresponding to the Pod name, that is, to turn on the RS weight. In other words, this application uses the event callback capability of Webhook to add configuration to the pod that needs load balancing through webhook, supports on-demand configuration, and ensures that the application configuration is non-invasive.

[0066] In other embodiments of the present application, before Step 12 queries the Pod name carried in the Pod creation event through the inter-cluster resource management component to see whether there is a service bound to it, the service can be created by the following method:

[0067] Step 21. Receive the input Pod name and Pod port through the inter-cluster resource management component;

[0068] Step 22. Use the inter-cluster resource management component to find the unused port on the host corresponding to the Pod name and obtain the host port.

[0069] Step 23. Create a service through the inter-cluster resource management component and assign cluster VIP and host port to the service.

[0070] In the embodiment of the present application, service creation is implemented through Step 21-Step 23, and when the service is created, a cluster VIP and port are allocated to the service.

[0071] For example, in the process of creating a service, in Step 21, when the input Pod name and Pod port are received by the inter-cluster resource management component, see Figure 5 As shown, the host corresponding to the Pod name is searched through the inter-cluster resource management component to check whether there is a port conflict. If a port conflict occurs, such as a port being occupied, it is migrated to another container. If there is an available port, an unused host port is selected as the above host port. Further, a service is created through the inter-cluster resource management component, and a cluster VIP and host port are allocated to the service, and the configuration is issued to the host.

[0072] In other embodiments of the present application, Step 22 searches for unused ports on the host machine corresponding to the Pod name through the inter-cluster resource management component. Before obtaining the host port, the correspondence between the Pod and the host machine can be set by the following method: through the inter-cluster resource management component, determine the host machine corresponding to the Pod name one by one in the Pod group within the cluster, which is the host machine corresponding to the Pod name.

[0073] In some feasible scenarios, when RM automatically allocates resources, it breaks up the instance allocation of the same Pod Group and does not allow two Pods to share the same host. Check the ports that have been allocated on the host. If there are no suitable ports, filter out this host.

[0074] In other embodiments of the present application, in step S202, before determining the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port through the inter-cluster load balancing server according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster, the load balancing configuration parameters of the Pod in the cluster can be updated by the following method:

[0075] Step 31. Call the load balancing server between clusters to obtain the total weight sum(c_wi) of all clusters and the cluster weight c_wi of cluster i in all clusters, where i is a positive integer greater than or equal to 1 and less than or equal to n, and n is the total number of all clusters;

[0076] Step 32. Call the load balancing server between clusters to obtain the total weight sum(r_wj) of all Pods in cluster i and the Pod weight r_wj of Pod j among all Pods, where j is a positive integer greater than or equal to 1 and less than or equal to m, and m is the total number of all Pods;

[0077] Step 33. Call the load balancing server between clusters and calculate the load balancing configuration parameters of the Pod in the cluster based on sum(c_wi), c_wi, sum(r_wj) and r_wj.

[0078] From the above, it can be seen that the embodiment of the present application supports cluster weighting, and different Pods in the cluster have version grayscale requirements, so it also supports Pods to define different weights, and the setting of Pod load balancing configuration parameters is realized through the above Step 31-Step 33.

[0079] Furthermore, before calling the load balancing server between clusters to forward the processing request to the backend Node, see Figure 6As shown in the figure, the configuration is sent to the load balancing director (LD) on the one hand, and to the Node on the other hand. The configuration on the LD mainly includes the backend Node IP list and weight. The configuration on the Node mainly includes the port mapping and IPIP tunnel configuration. Among them, the LD can adopt the Open Shortest Path First (OSPF) strategy to forward the processing request to the backend Node.

[0080] In other embodiments of the present application, Step 33, calling the load balancing server between clusters, and calculating the load balancing configuration parameters of the Pod in the cluster according to sum(c_wi), c_wi, sum(r_wj) and r_wj, can be implemented by the following method:

[0081] Call the load balancing server between clusters and divide c_wi by sum(c_wi) to get the value as the frequency of cluster i being accessed;

[0082] Call the load balancing server between clusters and divide r_wj by sum(r_wj) to get the value as the access frequency of Podj in cluster i;

[0083] The load balancing server between clusters is called, and the product of the frequency of cluster i being accessed and the frequency of Pod j being accessed is used as the global weight of Pod j. The load balancing configuration parameters of the Pods in the cluster include the global weight of Pod j.

[0084] In some embodiments, when the global weight of Pod j is obtained, if the value is not an integer, a rounding operation is performed. Further, different Pods in the cluster are marked to obtain RS set RS = {R0, R1, ..., Rn}; weight set W = {W0, W1, ..., Wn}. Exemplarily, an empty RS list (List) can also be initialized, and W0 R0s are added to the list, and W1 R1s are added to the list. Traversing the RS List implements the weighted scheduling method.

[0085] In other embodiments of the present application, the load balancing configuration parameters of the Pod in the cluster in step S202 can also be set in the following manner: through the application program interface API server in the cluster, the load balancing configuration parameters of the Pod corresponding to the Pod name carried in the Pod creation event and / or the Pod destruction event are set to preset values.

[0086] In the embodiment of the present application, there is no need to explicitly write the load balancing configuration in another markup language (yaml) of the Pod. Instead, the pod creation and destruction events of the webhook are used to call RM to query whether load balancing configuration is needed, and finally the welb admin Service is called to configure the node network. This maintains the simplicity of the Pod Yaml, avoids coupling with load balancing, and supports deployment in other non-specific K8s clusters. Furthermore, it supports deep integration of k8s management of pods and pod lifecycles, and automatically configures the traffic weight to a preset value such as 0 before pod creation and destruction to ensure the stability of the service.

[0087] The following is a description of an exemplary structure of a container load balancing device 154 provided in an embodiment of the present application implemented as a software module. In some embodiments, Figure 1 As shown, the software module stored in the container load balancing device 154 of the memory 150 may be a container load balancing device in the server cluster 100, including:

[0088] The inter-cluster load balancing server 1541 is used to receive a processing request sent by a service caller using a cluster virtual Internet Protocol address VIP and a host port corresponding to the cluster VIP;

[0089] The inter-cluster load balancing server 1541 is used to determine the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster;

[0090] The inter-cluster load balancing server 1541 is used to forward the processing request to the target Pod corresponding to the Pod IP and Pod port for processing according to the cluster VIP and the host port.

[0091] In other embodiments of the present application, the container load balancing device 154 also includes an application programming interface API server 1542 within the cluster and a resource management component 1543 between clusters, wherein the API server 1542 is used to call the network hook webhook configured by the load balancing server to listen to the Pod creation event; the resource management component 1543 between clusters is used to query whether the Pod name carried by the Pod creation event is bound to a service; the load balancing server 1541 between clusters is used to update the load balancing configuration parameters of the Pod in the cluster corresponding to the Pod name if it is found that there is a bound service.

[0092] In other embodiments of the present application, the inter-cluster resource management component 1543 is used to receive the input Pod name and Pod port; search for unused ports on the host machine corresponding to the Pod name to obtain the host machine port; create a service, and allocate a cluster VIP and host machine port to the service.

[0093] In other embodiments of the present application, the inter-cluster resource management component 1543 is used to determine the host machine corresponding to the Pod name in the Pod group within the cluster, which is the host machine corresponding to the Pod name.

[0094] In other embodiments of the present application, the inter-cluster load balancing server 1541 is used to obtain the total weight sum(c_wi) of all clusters and the cluster weight c_wi of cluster i in all clusters, where i is a positive integer greater than or equal to 1 and less than or equal to n, and n is the total number of all clusters; obtain the total weight sum(r_wj) of all Pods in cluster i and the Pod weight r_wj of Pod j in all Pods, where j is a positive integer greater than or equal to 1 and less than or equal to m, and m is the total number of all Pods; calculate the load balancing configuration parameters of the Pods in the cluster based on sum(c_wi), c_wi, sum(r_wj) and r_wj.

[0095] In other embodiments of the present application, the inter-cluster load balancing server 1541 is used to use the value obtained by dividing c_wi by sum(c_wi) as the frequency at which cluster i is accessed; use the value obtained by dividing r_wj by sum(r_wj) as the frequency at which Pod j in cluster i is accessed; and use the product of the frequency at which cluster i is accessed and the frequency at which Pod j is accessed as the global weight of Pod j. The load balancing configuration parameters of the Pod in the cluster include the global weight of Pod j.

[0096] In other embodiments of the present application, the API server 1542 is used to call the webhook configured by the load balancing server to listen to the Pod creation event and / or the Pod destruction event; and set the load balancing configuration parameters of the Pod corresponding to the Pod name carried in the Pod creation event and / or the Pod destruction event to a preset value.

[0097] The container load balancing device provided by the present application receives the processing request sent by the service caller using the cluster virtual Internet Protocol address VIP and the host port corresponding to the cluster VIP through the load balancing server between clusters; the load balancing server between clusters determines the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster; it can be seen that the present application establishes the load balancing strategy on the host, and the Pod and the host establish port mapping to realize access to the Pod outside the cluster and expose the cluster VIP to the outside; further, the load balancing server between clusters forwards the processing request to the target Pod corresponding to the Pod IP and the Pod port for processing according to the cluster VIP and the host port. That is to say, when accessing the Pod outside the cluster through the cluster VIP and the host port, the uniqueness of the Pod IP pointed to while using the host network can be ensured, thereby avoiding the node forwarding to the node where the Pod is located, thereby improving the processing performance.

[0098] It should be noted that the description of the device of the embodiment of the present application is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it is not repeated. For technical details not disclosed in the embodiment of the device, please refer to the description of the method embodiment of the present application for understanding.

[0099] The present application embodiment provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method provided by the present application embodiment, for example, Figure 2 The method shown.

[0100] The storage medium provided by the present application receives the processing request sent by the service caller using the cluster virtual Internet Protocol address VIP and the host port corresponding to the cluster VIP through the load balancing server between clusters; the load balancing server between clusters determines the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster; it can be seen that the present application establishes the load balancing strategy on the host, and the Pod and the host establish port mapping to realize access to the Pod outside the cluster and expose the cluster VIP to the outside; further, the load balancing server between clusters forwards the processing request to the target Pod corresponding to the Pod IP and the Pod port for processing according to the cluster VIP and the host port. That is to say, when accessing the Pod outside the cluster through the cluster VIP and the host port, the uniqueness of the Pod IP pointed to while using the host network can be ensured, thereby avoiding the node forwarding to the node where the Pod is located, thereby improving the processing performance.

[0101] In some embodiments, the storage medium can be a computer-readable storage medium, for example, a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disk read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above memories.

[0102] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0103] As examples, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a Hypertext Markup Language (Hypertext Markup Language) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As examples, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed at multiple sites and interconnected by a communication network.

[0104] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A container load balancing method, characterized in that: include: The service caller receives a processing request sent by a cluster virtual Internet Protocol address VIP and a host port corresponding to the cluster VIP through an inter-cluster load balancing server. The cluster VIP is an access entry address defined for the service. By defining the service outside the cluster, cross-cluster common load balancing is achieved; the inter-cluster load balancing server determines the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster; Forwarding the processing request to the target Pod corresponding to the Pod IP and the Pod port for processing by the inter-cluster load balancing server according to the cluster VIP and the host port; forwarding the processing request to the target Pod corresponding to the Pod IP and the Pod port for processing according to the cluster VIP and the host port, including: The kernel data packet configuration is implemented by calling the IP packet filtering system, and the destination address translation relationship is created during the configuration process; the rule for creating the destination address translation relationship includes: pointing the cluster VIP: port used by the service caller to the real IP and port of the Pod, and the destination address translation relationship is at the forefront of all rules.

2. The method according to claim 1, characterized in that Before determining the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port through the inter-cluster load balancing server according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster, the method also includes: Through the API server in the cluster, the webhook configured on the load balancing server is called to listen to the Pod creation event. Query the Pod name carried in the Pod creation event through the inter-cluster resource management component to see whether there is a service bound to it; If a service is found to be bound, the load balancing server between the clusters is called to update the load balancing configuration parameters of the Pod in the cluster corresponding to the Pod name.

3. The method according to claim 2, characterized in that Before querying, by the resource management component, whether the Pod name carried in the Pod creation event is bound to a service, the method further includes: Receive the input Pod name and the Pod port through the inter-cluster resource management component; Searching for an unused port on the host machine corresponding to the Pod name through the inter-cluster resource management component to obtain the host machine port; The service is created through the inter-cluster resource management component, and the cluster VIP and the host port are allocated to the service.

4. The method according to claim 3, characterized in that: Before searching for an unused port on the host machine corresponding to the Pod name through the inter-cluster resource management component and obtaining the host machine port, the method includes: A host machine corresponding to the Pod name in one-to-one relationship is determined in the Pod group within the cluster through the inter-cluster resource management component, and is the host machine corresponding to the Pod name.

5. The method according to claim 1, characterized in that Before determining the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port through the inter-cluster load balancing server according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster, the method further includes: Call the load balancing server between the clusters to obtain the total weight sum(c_wi) of all clusters and the cluster weight c_wi of cluster i in all clusters, where i is a positive integer greater than or equal to 1 and less than or equal to n, and n is the total number of all clusters; Call the load balancing server between the clusters to obtain the total weight sum(r_wj) of all Pods in the cluster i and the Pod weight r_wj of Pod j among all the Pods, where j is a positive integer greater than or equal to 1 and less than or equal to m, and m is the total number of all the Pods; The load balancing server between the clusters is called, and the load balancing configuration parameters of the Pods in the cluster are calculated according to the sum(c_wi), the c_wi, the sum(r_wj) and the r_wj.

6. The method according to claim 1, characterized in that The calling of the load balancing server between the clusters, and calculating the load balancing configuration parameters of the Pod in the cluster according to the sum(c_wi), the c_wi, the sum(r_wj) and the r_wj, includes: Calling the load balancing server among the clusters, and dividing the c_wi by the sum(c_wi) to obtain a value as the frequency of the cluster i being accessed; Call the load balancing server between the clusters, and divide the r_wj by the sum(r_wj) to obtain a value as the access frequency of the Pod j in the cluster i; The load balancing server among the clusters is called, and the product of the frequency at which the cluster i is accessed and the frequency at which the Pod j is accessed is used as the global weight of the Pod j. The load balancing configuration parameters of the Pods within the cluster include the global weight of the Pod j.

7. The method according to any one of claims 1 to 6, characterized in that Before determining the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port through the inter-cluster load balancing server according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster, the method also includes: Through the API server in the cluster, the webhook configured on the load balancing server is called to listen to Pod creation events and / or Pod destruction events; Through the application program interface API server in the cluster, the load balancing configuration parameters of the Pod corresponding to the Pod name carried in the Pod creation event and / or the Pod destruction event are set to preset values.

8. A container load balancing device, characterized in that: include: The inter-cluster load balancing server is used to receive a processing request sent by a service caller using a cluster virtual Internet Protocol address VIP and a host port corresponding to the cluster VIP. The cluster VIP is an access entry address defined for the service. By defining the service outside the cluster, cross-cluster common load balancing is achieved. The inter-cluster load balancing server is used to determine the running unit Pod IP of the container corresponding to the cluster VIP in the cluster and the Pod port corresponding to the host port according to the destination address conversion relationship and the load balancing configuration parameters of the Pod in the cluster; The inter-cluster load balancing server is used to forward the processing request to the target Pod corresponding to the Pod IP and the Pod port for processing according to the cluster VIP and the host port; Forwarding the processing request to the target Pod corresponding to the Pod IP and the Pod port for processing according to the cluster VIP and the host port, including: The kernel data packet configuration is implemented by calling the IP packet filtering system, and the destination address translation relationship is created during the configuration process; the rule for creating the destination address translation relationship includes: pointing the cluster VIP: port used by the service caller to the real IP and port of the Pod, and the destination address translation relationship is at the forefront of all rules.

9. A container load balancing device, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the method according to any one of claims 1 to 7 when executing the executable instructions stored in the memory.

10. A storage medium, characterized in that: Executable instructions are stored, which are used to cause a processor to execute and implement the method described in any one of claims 1 to 7.

Images