Cluster, cluster management method and cluster management program
The cluster management method facilitates quick switching to an alternate cluster by executing alternative programs and updating DNS records, addressing the inefficiencies of conventional methods and ensuring rapid application transition.
Patent Information
- Application Number
- JP2023014130
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-02-01
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2043-02-01
AI Technical Summary
Conventional cluster switching methods are slow and inefficient, particularly when switching from a production server to another server with a changed configuration, due to the need to update DNS records across multiple servers and client caches, which can lead to incomplete switching and inability to switch applications promptly.
A cluster management method that includes a storage unit and processor to execute alternative programs on an alternate cluster in response to client requests, allowing for quick switching by forwarding requests to the alternate cluster and updating DNS records to point to the alternate IP address.
Enables faster cluster switching by immediately transitioning to an alternate cluster without waiting for DNS updates across all servers and caches, ensuring seamless application switching and reducing downtime.
Smart Images

Figure 0007578735000001 
Figure 0007578735000002 
Figure 0007578735000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a cluster, a cluster management method, and a cluster management program. [Background technology]
[0002] A system called a cluster is used. A cluster is a system that combines multiple computers and operates them as if they were a single computer. In many cases, the cluster is connected to a network. Users can connect their client terminals to the cluster via the network and use the software of the cluster by operating the client terminals.
[0003] Even if some of the computers constituting the cluster fail due to a fault or the like and stop working, the users can still use the cluster using their client terminals. Furthermore, even while the failed computers are being repaired or replaced, the users can still use the cluster using their client terminals.
[0004] There are cases where a program running on a server that is a component of a cluster is changed. For example, there are cases where the operating system of the server is updated, the application of the server is updated, or new software is introduced to the server. There is a technique for suppressing the occurrence of problems when changing the configuration of a program running on a server of a cluster. For example, in the technique described in Patent Document 1, when a newer version of a container image is released than the version of the container image used on a production server that uses containers, the newer version of the container image is used on a verification server. In the technique described in Patent Document 1, the operation of the newer version of the container image is monitored and verified at that time. Therefore, by using the technique described in Patent Document 1, it is possible to verify whether or not a newer version of a container image can be used without problems before using a newer version of the container image than the version used on the production server on the production server. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2019-56986 A Summary of the Invention [Problem to be solved by the invention]
[0006] However, the technology described in Patent Document 1 does not assume a case where a production server is switched to another server with a part of the configuration of the production server modified. Therefore, even if the technology described in Patent Document 1 is used, it cannot deal with problems that arise when a production server is switched to another server with a part of the configuration of the production server modified.
[0007] For example, in the past, when switching from a first cluster to a second cluster, the DNS record of the domain name of the first cluster stored in the DNS server (hereinafter referred to as the "first DNS record") is rewritten as follows: Before switching, the domain name of the first cluster and the IP address of the first cluster are stored in association with each other in the first DNS record of the DNS server. When a client terminal attempts to access the first cluster using the domain name of the first cluster, the IP address of the first cluster stored in the first DNS record of the DNS server is referenced, and the client terminal can access the first cluster.
[0008] When switching, the domain name of the first cluster and the IP address of the second cluster are associated and stored in the first DNS record of the DNS server. As a result, when a client terminal tries to access the first cluster using the domain name of the first cluster, the IP address of the second cluster stored in the first DNS record of the DNS server is referenced, and the client terminal can access the second cluster. In this way, by changing the IP address stored in the first DNS record, it is possible to switch from the first cluster to the second cluster.
[0009] However, there are many DNS servers. It is not easy to immediately change the first DNS records of all DNS servers. Until the first DNS records of all DNS servers have been changed, there is a risk that a client terminal will attempt to access the first cluster using the domain name of the first cluster, and will access the first cluster by referring to the first DNS record of a DNS server whose first DNS record has not been changed. Therefore, even if the first DNS records of some DNS servers are rewritten, the switch from the first cluster to the second cluster will not be completed until the first DNS records of all DNS servers have been changed.
[0010] Furthermore, the client terminal usually has a cache that stores the DNS record, and when the client terminal tries to access the first cluster, the client terminal accesses the first cluster until the change of the first DNS record stored in the cache is completed, i.e., the switch from the first cluster to the second cluster is not completed until the change of the first DNS record stored in the cache of the client terminal is completed.
[0011] As described above, in the conventional cluster switching method, even if the DNS record of the DNS server is changed, the cluster switching may not be completed immediately. Also, in the conventional cluster switching method, although the cluster switching can be performed on a cluster basis, the application (program) switching cannot be performed.
[0012] SUMMARY OF THE PRESENT EMBODIMENTS An object of the present invention is to provide a cluster, a cluster management method, and a cluster management program that enable faster cluster switching. [Means for solving the problem]
[0013] In order to achieve the above-mentioned object, one aspect of the cluster management method of the present invention is a cluster management method in a cluster having multiple nodes each having a memory unit that stores a program to be executed in response to a request from a client terminal connected to a network, and a processor that executes the program, wherein when the processor receives a request for the target program sent from the client terminal and executes at least one alternative program stored in an alternative cluster in place of at least one target program stored in the cluster, the processor executes a request forwarding process to forward the request for the target program to the alternative cluster so that the alternative program is executed in response to the request for the target program.
[0014] Furthermore, one aspect of a cluster of the present invention is a cluster having a plurality of nodes each having a memory unit for storing a program to be executed in response to a request from a client terminal connected to a network, and a processor for executing the program, and when an alternative cluster is to execute at least one of a tons of alternative programs stored therein in place of at least one target program stored in the cluster, upon receiving a request for the target program sent from the client terminal, the cluster executes a request forwarding process in which the request for the target program is forwarded to the alternative cluster so that the alternative program is executed in response to the request for the target program. Effect of the Invention
[0015] According to a representative embodiment of the present invention, cluster switching can be performed more quickly. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]
[0016] [Figure 1] FIG. 1 is a block diagram showing an outline of the configuration of a cluster system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of the worker node 200A. [Diagram 3] FIG. 3 is a block diagram showing an example of the functional configuration of the worker pod 230. As shown in FIG. [Figure 4] FIG. 4 is a diagram illustrating an example of data stored in the request queue. [Diagram 5] FIG. 5 is an explanatory diagram illustrating a configuration in a state (before switching) in which the cluster 1A is used and the alternative cluster 1B is not used. [Figure 6] FIG. 6 is a flowchart showing an example of a procedure for switching the cluster 1A to the alternative cluster 1B in the embodiment (A). [Figure 7] 1 is an explanatory diagram showing a state in which the transfer pods 210A and 210B have been deployed in step S101. [Figure 8] FIG. 8 is an explanatory diagram showing a state in which the load balancers 300A and 300B and the router pods 220A and 220B are set in step S102. [Figure 9] FIG. 9 is an explanatory diagram showing a state in which the transfer pod 210A is set in step S103. [Figure 10] FIG. 10 is an explanatory diagram showing the state in which the DNS server 600 is set in step S107. [Figure 11] FIG. 11 is an explanatory diagram showing a state in which the load balancers 300A and 300B have been set and the transfer pods 210A and 210B have been deleted in step S110. [Figure 12] FIG. 12 is an explanatory diagram illustrating the configuration of the state before the procedure for switching the worker pod 230Ax of the cluster 1A to the worker pod 230B of the alternative cluster 1B is executed (state before switching). [Figure 13] FIG. 13 is a flowchart showing an example of a procedure for switching a worker pod 230Ax in cluster 1A to a worker pod 230B in an alternative cluster 1B. [Figure 14] FIG. 14 is an explanatory diagram showing a state in which the transfer pods 210A and 210B have been deployed in step S201. [Figure 15] FIG. 15 is an explanatory diagram showing a state in which the load balancers 300A and 300B and the router pods 220A and 220B are set in step S202. [Figure 16] FIG. 16 is an explanatory diagram showing the state in which the transfer pod 210A is set in step S203. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0017] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention should not be interpreted as being limited to the description of the embodiment shown below. It will be easily understood by those skilled in the art that the specific configuration can be changed without departing from the concept or purpose of the present invention.
[0018] In the configurations of the invention described below, the same or similar configurations or functions are given the same reference numerals, and duplicated explanations are omitted.
[0019] In this specification, the terms "first," "second," "third," and the like are used to identify components and do not necessarily limit the number or order.
[0020] In this specification, as an example of various information, the term "XX table" may be used, but it may also be expressed as a data structure such as an "XX list" or "XX queue." Furthermore, the "XX table" may also be expressed as "XX information." When explaining identification information, the terms "identification information," "identifier," "name," "ID," "number," and the like are used, but these are interchangeable.
[0021] <<System configuration>> Fig. 1 is a block diagram showing an outline of the configuration of a cluster system 1000 according to an embodiment. As shown in Fig. 1, the cluster system 1000 includes a cluster 1A, a load balancer 300A, an alternative cluster 1B, and a load balancer 300B. The cluster system 1000 is connected to client terminals 500 and 600 via a network NW.
[0022] A cluster is a system that groups multiple computers together to operate as if they were a single computer. A cluster has multiple nodes. A node is a virtual or physical computer. Containers and pods are created on the nodes. A container is a virtual OS environment that contains software. A pod contains one or more containers. A pod may contain one or more volumes.
[0023] The cluster 1A is connected to a load balancer 300A, a client terminal 500, a DNS server 600, and an alternative cluster 1B via a network NW. The alternative cluster 1B is a cluster used to replace the cluster 1A. The alternative cluster 1B has the same configuration as the cluster 1A.
[0024] Cluster 1A includes one master node 100A and multiple worker nodes 200A. Cluster 1A is a type of cluster. In this embodiment, cluster 1A is an example of a cluster in which a virtual OS environment (container) is created in the form of a pod on multiple servers and operated.
[0025] The master node 100A and the worker node 200A are nodes that include a storage device and a processor. The master node 100A and the worker node 200A can be realized by a general information processing device such as a PC or a server computer. In addition, there may be one or more worker nodes 200A.
[0026] The master node 100A manages a plurality of worker nodes 200A. As shown in Fig. 1, a transfer pod 210A, a router pod 220A, and worker pods 230A1 to 230An are created (deployed) as pods in each worker node 200A. The worker pods 230A to 230An are collectively referred to as "worker pod 230A." The number of each of the worker pods 230A1 to 230An may be one or more.
[0027] The master node 100A has a control plane 110A that manages the pods (forwarder pod 210A, router pod 220A, and worker pod 230A) of the multiple worker nodes 200A. The program that constitutes the control plane 110A is realized by executing one or more programs.
[0028] The transfer pod 210A is a pod that transfers a request from the client terminal 500 to the worker pod 230A to the alternative cluster 1B when the alternative cluster 1B is used as an alternative to the cluster 1A. The router pod 220A is a pod that transfers a request from the client terminal 500 to the worker pod 230A that is relatively lightly loaded. The transfer destination of the router pod 220A may be the worker pod 230A of a worker node 200A other than the worker node 200A in which the router pod 220A is created. The worker pod 230A is a pod that includes a program and executes processing in response to a request from the client terminal 500. The transfer pod 210A and the transfer pod 210B of the alternative cluster 1B are collectively referred to as the "transfer pod 210". The router pod 220A and the router pod 220B of the alternative cluster 1B are collectively referred to as the "router pod 220".
[0029] The load balancer 300A transfers a request from the client terminal 500 to the worker node 200A that is relatively lightly loaded. The load balancer 300A and the load balancer 300B are collectively referred to as "load balancer 300."
[0030] IP addresses are assigned to the master node 100A and the worker node 200A, the load balancers 300A and 300B, the client terminal 500, and the DNS server 600. Port numbers are assigned to the pods (transfer pods 210A and 210B, router pods 220A and 220B, and worker pods 230A and 230B) created in the nodes.
[0031] A domain name (for example, "example.com") is assigned to cluster 1A. Furthermore, among the multiple worker pods 230A, the worker pods 230A having the same configuration are assigned the same subdomain name (for example, "app1").
[0032] In the DNS server 600, the domain name of cluster 1A (for example, "example.com") is associated with the IP address of the load balancer 300A. The domain name of the worker pod 230A is the domain name of cluster 1A plus the subdomain name of the worker pod 230A (for example, "app1.example.com"). Of the worker pods 230A1 to 230An, the worker pod 230 to be used by the user can be specified by the domain name of the worker pod 230 (for example, "app1.example.com").
[0033] The DNS server 600 stores a DNS record 601 including information associating the IP address of a cluster with the domain name of the cluster. In Fig. 1 and other figures, many DNS records are omitted and only the DNS record 601 related to cluster 1A is shown.
[0034] The network NW may be a wired network or a wireless network, or may be a global network such as the Internet.
[0035] A user can execute a program held by the worker pod 230A by operating the client terminal 500 as follows. Here, information including an instruction for the program held by the worker pod 230A and the domain name of the worker pod 230A is referred to as a "request to the worker pod 230A."
[0036] The user operates the client terminal 500 to send a request to the worker pod 230A to a destination specified by the domain name (for example, "app1.example.com") of the worker pod 230A. Then, the DNS server 600 or the like is referenced, and the request to the worker pod 230A is sent to the load balancer 300A.
[0037] When the load balancer 300A receives a request for the worker pod 230A, the load balancer 300A transfers the request for the worker pod 230A to a worker node 200A that is relatively lightly loaded among the worker nodes 200A.
[0038] In worker node 200A, router pod 220A typically receives a request for worker pod 230A and transmits the request to worker pod 230A to a worker pod 230A that is relatively lightly loaded among the worker pods 230A specified in the request to worker pod 230A.
[0039] When the worker pod 230A receives a request for the worker pod 230A, a program contained in the worker pod 230A executes processing in response to the request for the worker pod 230A.
[0040] The alternative cluster 1B may be used in place of the cluster 1A on a trial basis, and if there are no problems with the alternative cluster 1B, the alternative cluster 1B may be used in place of the cluster 1A. In this manner, when the alternative cluster 1B is used in place of the cluster 1A, as will be described in detail below, in the worker node 200A, instead of the router pod 220A receiving a request to the worker pod 230A, the forwarding pod 210A receives the request.
[0041] Similar to the cluster 1A, the alternative cluster 1B includes a master node 100B having a control plane 110B, a forwarding pod 210B, a router pod 220B, and a worker node 200B having worker pods 230B (230B1 to 230Bn).
[0042] <Hardware configuration of worker node 200A, Fig. 2> Fig. 2 is a block diagram showing an example of a hardware configuration of the worker node 200A. As shown in Fig. 2, the worker node 200A has a processor 21, a main memory device 22, a secondary memory device 23, an input device 24, an output device 25, a network I / F 26, and a bus 27 connecting these. The worker node 200A can be realized by a general information processing device such as a PC or a server computer.
[0043] The processor 21 reads data and programs stored in the secondary storage device 23 into the primary storage device 22 and executes processing determined by the programs. The transfer pod 210A described above with reference to FIG. 1 is a transfer pod image 210Aa stored in the secondary storage device 23 that is deployed (placed) in the primary storage device 22. Similarly, the router pod 220A and the worker pods 230A1 to 230An are a router pod image 220Aa and the worker pod images 230A1a to 230Ana stored in the secondary storage device 23 that are deployed (placed) in the primary storage device 22. The deployment of these pods (the transfer pod 210A, the router pod 220A, and the worker pods 230A1 to 230An) is executed by instructions from the control plane 110A of the master node 100A.
[0044] Also, the location where the transfer pod image 210Aa, the router pod image 220Aa, and the worker pod images 230A1a to 230Ana are stored is the worker node 200A. The location where the transfer pod image 210Aa, the router pod image 220Aa, and the worker pod images 230A1a to 230Ana are stored may be any location where the control plane 110A can read them.
[0045] The main memory device 22 includes a volatile memory element such as a RAM, and stores the programs executed by the processor 21 and data.
[0046] The secondary storage device 23 is a device having a non-volatile storage element such as a hard disk drive (HDD) or a solid state drive (SSD) and stores programs, data, etc. The secondary storage device 23 stores a transfer pod image 210Aa, a router pod image 220Aa, worker pod images 230A1a to 230Ana, etc.
[0047] The input device 24 is a device that accepts user operations such as a keyboard or a mouse, and acquires information input by the user's operation. The output device 25 is a device that outputs information such as a display, and presents information to the user by displaying it on a screen, for example. The worker node 200A may be equipped with a touch panel that serves as both the input device 24 and the output device 25.
[0048] The network I / F 26 is an interface (transmission / reception device) capable of transmitting and receiving data via the network NW with devices such as the master node 100A, the load balancers 300A and 300B, the alternative cluster 1B, the client terminal 500, and the DNS server 600. The worker node 200A can use the network I / F 26 to transmit and receive data with devices such as the master node 100A, the load balancers 300A and 300B, the alternative cluster 1B, the client terminal 500, and the DNS server 600 that are connected to the network NW.
[0049] The master node 100A, the master node 100B and worker node 200B of the alternative cluster 1B, the load balancers 300A and 300B, the client terminal 500, and the DNS server 600, like the worker node 200A, can be realized by general information processing devices such as a PC or a server computer.
[0050] <Configuration of the transport pod 210, Fig. 3> 3 is a block diagram showing an outline of the configuration of the transfer pod 210 A. As shown in FIG. 3, the transfer pod 210 A includes a reception API unit 211, a queue unit 212, a transfer API unit 213, a proxy unit 214, and a monitor unit 215.
[0051] The reception API unit 211 receives a request to the worker pod 230A that is sent toward the transfer pod 210A. The reception API unit 211 waits to receive a request to the worker pod 230A. When the reception API unit 211 receives a request to the worker pod 230A, the reception API unit 211 stores the received request to the worker pod 230A in a request queue of the queue unit 212.
[0052] The queue unit 212 has a request queue that stores requests to the worker pod 230A. The queue unit 212 then transmits the requests to the worker pod 230A stored in the request queue to the transfer API unit 213 in response to an inquiry from the transfer API unit 213.
[0053] 4 is a diagram showing an example of data stored in the request queue. As shown in FIG. 4, the information stored in the request queue includes a reception time 401 at which a request to the worker pod 230A was received, an address 402 of a source host that transmitted the request to the worker pod 230A, a port number 403 of a transfer source that transferred the request to the worker pod 230A to the transfer pod 210A, a destination host 404 that is a domain name of the transfer destination worker pod 230A, a destination port 405 that is a port number of the transfer destination worker pod 230A, a request method 406 that is an instruction to the worker pod 230A included in the request to the worker pod 230A, and a request URL 407 related to the domain name of the worker pod 230A.
[0054] 3, the transfer API unit 213 inquires whether or not an unprocessed request to the worker pod 230A is stored in the request queue of the queue unit 212. If an unprocessed request to the worker pod 230A is stored in the request queue, the transfer API unit 213 obtains the unprocessed request to the worker pod 230A from the request queue and transmits it to the proxy unit 214.
[0055] The proxy unit 214 receives a request to the worker pod 230A from the transfer API unit 213. In addition, the proxy unit 214 transfers the received request to the worker pod 230A to a destination of the request to the worker pod 230A calculated based on the transfer destination information transmitted from the monitoring unit 215.
[0056] The monitoring unit 215 monitors the queue unit 212, the worker pod 230A, and the load balancer 300A. That is, the monitoring unit 215 acquires from the queue unit 212 the amount of data of unprocessed requests to the worker pod 230A that are accumulated in the request queue of the queue unit 212. The monitoring unit 215 also acquires from the worker pod 230A the frequency of errors that occur when executing a program held by the worker pod 230A. Then, the monitoring unit 215 acquires from the load balancer 300A the amount of data of requests to the worker pod 230 (requests to the target program) sent from the client terminal 500 to the load balancer 300A (cluster 1A).
[0057] Furthermore, when the monitoring unit 215 receives a message that the forwarding destination of a request to the unprocessed worker pod 230A is "load balancer 300", it transmits forwarding destination information indicating that the forwarding destination of a request to the unprocessed worker pod 230A will be "load balancer 300" to the proxy unit 214. Similarly, when the monitoring unit 215 receives a message that the forwarding destination of a request to the unprocessed worker pod 230A is "router pod 220A", it transmits forwarding destination information indicating that the forwarding destination of a request to the unprocessed worker pod 230A will be "router pod 220A" to the proxy unit 214.
[0058] <<Processing Procedure>> Next, (A) the procedure for switching cluster 1A to alternative cluster 1B (see FIGS. 6 to 11), and (B) the procedure for switching worker pod 230Ax of cluster 1A to worker pod 230B of alternative cluster 1B (see FIGS. 12 to 16) will be described.
[0059] 5 is an explanatory diagram illustrating a configuration in a state (before switching) in which the cluster 1A is used and the alternative cluster 1B is not used. As shown in FIG. 5, in the state before switching, the transfer pods 210 (transfer pods 210A and 210B) have not been created in the cluster 1A and the alternative cluster 1B.
[0060] <(A) Procedure for switching cluster 1A to alternative cluster 1B, Figures 6 to 11> (A) Cluster 1A may be switched to alternative cluster 1B in the following cases. For example, in order to update the container infrastructure of cluster 1A, a system with the updated container infrastructure of cluster 1A may be built in alternative cluster 1B. Then, as described below, alternative cluster 1B is used on a trial basis in place of cluster 1A, and if there are no problems with alternative cluster 1B, alternative cluster 1B is used instead of cluster 1A.
[0061] In preparation for switching cluster 1A to alternative cluster 1B, after completing the settings and configuration changes for alternative cluster 1B, you can switch cluster 1A to alternative cluster 1B by following the procedure (A) for switching cluster 1A to alternative cluster 1B described below. FIG. 6 is a flowchart showing an example of a procedure for switching cluster 1A to alternative cluster 1B.
[0062] First, a transfer pod 210A is deployed in the cluster 1A, and a transfer pod 210B is deployed in the alternative cluster 1B (step S101). That is, in the cluster 1A, the control plane 110A of the master node 100A deploys the transfer pod 210A to each of the worker nodes 200A using a transfer pod image 210Aa (see FIG. 2). Also, in the alternative cluster 1B, the control plane 110B of the master node 100B deploys the transfer pod 210B to each of the worker nodes 200B using a transfer pod image 210Ba (not shown).
[0063] 7 is an explanatory diagram showing a state in which the transfer pods 210A and 210B are deployed in step S101. Here, the request to the worker pod 230A sent by the client terminal 500 is received by the load balancer 300A. Then, the load balancer 300A sends the request to the worker pod 230A to the router pod 220A of the worker node 200A. Here, the transfer pod 210A does not receive the request to the worker pod 230A. The router pod 220A transfers the request to the worker pod 230A to the worker pod 230A. The worker pod 230A executes processing according to the request to the worker pod 230A.
[0064] Next, the load balancer 300A and the forwarding pod 210A are configured so that the request sent by the client terminal 500 to the worker pod 230A is received by the router pod 220A via the load balancer 300A and the forwarding pod 210A, and the load balancer 300B and the forwarding pod 210B of the alternative cluster 1B are similarly configured (step S102).
[0065] FIG. 8 is an explanatory diagram showing a state in which the load balancers 300A and 300B and the router pods 220A and 220B are set in step S102. In step S102, in the cluster 1A, the control plane 110A of the master node 100A sets the forwarding pod 210A so that when the forwarding pod 210A receives a request for the worker pod 230A, the forwarding pod 210A transmits the received request for the worker pod 230A to the router pod 220A. At this point, the request for the worker pod 230A transmitted by the client terminal 500 is transmitted to the worker pod 230A via the load balancer 300A and the router pod 220A. Therefore, at this point, the request for the worker pod 230A is not transmitted to the forwarding pod 210A but is transmitted to the worker pod 230A, so that the worker pod 230A can execute processing according to the request for the worker pod 230A.
[0066] Furthermore, the control plane 110A of the master node 100A configures the load balancer 300A so that when the load balancer 300A receives a request for the worker pod 230A, the load balancer 300A transmits the received request for the worker pod 230A to the transfer pod 210A. As a result, the request for the worker pod 230A transmitted by the client terminal 500 is transmitted to the worker pod 230A via the load balancer 300A, the transfer pod 210A, and the router pod 220A.
[0067] Also, in the alternative cluster 1B, the control plane 110B of the master node 100B sets the forwarding pod 210B and the load balancer 300B in the same manner as described above. That is, the control plane 110B of the master node 100B sets the forwarding pod 210B so that when the forwarding pod 210B receives a request for the worker pod 230A, the forwarding pod 210B transmits the received request for the worker pod 230A to the router pod 220B. Also, the master node 100B sets the load balancer 300B so that when the load balancer 300B receives a request for the worker pod 230A, the load balancer 300B transmits the received request for the worker pod 230A to the forwarding pod 210B.
[0068] Next, in the cluster 1A, the master node 100A sets the transfer pod 210A so that the transfer pod 210A transfers a request to the worker pod 230A to the load balancer 300B (step S103).
[0069] 9 is an explanatory diagram showing a state in which the transfer pod 210A is set in step S103. As shown in Fig. 9, a request to the worker pod 230A sent by the client terminal 500 is transferred to the worker pod 230B of the alternative cluster 1B via the load balancer 300A, the transfer pod 210A and the load balancer 300B, the transfer pod 210B, and the router pod 220B. Then, the worker pod 230B executes a process according to the request to the worker pod 230A.
[0070] Next, in each worker node 200B of the alternative cluster 1B, the transfer pod 210B determines whether or not there is a problem with the execution of the alternative program of the worker pod 230 (step S104). The alternative program is a program of the worker pod 230B that is executed in response to a request to the worker pod 230A. If it is determined that there is a problem with the execution of the alternative program (step S104: Yes), the process proceeds to step S105, and in the following step S106, the use of the alternative cluster 1B is stopped. On the other hand, if it is determined that there is no problem with the execution of the alternative program (step S104: No), the process proceeds to step S107.
[0071] If at least one of the following two conditions is met, the transfer pod 210B determines that there is a problem with the execution of the alternative program possessed by the worker pod 230 (step S204: Yes). If neither of the following two conditions is met, the transfer pod 210B determines that there is no problem with the execution of the alternative program possessed by the worker pod 230 (step S104: No). (Condition 1) The frequency of errors occurring when executing a program (alternative program) possessed by the worker pod 230B is higher than a predetermined upper limit of error frequency. In this case, there is a problem with the worker pod 230B of the alternative cluster 1B. As described above with reference to FIG. 3, the monitoring unit 215 of each transfer pod 210B acquires the frequency of errors of the worker pods 230B1 to 230Bn (all worker pods 230B) of the worker node 200B in which the transfer pod 210B exists. Here, the frequency of errors of each of the worker pods 230B1 to 230Bn (all worker pods 230B) is the frequency of errors of the alternative program possessed by each of the worker pods 230B1 to 230Bn (all worker pods 230B). Furthermore, the monitoring unit 215 determines that there is a problem with the execution of the alternative program when the acquired error frequency (frequency of errors in the alternative program) is higher than a predetermined upper limit of error frequency. Then, when at least one of the monitoring units 15 of the worker pods 230B1 to 230Bn determines that there is a problem with the execution of the alternative program, it determines that there is a problem with the execution of the alternative program. (Condition 2) When the transfer speed of the request (request to the target program) to the worker pod 230A transferred from the cluster 1A to the alternative cluster 1B is greater than a predetermined upper limit of the data transfer speed. As described above with reference to FIG. 3, the monitoring unit 215 of the transfer pod 210B regards the data amount of the request to the worker pod 230A stored in the request queue of the queue unit 212 as the transfer speed of the request (request to the target program) to the worker pod 230A transferred from the cluster 1A to the alternative cluster 1B. When the data amount of the request to the worker pod 230A stored in the queue unit 212 is greater than a predetermined upper limit of the data transfer speed, the monitoring unit 215 can consider that the worker pods 230B1 to 230Bn (all the worker pods 230B) are not able to complete the processing according to the request to the worker pod 230A, and therefore the monitoring unit 215 determines that there is a problem with the execution of the alternative program. Here, among the monitoring units 15 of the worker pods 230B1 to 230Bn, all of the monitoring units 15 determine whether or not there is a problem with the execution of the alternative program at a predetermined time interval. Then, if at least one of the monitoring units 15 of the worker pods 230B1 to 230Bn determines that there is a problem with the execution of the alternative program during the predetermined time interval, it is determined that "there is a problem with the execution of the alternative program."
[0072] Next, the proxy unit 214 of the transfer pod 210B of the alternative cluster 1B transmits transfer stop information including information to the effect that transfer is stopped to the transfer pod 210A of the cluster 1A (step S105). The process of step S105 is executed when the transfer pod 210B determines in step S104 that there is a problem with the execution of the alternative program possessed by the worker pod 230 (step S104: Yes). In step S105, the proxy unit 214 of the transfer pod 210B of the alternative cluster 1B transmits transfer stop information including information to the effect that transfer is stopped to the control plane 110A of the master node 100A of the cluster 1A. When the control plane 110A receives the transfer stop information, it transmits the transfer stop information to the transfer pod 210A of each worker node 200A.
[0073] Next, when the transfer pod 210A of each worker node 200 of cluster 1A receives the transfer stop information from the control plane 110A, the transfer pod 210A sets the request for the worker pod 230A received by the transfer pod 210A to be transferred to the router pod 220A, stops transferring the request for the worker pod 230A received by the transfer pod 210A to the load balancer 300B (alternative cluster 1B), and ends the process (step S106). As a result, as described above with reference to FIG. 8, the request for the worker pod 230A is sent to the worker pod 230A via the load balancer 300A, the transfer pod 210A, and the router pod 220A.
[0074] If the transfer pod 210B of the worker node 200B determines that there is a problem with the execution of the alternative program through the processes in steps S104 to S106 described above (step S104: Yes), it stops transferring the request to the worker pod 230A from cluster 1A to the alternative cluster 1B. Then, the worker pods 230A1 to 230An (all worker pods 230A) of the worker node 200 in cluster 1A execute processing according to the request to the worker pod 230A.
[0075] Next, the DNS record 601 of the DNS server 600 is changed so as to associate the domain name of cluster 1A with the IP address of the alternative cluster 1B (step S107). That is, at least one of the forwarding pods 210B of the alternative cluster 1B transmits to the DNS server 600 DNS record update information including information to rewrite the DNS record 601 stored in the DNS server 600 to a DNS record in which the IP address of the alternative cluster 1B is associated with the domain name of the cluster. Upon receiving the DNS record update information, the DNS server 600 changes the DNS record 601 so as to associate the domain name of cluster 1A with the IP address of the alternative cluster 1B.
[0076] By the process of changing the DNS record 601 (a record that stores the domain name of cluster 1A in association with the IP address) in step S107, the destination to which the client terminal 500 sends a request to the worker pod 230A is changed from cluster 1A to the alternative cluster 1B. As a result, the cluster that executes the process for the request to the worker pod 230A of cluster 1A is switched from cluster 1A to the alternative cluster 1B.
[0077] However, since there are many DNS servers as the DNS server 600, it is not easy to immediately change the DNS records 601 of all the DNS servers. Even if the client terminal 500 sends a request (including the domain name of the cluster 1A) to the worker pod 230A until the change in the DNS records 601 of all the DNS servers is completed, there is a risk that the request will access the cluster 1A by referring to the DNS record 601 of the DNS server whose DNS record 601 has not been changed. Therefore, even if the client terminal 500 changes the destination of the request to the worker pod 230A from the cluster 1A to the alternative cluster 1B by executing the process of step S107, the switch from the cluster 1A to the alternative cluster 1B is not completed until the change in the DNS records 601 of all the DNS servers is completed.
[0078] In contrast, after the process of step S103 is executed, the transfer pod 210A transfers all requests to the worker pod 230A that are sent to the cluster 1A to the alternative cluster 1B. Therefore, by executing the process of step S103, the cluster that executes the process for the request to the worker pod 230A can be immediately switched from the cluster 1A to the alternative cluster 1B.
[0079] In step S103, the cluster that executes the process for the request to the worker pod 230A is immediately switched from cluster 1A to alternative cluster 1B, and then, in step S104, if it is determined that there is no problem with the execution of the alternative program of the worker pod 230 (step S104: No), the process of step S107 is executed. In the process of step S107, the destination to which the client terminal 500 sends the request to the worker pod 230A is changed from cluster 1A to alternative cluster 1B. Therefore, after the process of step S107, when a necessary time (for example, several tens of minutes) has elapsed, the transfer pod 210A no longer needs to transfer the request to the worker pod 230A.
[0080] Furthermore, when it is determined that there is no problem with the execution of the alternative program of the worker pod 230 (step S104: No), instead of changing the DNS record 601 of the DNS server 600, in steps S105 and S106, the worker pod 230A stops forwarding requests to the worker pod 230A, and the cluster that executes the process for the request to the worker pod 230A is switched from the alternative cluster 1B to the cluster 1A by using the worker pod 230A (steps S105 and S106). Here, the switching from the alternative cluster 1B to the cluster 1A is completed relatively quickly by changing the settings of the transfer pod 210A and the transfer pod 210B.
[0081] 10 is an explanatory diagram showing a state in which the DNS server 600 is set in step S107. As shown in FIG 10, the DNS record 601 of the DNS server 600 stores information that associates the domain name of the cluster 1A with the IP address of the alternative cluster 1B.
[0082] Next, the cluster 1A determines whether or not it has received a request for the worker pod 230 (step S108). If it is determined that the cluster 1A has received a request for the worker pod 230 (step S108: Yes), the process proceeds to step S109. On the other hand, if it is determined that the cluster 1A has not received a request for the worker pod 230 (step S108: No), the process proceeds to step S110, where the cluster 1A deletes the transfer pods 210A and 210B.
[0083] Here, the monitoring unit 215 (see FIG. 3) of at least one of the worker pods 230A1-230An (all the worker pods 230A) of the cluster 1A monitors the load balancer 300A and acquires the data amount of the request to the target program transmitted from the client terminal 500 to the load balancer 300A (cluster 1A) during a predetermined time interval. Then, when the acquired data amount of the request to the target program transmitted from the client terminal 500 to the load balancer 300A (cluster 1A) is smaller than a predetermined data transmission amount lower limit, the monitoring unit 215 determines that the cluster 1A has not received the request to the worker pod 230. On the other hand, when the acquired data amount of the request to the target program transmitted from the client terminal 500 to the cluster 1A is equal to or larger than a predetermined data transmission amount lower limit, the monitoring unit 215 determines that the cluster 1A has received the request to the worker pod 230.
[0084] Next, the monitoring unit 215 of the transfer pod 210A of the cluster 1A waits for a predetermined time and executes the process of step S108 (step S109). Here, by repeating the processes of steps S108 and S109, the cluster 1A determines whether or not it has received a request to the worker pod 230 in step S108 at predetermined time intervals until it is determined that the cluster 1A has not received a request to the worker pod 230 (step S108: No).
[0085] Next, the settings of the load balancers 300A and 300B are restored, the forwarding pod 210A of cluster 1A and the forwarding pod 210B of the alternative cluster 1B are deleted, and the process ends (step S110). That is, the master node 100A of cluster 1A sets the load balancer 300A so that, when the load balancer 300A receives a request for the worker pod 230A, the load balancer 300A transmits the received request for the worker pod 230A to the router pod 220A. Similarly, the control plane 110B of the master node 100B of the alternative cluster 1B sets the load balancer 300B.
[0086] 11 is an explanatory diagram showing a state in which the load balancers 300A and 300B are set and the transfer pods 210A and 210B are deleted in step S110. As shown in FIG 11, a request sent by the client terminal 500 to the worker pod 230A is sent to the worker pod 230B via the load balancer 300B and the router pod 220B.
[0087] By the processing of steps S108 and S110 described above, the processor of the worker node 200 obtains the amount of data of the request for the target program sent from the client terminal 500 to cluster 1A during a predetermined time interval, and deletes the transfer pod if the obtained amount of data of the request for the target program sent from the client terminal 500 to cluster 1A is smaller than a predetermined lower limit of the data transmission amount.
[0088] 9 and 10, the transfer pod 210A executes the following request transfer process by the process of step S103. That is, in the request transfer process, when a program (alternative program) of at least one worker pod 230B stored in the alternative cluster 1B is executed in place of a program (target program) of at least one worker pod 230A stored in the cluster 1A, upon receiving a request (request to the target program) to the worker pod 230A transmitted from the client terminal 500, the request (request to the target program) to the worker pod 230A is transferred to the alternative cluster 1B so that the program (alternative program) of the worker pod 230B is executed in response to the request (request to the target program) to the worker pod 230A.
[0089] The process of step S104 includes the following alternative program determination process. That is, in the alternative program determination process, in step S104, the transfer pod 210B executes a process of determining whether or not there is a problem in the execution of the program (alternative program) of the worker pod 230B of the alternative cluster 1B.
[0090] The process of step S104 includes the following first alternative program problem detection process: In the first alternative program problem detection process, in step S104, if the frequency of errors in the worker pod 230B (the error frequency of the alternative program in the worker pod 230B) is higher than a predetermined upper error frequency limit, it is determined that there is a problem with the execution of the alternative program in the worker pod 230B.
[0091] The process of step S104 also includes the following second alternative program problem detection process: In the second alternative program problem detection process, in step S104, if the transfer speed of a request (request to the target program) to the worker pod 230A transferred from the cluster 1A to the alternative cluster 1B is greater than a predetermined upper limit of the data transfer speed, it is determined that there is a problem with the execution of the alternative program of the worker pod 230B.
[0092] The process of step S107 includes the following DNS record change process: In the DNS record change process, in step S107, the forwarding pod 210B transmits to the DNS server 600 DNS record update information including information to rewrite the DNS record 601 stored in the forwarding pod 210 DNS server 600 to the DNS record 601 that associates the IP address of the alternative cluster 1B with the domain name of the cluster 1A.
[0093] The process of step S105 includes the following transfer abort request process: In other words, in the transfer abort request process, if it is determined that there is a problem with the execution of the program (alternative program) of the worker pod 230B of the alternative cluster 1B (step S104: Yes in the flowchart of FIG. 6), transfer abort information including information to the effect that the transfer is to be stopped is transmitted to the transfer pod 210A of the cluster 1A.
[0094] The process of step S106 includes the following transfer abort process. That is, in the transfer abort process, when the transfer pod 210A (processor 21) of the cluster 1A receives the transfer abort information, it stops transferring the request to the worker pod 230A (executing the request transfer process to transfer the request to the target program to the alternative cluster).
[0095] The processes of steps S108 and S110 include the following transfer pod deletion process. That is, in the transfer pod deletion process, in steps S108 and S110, the data amount of the request (request to the target program) to the worker pod 230A sent from the client terminal 500 to the cluster 1A during a predetermined time interval is acquired (step S108). Then, if the acquired data amount of the request (request to the target program) to the worker pod 230A sent from the client terminal 500 to the cluster 1A is smaller than a predetermined data transmission amount lower limit, the transfer pod is deleted (step S110).
[0096] <(B) Procedure for switching worker pod 230Ax of cluster 1A to worker pod 230B of alternative cluster 1B, Figures 12 to 16> In the procedure for switching (A) cluster 1A to alternative cluster 1B described above with reference to Figs. 6 to 11, the switching is performed on a cluster-by-cluster basis. In the case of switching worker pod 230Ax of cluster 1A to worker pod 230B of alternative cluster 1B described below, the switching is performed on a worker pod 230 (program)-by-program basis. In the following description and drawings of this case, the worker pod 230Ax of the worker pods 230 of cluster 1A is switched to worker pod 230Bx of alternative cluster 1B. That is, worker pod 230Ax is the worker pod 230 from which the switching is to be performed. Also, worker pod 230Bx is the worker pod 230 to which the switching is to be performed.
[0097] (B) Complete the configuration of alternative cluster 1B and the deployment of worker pod 230Bx before performing steps to switch worker pod 230Ax of cluster 1A to worker pod 230B of alternative cluster 1B.
[0098] FIG. 12 is an explanatory diagram illustrating the configuration of the state (before switching) before executing the procedure to switch the worker pod 230Ax of the cluster 1A to the worker pod 230B of the alternative cluster 1B. As shown in FIG. 12, in the state before switching, the transfer pods 210 (transfer pods 210A and 210B) have not been created in the cluster 1A and the alternative cluster 1B. FIG. 13 is a flowchart showing an example of a procedure for switching a worker pod 230Ax in cluster 1A to a worker pod 230B in an alternative cluster 1B.
[0099] First, a transfer pod 210A is deployed in the cluster 1A, and a transfer pod 210B is deployed in the alternative cluster 1B (step S201). That is, in the cluster 1A, the control plane 110A of the master node 100A deploys the transfer pod 210A in each of the worker nodes 200A using the transfer pod image 210Aa. In addition, in the alternative cluster 1B, the control plane 110B of the master node 100B deploys the transfer pod 210B in each of the worker nodes 200B using the transfer pod image 210Ba.
[0100] 14 is an explanatory diagram showing a state in which the transfer pods 210A and 210B are deployed in step S201. Here, the request to the worker pod 230Ax sent by the client terminal 500 is received by the load balancer 300A. Then, the load balancer 300A sends the request to the worker pod 230Ax to the router pod 220A of the worker node 200A. The router pod 220A transfers the request to the worker pod 230Ax to the worker pod 230Ax. The worker pod 230Ax executes processing according to the request to the worker pod 230Ax.
[0101] Next, the load balancer 300Ax and the forwarding pod 210Ax are configured so that the request sent by the client terminal 500 to the worker pod 230Ax is received by the router pod 220Ax via the load balancer 300Ax and the forwarding pod 210Ax, and the load balancer 300Bx and the forwarding pod 210Bx of the alternative cluster 1B are also configured in the same way (step S202). FIG. 15 is an explanatory diagram showing a state in which the load balancers 300A and 300B and the router pods 220A and 220B are set in step S202. In step S202, in the cluster 1A, the control plane 110A of the master node 100A sets the transfer pod 210A so that when the transfer pod 210A receives a request for the worker pod 230Ax, the transfer pod 210A transmits the received request for the worker pod 230Ax to the router pod 220A. At this point, the request for the worker pod 230Ax transmitted by the client terminal 500 is transmitted to the worker pod 230Ax via the load balancer 300A and the router pod 220A. Therefore, at this point, the request for the worker pod 230Ax is not transmitted to the transfer pod 210A but is transmitted to the worker pod 230Ax, so that the worker pod 230Ax can execute processing according to the request for the worker pod 230A.
[0102] Furthermore, the control plane 110A of the master node 100A configures the load balancer 300A so that, when the load balancer 300A receives a request for the worker pod 230Ax, the load balancer 300A transmits the received request for the worker pod 230Ax to the transfer pod 210A. As a result of the above, the request for the worker pod 230Ax transmitted by the client terminal 500 is transmitted to the worker pod 230Ax via the load balancer 300A, the transfer pod 210A, and the router pod 220A.
[0103] Also, in the alternative cluster 1B, the control plane 110B of the master node 100B sets the forwarding pod 210B and the load balancer 300B in the same manner as described above. That is, the control plane 110B of the master node 100B sets the forwarding pod 210B so that when the forwarding pod 210B receives a request for the worker pod 230Ax, the forwarding pod 210B transmits the received request for the worker pod 230Ax to the router pod 220B. Also, the master node 100B sets the load balancer 300B so that when the load balancer 300B receives a request for the worker pod 230Ax, the load balancer 300B transmits the received request for the worker pod 230Ax to the forwarding pod 210B.
[0104] Next, in the cluster 1A, the master node 100A sets the transfer pod 210A so that the transfer pod 210A transfers a request to the worker pod 230Ax to the load balancer 300B (step S203).
[0105] Fig. 16 is an explanatory diagram showing a state in which the transfer pod 210A is set in step S203. As shown in Fig. 16, a request to the worker pod 230Ax sent by the client terminal 500 is transferred to the worker pod 230Bx of the alternative cluster 1B via the load balancer 300A, the transfer pod 210A and the load balancer 300B, the transfer pod 210B, and the router pod 220B. Then, the worker pod 230Bx executes a process according to the request to the worker pod 230Ax.
[0106] Next, in each worker node 200B of the alternative cluster 1B, the transfer pod 210B judges whether or not there is a problem with the execution of the alternative program held by the worker pod 230Bx (step S204). The alternative program is a program of the worker pod 230Bx that is executed in response to a request to the worker pod 230Ax. If it is judged that there is a problem with the execution of the alternative program (step S204: Yes), the process proceeds to step S205, where the use of the alternative cluster 1B is stopped. On the other hand, if it is judged that there is no problem with the execution of the alternative program (step S205: No), the process ends.
[0107] The transfer pod 210B determines that there is a problem with the execution of the alternative program possessed by the worker pod 230Bx when at least one of the following two conditions is met (step S204: Yes). Also, the transfer pod 210B determines that there is no problem with the execution of the alternative program possessed by the worker pod 230 when neither of the following two conditions is met (step S104: No). (Condition 1) When the frequency of errors occurring when executing a program (alternative program) held by the worker pod 230Bx is higher than a predetermined error frequency upper limit value. In this case, there is a problem with the worker pod 230Bx of the alternative cluster 1B. As described above with reference to FIG. 3, the monitoring unit 215 of each transfer pod 210B acquires the frequency of errors of the worker pod 230Bx of the worker node 200B in which the monitoring unit 215 exists. Here, the frequency of errors of the worker pod 230Bx is the error frequency of the alternative program held by the worker pod 230Bx. Furthermore, when the acquired error frequency (frequency of errors in the alternative program) is higher than a predetermined error frequency upper limit value, the monitoring unit 215 determines that there is a problem with the execution of the alternative program. Then, when at least one monitoring unit 15 of the worker pod 230Bx determines that there is a problem with the execution of the alternative program, it determines that "there is a problem with the execution of the alternative program." (Condition 2) When the transfer speed of the request (request to the target program) to the worker pod 230Ax transferred from the cluster 1A to the alternative cluster 1B is greater than a predetermined upper limit of data transfer speed. As described above with reference to FIG. 3, the monitoring unit 215 of the transfer pod 210B regards the data amount of the request to the worker pod 230Ax stored in the request queue of the queue unit 212 as the transfer speed of the request (request to the target program) to the worker pod 230Ax transferred from the cluster 1A to the alternative cluster 1B. When the data amount of the request to the worker pod 230Ax stored in the queue unit 212 is greater than a predetermined upper limit of data transfer speed, the monitoring unit 215 can consider that the worker pod 230Bx has not been able to complete the processing according to the request to the worker pod 230A, and therefore the monitoring unit 215 determines that there is a problem with the execution of the alternative program. Here, all of the monitoring units 15 of the worker pod 230Bx determine whether there is a problem with the execution of the alternative program at a predetermined time interval. Then, if at least one of the monitoring units 15 of the worker pod 230Bx determines that there is a problem with the execution of the alternative program within a specified time interval, it is determined that there is a problem with the execution of the alternative program.
[0108] Next, the proxy unit 214 of the transfer pod 210B of the alternative cluster 1B transmits transfer suspension information including information to the effect that transfer is to be stopped to the transfer pod 210A of the cluster 1A (step S205). In step S205, the proxy unit 214 of the transfer pod 210B of the alternative cluster 1B transmits transfer suspension information including information to the effect that transfer is to be stopped to the control plane 110A of the master node 100A of the cluster 1A. Upon receiving the transfer suspension information, the control plane 110A transmits the transfer suspension information to the transfer pod 210A of each worker node 200A.
[0109] Next, when the transfer pod 210A of each worker node 200 of cluster 1A receives the transfer stop information from the control plane 110A, the transfer pod 210A sets the request for the worker pod 230Ax received by the transfer pod 210A to be transferred to the router pod 220A, the transfer pod 210A stops transferring the request for the worker pod 230Ax received by the transfer pod 210A to the load balancer 300B (alternative cluster 1B), and ends the process (step S206). As a result, as described above with reference to FIG. 8, the request for the worker pod 230Ax is sent to the worker pod 230Ax via the load balancer 300A, the transfer pod 210A, and the router pod 220A.
[0110] Furthermore, if the transfer pod 210B of the worker node 200B determines that there is a problem with the execution of the alternative program through the processes in steps S204 to S206 described above (step S204: Yes), the transfer of the request to the worker pod 230Ax from the cluster 1A to the alternative cluster 1B is stopped. Then, the worker pod 230Ax of the worker node 200 in the cluster 1A executes processing according to the request to the worker pod 230Ax.
[0111] After executing the process of step S206, the load balancer 300A may be configured to forward requests to the worker pod 230Ax to the router pod 220A, and further, the forwarding pods 210A and 210B may be deleted.
[0112] 16, the transfer pod 210A executes the following request transfer process by the process of step S203. That is, in the request transfer process, when a program (alternative program) of at least one worker pod 230Bx stored in the alternative cluster 1B is executed in place of a program (target program) of at least one worker pod 230Ax stored in the cluster 1A, upon receiving a request (request to the target program) to the worker pod 230Ax transmitted from the client terminal 500, the request (request to the target program) to the worker pod 230Ax is transferred to the alternative cluster 1B so that the program (alternative program) of the worker pod 230Bx is executed in response to the request (request to the target program) to the worker pod 230Ax.
[0113] The process of step S204 includes the following alternative program determination process. That is, in the alternative program determination process, in step S204, the transfer pod 210B executes a process of determining whether or not there is a problem in the execution of the program (alternative program) of the worker pod 230Bx of the alternative cluster 1B.
[0114] The process of step S204 includes the following first alternative program problem detection process: In the first alternative program problem detection process, in step S204, if the frequency of errors in the worker pod 230Bx (the frequency of errors in the alternative program of the worker pod 230Bx) is higher than a predetermined upper error frequency limit, it is determined that there is a problem in the execution of the alternative program of the worker pod 230Bx.
[0115] The process of step S204 also includes the following second alternative program problem detection process: In the second alternative program problem detection process, in step S204, if the transfer speed of a request (request to a target program) to the worker pod 230Ax transferred from the cluster 1A to the alternative cluster 1B is greater than a predetermined upper limit of the data transfer speed, it is determined that there is a problem with the execution of the alternative program of the worker pod 230Bx.
[0116] The process of step S205 includes the following transfer abort request process: In other words, in the transfer abort request process, when it is determined that there is a problem with the execution of the program (alternative program) of the worker pod 230Bx of the alternative cluster 1B (step S204: Yes in the flowchart of FIG. 13), transfer abort information including information to the effect that the transfer is to be stopped is transmitted to the transfer pod 210A of the cluster 1A.
[0117] The process of step S206 includes the following transfer abort process. That is, in the transfer abort process, when the transfer pod 210A (processor 21) of the cluster 1A receives the transfer abort information, it stops transferring the request to the worker pod 230Ax (executing the request transfer process to transfer the request to the target program to the alternative cluster). <Effects of the Invention> Thus, in the embodiment, (A) the procedure of switching cluster 1A to alternative cluster 1B (see FIG. 6) and (B) the procedure of switching worker pod 230Ax of cluster 1A to worker pod 230B of alternative cluster 1B (see FIG. 13) are the cluster management method of the present invention.
[0118] The cluster management method is to transfer a request (request for the target program) to worker pod 230A to the alternative cluster 1B to execute the program (alternative program) of worker pod 230B in the alternative cluster 1B in response to the request (request for the target program) to worker pod 230A (see step S103 of the flowchart in Figure 6 and step S203 of the flowchart in Figure 13).
[0119] Here, when a client terminal transmits a request to worker pod 230A (a request to a target program) to cluster 1A, the request to worker pod 230A (a request to a target program) is forwarded to alternative cluster 1B. This ensures that the program (alternative program) of worker pod 230B stored in alternative cluster 1B can be used in place of the program of worker pod 230A stored in cluster 1A. Therefore, the cluster management method of the present invention enables faster cluster switching.
[0120] Note that the switching from the program of cluster 1A to the alternative program of alternative cluster 1B here may be a cluster switching in which all programs in cluster 1A are switched to all programs in alternative cluster 1B.
[0121] Furthermore, if a defect such as a security hole is found in cluster 1A before the switch, cluster 1A can be quickly and reliably switched to alternative cluster 1B. As a result, the cluster management method of the present invention can increase safety, such as reducing security risks when using a cluster. Furthermore, if a defect is found in alternative cluster 1B after switching to alternative cluster 1B, the switch can be quickly stopped (the cluster to be used can be quickly switched from alternative cluster 1B to cluster 1A). As a result, damage caused by a defect after the switch can be suppressed.
[0122] Furthermore, in the procedure (A) of switching cluster 1A to alternative cluster 1B (see FIG. 6) of the cluster management method of the present invention, all requests to the programs of the worker pod 230A (requests to the programs of the worker pods) are transferred to the alternative cluster as requests to the target program. In other words, in this procedure (A) of switching cluster 1A to alternative cluster 1B (see FIG. 6), cluster switching is performed. The cluster management method of the present invention can perform cluster switching more quickly even when cluster switching is performed.
[0123] Furthermore, in the cluster management method of the present invention, DNS record update information including information to rewrite the DNS record 601 stored in the DNS server 600 to a DNS record in which the IP address of the alternative cluster 1B (the IP address of the load balancer 300B) is associated with the domain name of the cluster 1A is transmitted to the DNS server 600. As a result, when the DNS server 600 receives a name resolution inquiry for the domain name of the cluster 1A, the DNS server 600 returns the IP address of the alternative cluster 1B for the domain name of the cluster 1A. This makes it possible to more reliably switch from the worker pod 230A (program) of the cluster 1A to the worker pod 230B (alternative program) of the alternative cluster 1B.
[0124] In addition, in the cluster management method of the present invention, when it is determined that there is a problem with the execution of the program (alternative program) of the worker pod 230B of the alternative cluster 1B (step S104: Yes in the flowchart of FIG. 6), transfer stop information including information to the effect that transfer is stopped is sent to the transfer pod 210A of the cluster 1A (step S105 in the flowchart of FIG. 6). Then, when the transfer pod 210A (processor 21) of the cluster 1A receives the transfer stop information, it stops the transfer of the request to the worker pod 230A (execution of the request transfer process to transfer the request to the target program to the alternative cluster) (step S106 in the flowchart of FIG. 6). Therefore, in the event of a problem occurring in the alternative cluster 1B, the cluster 1A is used. The adverse effects caused by the problem occurring in the alternative cluster 1B can be suppressed. And the cluster switching can be performed more quickly.
[0125] Furthermore, in the cluster management method of the present invention, when the amount of data of a request (request to a target program) to the worker pod 230A transmitted from the client terminal 500 to the cluster 1A is smaller than a predetermined lower limit of the amount of data transmitted, the transfer pods 210A and 210B are deleted (steps S108 and S110 in the flowchart of FIG. 6). This makes it possible to suppress the deletion of the transfer pods 210A and 210B when they are needed. And, when the transfer pods 210A and 210B are definitely not needed, the transfer pods 210A and 210B can be deleted.
[0126] In addition, in the cluster management method, the transfer pods 210A and 210B (transfer pods) transfer the request to the worker pod 230A (request to the target program) to the alternative cluster 1B so that the program (alternative program) of the worker pod 230B of the alternative cluster 1B is executed in response to the request to the worker pod 230A (request to the target program) (see step S103 of the flowchart in FIG. 6 and step S203 of the flowchart in FIG. 13). Since the transfer pods 210A and 210B (transfer pods) are pods, they can be easily created (deployed) and deleted as needed. When it is not necessary to use the cluster management method of the present invention, the transfer pods 210A and 210B (transfer pods) can be easily deleted. As a result, the cluster management method of the present invention can utilize the resources of the cluster more efficiently.
[0127] Furthermore, the transfer pod image 210Aa (the image of the transfer pods 210A and 210B) can be easily repurposed. Therefore, the cluster management method can be easily applied to a new cluster.
[0128] Furthermore, in the cluster management method of the present invention, if the frequency of errors in worker pod 230B (error frequency of the alternative program in worker pod 230B) is higher than a predetermined upper error frequency limit, it is determined that there is a problem with the execution of the alternative program in worker pod 230B (step S104 in the flowchart in FIG. 6, step S204 in the flowchart in FIG. 13). As a result, problems that occur when using alternative cluster 1B as an alternative can be easily detected. This also makes it easier to suppress problems when switching clusters.
[0129] Furthermore, in the cluster management method of the present invention, if the transfer speed of a request (request to a target program) to worker pod 230A transferred from cluster 1A to alternative cluster 1B is greater than a predetermined upper limit of data transfer speed, it is determined that there is a problem with the execution of the alternative program in worker pod 230B (step S104 in the flowchart of FIG. 6, step S204 in the flowchart of FIG. 13). As a result, it is possible to easily detect a lack of processing power of alternative cluster 1B when alternative cluster 1B is used as an alternative. This also makes it easier to suppress problems when switching clusters.
[0130] The present invention is not limited to the above-mentioned embodiment, but includes various modified examples. For example, the above-mentioned embodiment describes the configuration in detail to easily explain the present invention, and the present invention is not necessarily limited to the configuration including all the described configurations. Also, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.
[0131] In addition, each of the above configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium storing it constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, etc.
[0132] Furthermore, the program code for realizing the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Python, Java (registered trademark), etc.
[0133] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed over a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read out and execute the program code stored in the storage means or storage medium.
[0134] In the above-mentioned embodiment, the control lines and information lines are shown as those considered necessary for the explanation, and not all the control lines and information lines are shown in the product. All the components may be connected to each other. [Explanation of symbols]
[0135] 1000;Cluster system 1A; Cluster 1B; Alternative cluster 21; Processor 22; Main memory 23; Secondary storage device 24. Input device 25; Output device 27; Bus 100A, 100B; Masternodes 110A, 110B; Control plane 200A, 200B: worker nodes 210A, 210B; Transport pods 210Aa;Transporter pod image 211; Receiving API section 212; Queue 213; Transfer API section 214; Proxy Department 215;Monitoring Department 220A, 220B; Router Pod 220Aa; Router pod image 230, 230A, 230A1-230An; Worker Pods 230A1a; Worker pod image 300, 300A, 300B; Load Balancer 500; Client terminal 600; DNS server NW; Network
Claims
1. A cluster management method for a cluster including a plurality of nodes, each having a memory unit for storing a program to be executed in response to a request from a client terminal connected to a network, and a processor for executing the program, comprising the steps of: The processor, When at least one alternative program stored in the alternative cluster is executed in place of at least one target program stored in the cluster, When a request for the target program transmitted from the client terminal is obtained, performing a request forwarding process to forward a request for the target program to the alternative cluster so as to execute the alternative program in response to the request for the target program; How to manage your cluster.
2. A method for managing a cluster according to claim 1, comprising the steps of: the cluster having at least one worker pod; In the request forwarding process, all requests to the programs of the worker pods are forwarded to the alternative cluster as requests to the target program. How to manage your cluster.
3. A method for managing a cluster according to claim 2, comprising the steps of: The cluster and the alternative cluster are assigned different IP addresses, the DNS server stores a DNS record that associates an IP address of the cluster with a domain name of the cluster; The processor further comprises: When an alternative program of an alternative cluster is executed in place of the target program, execute a DNS record change process to transmit DNS record update information to the DNS server, the DNS record update information including information to rewrite the DNS record stored in the DNS server to a DNS record in which the IP address of the alternative cluster is associated with the domain name of the cluster; How to manage your cluster.
4. A method for managing a cluster according to claim 3, comprising the steps of: The processor of the alternative cluster When an alternative program of an alternative cluster is executed in place of the target program, Executing an alternative program determination process to determine whether there is a problem with the execution of the alternative program; If it is determined in the alternative program determination process that there is a problem with the execution of the alternative program, a DNS record update process is executed; When it is determined in the alternative program determination process that there is a problem with the execution of the alternative program, a transfer abort request process is executed to transmit transfer abort information including information to the cluster that the transfer is to be stopped; The processors of the cluster further include: When the transfer interruption information is received, the execution of the request transfer process for transferring the request to the target program to the alternative cluster is stopped, and a transfer interruption process is executed. How to manage your cluster.
5. A method for managing a cluster according to claim 4, comprising the steps of: The request forwarding process is executed by a forwarding pod provided in the cluster; The processors of the cluster Obtaining a data amount of a request for the target program sent from the client terminal to the cluster during a predetermined time interval; When the amount of data of the request to the target program transmitted from the acquired client terminal to the cluster is smaller than a predetermined data transmission amount lower limit, a transfer pod is deleted. How to manage your cluster.
6. A method for managing a cluster according to claim 1, comprising the steps of: The request forwarding process is executed by a forwarding pod provided in the cluster. How to manage your cluster.
7. A method for managing a cluster according to claim 6, comprising the steps of: The processor further comprises: Obtaining a data amount of a request for the target program sent from the client terminal to the cluster during a predetermined time interval; When the amount of data of the request to the target program transmitted from the acquired client terminal to the cluster is smaller than a predetermined data transmission amount lower limit, a transfer pod is deleted. How to manage your cluster.
8. A method for managing a cluster according to claim 1, comprising the steps of: The processor of the alternative cluster When an alternative program of an alternative cluster is executed in place of the target program, Executing an alternative program determination process to determine whether there is a problem with the execution of the alternative program; When it is determined in the alternative program determination process that there is a problem with the execution of the alternative program, a transfer abort request process is executed to transmit transfer abort information including information to the cluster that the transfer is to be stopped; The processors of the cluster further include: Upon receiving the transfer abort information, a transfer abort process is executed to stop the execution of the request transfer process for transferring the request to the target program to the alternative cluster so that the alternative program is executed in response to the request to the target program. How to manage your cluster.
9. A method for managing a cluster according to claim 8, comprising the steps of: The processor of the alternative cluster: Obtaining a frequency of errors in the alternative program; executing a first alternative program problem detection process for determining that there is a problem in the execution of the alternative program when the acquired frequency of errors in the alternative program is higher than a predetermined error frequency upper limit value; How to manage your cluster.
10. A method for managing a cluster according to claim 8, comprising the steps of: The processor of the alternative cluster: Obtaining a data amount of a request for the target program to be transferred from the cluster to the alternative cluster; execute a second alternative program problem detection process to determine that there is a problem with the execution of the alternative program when a transfer speed of a request to the target program transferred from the acquired cluster to the alternative cluster is greater than a predetermined data transfer speed upper limit value; How to manage your cluster.
11. A cluster including a plurality of nodes each having a memory unit for storing a program to be executed in response to a request from a client terminal connected to a network, and a processor for executing the program, When at least one alternative program stored in the alternative cluster is executed in place of at least one target program stored in the cluster, When a request for the target program transmitted from the client terminal is obtained, performing a request forwarding process to forward a request for the target program to the alternative cluster so as to execute the alternative program in response to the request for the target program; cluster.
12. A cluster management program to be executed by a cluster having a plurality of nodes, the cluster having a memory unit for storing a program to be executed in response to a request from a client terminal connected to a network, and a processor for executing the program, The processor, When a request for a target program transmitted from the client terminal is received, executing a request forwarding process that forwards a request to the target program to an alternative cluster so as to execute an alternative program in response to the request to the target program; Cluster management program.
Citation Information
Patent Citations
Cluster system, load balancer, method of transferring node, and node transfer program
JP2007156569A
Verification device and verification method
JP2019056986A
METHOD FOR PROVIDING TIERED LOAD BALANCING FOR A HOSTED VOICE-OVER INTERNET PROTOCOL (VoIP) PRIVATE BRANCH EXCHANGE (PBX)
US20150295834A1
