Processing system and distribution method

Abstract

This processing system comprises: a plurality of processing servers 2 that execute a stateful application; and a distribution server 1 that distributes, to the processing servers, a connection request directed to the application. The distribution server comprises: a calculation unit 12 that calculates a resource consumption amount that can be consumed by the processing servers, with respect to the connection request; an acquisition unit 13 that acquires the remaining resource amounts of each processing server from the plurality of processing servers; a determination unit 14 that determines whether the resource consumption amount can be allocated with respect to the remaining resource amounts of each processing server, and if the resource consumption amount can be allocated, determines, as the allocation destination processing server, one of the plurality of processing servers to which the resource consumption amount can be allocated; and a distribution unit 15 that distributes the connection request to the allocation destination processing server. Each of the plurality of processing servers 2 comprises a processing unit 21 that executes the application, and a notification unit 22 that issues a notification of the remaining resource amount.

Description

Processing system and distribution method

[0001] The present disclosure relates to a processing system and a distribution method.

[0002] There is a technology for distributing a connection request for an application to a processing server that executes the application (Patent Documents 1 and 2). The distribution server calculates the amount of consumption resources that can be consumed by the processing server due to the connection request, and determines the destination processing server so that the bias of the consumption resource amount becomes small among a plurality of processing servers, thereby improving the utilization efficiency of the resources of each processing server.

[0003] Japanese Patent No. 6511006, Japanese Patent No. 5580253

[0004] However, since the conventional distribution server fixedly allocates the amount of consumption resources that can be consumed to the processing server and manages the usage status of the resources of each processing server, in the case of a stateful application where the amount of consumption resources changes with time, there is a problem that the utilization efficiency of the resources of each processing server is not improved.

[0005] The present disclosure has been made in view of the above circumstances, and an object of the present disclosure is to provide a technology capable of surely improving the utilization efficiency of resources of a processing server that executes a stateful application.

[0006] A processing system in one aspect of the present disclosure comprises a plurality of processing servers that execute a stateful application, and a distribution server that distributes connection requests to the application to any of the plurality of processing servers, wherein the distribution server comprises a calculation unit that calculates the amount of resources that can be consumed by the processing servers due to the connection request, an acquisition unit that obtains the remaining resource amount of each processing server from the plurality of processing servers, a determination unit that determines whether the amount of resources can be allocated to the remaining resource amount of each processing server, and if the amount of resources can be allocated, determines which of the plurality of processing servers to which the amount of resources can be allocated is the processing server to which the resources will be allocated, and a distribution unit that distributes the connection request to the processing server to which the resources will be allocated, and each of the plurality of processing servers comprises a processing unit that executes the application and a notification unit that notifies the remaining resource amount.

[0007] A distribution method in one aspect of the present disclosure is a distribution method for distributing a connection request to a stateful application to one of a plurality of processing servers that execute the application, wherein the distribution server calculates the amount of resources that may be consumed by the processing server due to the connection request, obtains the remaining resources of each processing server from the plurality of processing servers, determines whether the amount of resources to be consumed can be allocated to the remaining resources of each processing server, and if the amount of resources to be consumed can be allocated, determines one of the plurality of processing servers to which the amount of resources to be consumed can be allocated as the destination processing server, and distributes the connection request to the destination processing server.

[0008] According to this disclosure, it is possible to reliably improve the resource utilization efficiency of processing servers running stateful applications.

[0009] Figure 1 shows an example of the processing system configuration. Figure 2 is a flowchart showing an example of the distribution server's operation. Figure 3 shows examples of the resource management table and resource utilization status. Figure 4 shows an image of resource allocation. Figure 5 is a flowchart showing an example of the processing server's operation. Figure 6 shows an example of the distribution server's hardware configuration.

[0010] Embodiments of this disclosure will be described below with reference to the drawings. In the drawings, the same parts are denoted by the same reference numerals and their descriptions are omitted.

[0011] [Summary of this disclosure] This disclosure calculates the amount of resources that a stateful application may consume in the future on processing servers based on a connection request, obtains the current remaining resource amount from each processing server, and determines which processing server to assign the connection request to based on the consumed resource amount and the remaining resource amount of each server. In this way, since the processing server to which a connection request is assigned is determined based on the current remaining resource amount obtained from each processing server, the resource utilization efficiency of each processing server running the stateful application can be reliably improved.

[0012] Furthermore, this disclosure dynamically increases or decreases the number of processing servers as needed. This ensures a more reliable improvement in the resource utilization efficiency of processing servers running stateful applications.

[0013] [Example of Processing System Configuration] Figure 1 shows an example of the configuration of the processing system according to this embodiment.

[0014] The processing system is a system that provides stateful application services. The processing system comprises a distribution server 1, multiple processing servers 2, and a management server 3.

[0015] The distribution server 1 has the function of receiving connection requests from multiple user terminals 4 to a stateful application and distributing said connection requests to one of the multiple processing servers 2.

[0016] Multiple processing servers 2 are equipped with the capability to execute stateful applications.

[0017] The management server 3 has the function of managing multiple processing servers 2.

[0018] [Specific Example of a Processing System] A stateful application is an application in which the amount of server resources consumed changes over time. Server resource consumption refers to the amount of CPU, memory, hard disk, etc., consumed by the server.

[0019] Specifically, a stateful application is one in which the application manages the state of various service requests from the user, and the response changes depending on the state of the service request. In other words, a stateful application is not a stateless application like a static website that returns the same response every time.

[0020] For example, stateful applications include web meetings and webinars. These are applications that require state management, such as user entry and exit, host / participant / viewer status, screen sharing, video camera, and audio microphone.

[0021] The multiple processing servers 2 are a group of processing servers that process stateful applications independently of each other. The multiple processing servers 2 are independent of each other. A processing server that receives a service request from a user cannot reallocate the processing to another processing server. In other words, the multiple processing servers 2 do not have shared storage and are not a group of processing servers that can reallocate processing among themselves.

[0022] Furthermore, the multiple processing servers 2 are a group of processing servers that can be automatically scaled up or down (auto-scale-out and scale-in). Unlike on-premises servers, the multiple processing servers 2 are not a group of processing servers that require manual scaling up or down. For example, the multiple processing servers 2 are a group of processing servers built on a virtual server infrastructure, a container infrastructure, and a cloud infrastructure.

[0023] The distribution server 1 and processing server 2 mentioned above are, for example, an SFU (Selective Forwarding Unit), an MCU (Multipoint Control Unit), and a WSR (WebRTC Signalling Runner).

[0024] [Configuration example and functions of the distribution server] As shown in Figure 1, the distribution server 1 comprises a reception unit 11, a calculation unit 12, an acquisition unit 13, a determination unit 14, and a distribution unit 15.

[0025] The reception unit 11 has the function of receiving connection requests from the user terminal 4 to a stateful application.

[0026] The calculation unit 12 has a function to estimate and calculate the amount of resources that may be consumed by the processing server as a result of the connection request.

[0027] The acquisition unit 13 has a function to acquire the remaining resource amount of each processing server 2 from a plurality of processing servers 2. The acquisition unit 13 may periodically acquire the remaining resource amount of each processing server 2.

[0028] The determination unit 14 has a function to determine whether or not the amount of resources consumed estimated by the calculation unit 12 can be allocated to the remaining resource amount of each processing server 2 acquired by the acquisition unit 13.

[0029] Furthermore, the determination unit 14 has a function to determine, if the amount of resources consumed estimated and calculated by the calculation unit 12 is available for allocation, to select one of the multiple processing servers 2 to which the amount of resources consumed is available as the destination processing server.

[0030] Furthermore, the determination unit 14 has a function to determine, if the amount of resources consumed estimated and calculated by the calculation unit 12 can be allocated, to select the processing server that will improve resource utilization efficiency (for example, the processing server that will improve it the most) from among the multiple processing servers 2 to which the amount of resources consumed can be allocated as the processing server to which the resources consumed can be allocated.

[0031] Furthermore, the determination unit 14, after allocating the estimated resource consumption amount calculated by the calculation unit 12, has a function to instruct the management server 3 to add processing servers if the remaining resource amounts of all of the multiple processing servers 2 meet the remaining resource amounts indicated in the processing server expansion conditions.

[0032] The distribution unit 15 has the function of distributing connection requests received by the reception unit 11 to the assigned processing server determined by the determination unit 14.

[0033] [Example of Processing Server Configuration and Function] Each of the multiple processing servers 2 comprises a processing unit 21, a notification unit 22, and an instruction unit 23, as shown in Figure 1.

[0034] The processing unit 21 has the function of executing stateful applications.

[0035] The notification unit 22 has a function to notify the distribution server 1 of the remaining resource amount of the processing server 2.

[0036] The instruction unit 23 has a function to instruct the management server 3 to reduce the number of processing servers 2 when the execution of a stateful application has finished.

[0037] [Example configuration and functions of the management server] The management server 3 is equipped with an expansion / decreasing unit 31, as shown in Figure 1.

[0038] The expansion / decreasing unit 31 has the function of adding a processing server that runs the same stateful application when the distribution server 1 instructs it to add a processing server.

[0039] The addition / removal unit 31 has the function of removing the processing server 2 that was instructed to be removed from the processing server 2.

[0040] [Example of Distribution Server Operation] Figure 2 is a flowchart showing an example of the operation of distribution server 1.

[0041] In this operation example, after receiving a new connection request to a stateful application, the destination processing server is determined based on the estimated consumed resource amount and the remaining resource amount of each obtained processing server. Also, in this operation example, when it is determined that the assigned processing server is insufficient, an instruction to add a processing server is issued.

[0042] Step S101; The reception unit 11 receives a new connection request from the user terminal 4 for a stateful application.

[0043] Step S102; The calculation unit 12 estimates and calculates the amount of consumed resources that can be consumed by the processing server due to the new connection request. That is, the calculation unit 12 estimates and calculates the amount of consumed resources of the processing server that can be consumed in the future by the stateful application due to the new connection request.

[0044] Step S 103; The acquisition unit 13 acquires the remaining resource amount of each processing server 2 from the plurality of processing servers 2. [[ID=X]] [[ID=Y]]

[0045] Step S104; The determination unit 14 determines whether the estimated consumed resource amount can be allocated to the remaining resource amount of each processing server 2.

[0046] Steps S105 to S106; When the estimated consumed resource amount estimated by the calculation unit 12 can be allocated, the determination unit 14 determines any one of the plurality of processing servers 2 to which the consumed resource amount can be allocated as the destination processing server. After that, the distribution unit 15 distributes the new connection request to the destination processing server.

[0047] Step S107; When the estimated consumed resource amount estimated by the calculation unit cannot be allocated, the determination unit 14 transmits an error response to the user terminal 4 that is the source of the new connection request.

[0048] Step S 108; After step S106, that is, after allocating the consumed resource amount, the determination unit 14 determines whether the remaining resource amounts of all the plurality of processing servers 2 satisfy the remaining resource amount indicated in the processing server addition condition.

[0049] Step S109; If the remaining resource amount specified in the processing server expansion conditions is met, the determination unit 14 instructs the management server 3 to add a processing server. Subsequently, the expansion / decreasing unit 31 of the management server 3 adds a processing server.

[0050] If the remaining resources do not meet the requirements for adding processing servers, the flow is terminated. The determination unit 14 does not instruct the management server 3 to add processing servers.

[0051] We have now explained the basic operation of distribution server 1. The following sections will detail each step.

[0052] (Regarding step S102) The calculation unit 12 calculates the expected value of the amount of resources consumed using the connection information obtainable from the reception unit 11.

[0053] For example, the calculation unit 12 calculates the average number of users per room from the number of rooms requested (created) from the processing server within a certain period of time and the number of users who entered each room, and then calculates the amount of resources consumed using this average number of users.

[0054] For example, the calculation unit 12 calculates the amount of resources consumed using the number of users entering the room (a fixed value) that is assumed in advance from the service specifications.

[0055] (Regarding step S104) As shown on the left side of Figure 3, the determination unit 14 allocates the estimated amount of resources consumed to the processing servers and manages the resource utilization status of each processing server. On the other hand, in the case of a stateful application, the amount of resources consumed changes over time, so the actual amount of resources consumed by each processing server is as shown on the right side of Figure 3.

[0056] Therefore, the distribution server 1 sequentially obtains the current remaining resource amount of each processing server 2 from the multiple processing servers 2, updates the resource management table with the current remaining resource amount, and determines whether or not the estimated amount of consumed resources can be allocated to the current remaining resource amount. An image of this determination is shown in Figure 4. For ease of understanding, the resource management table shown on the left side of Figure 4 depicts the resource status of the processing server 1 shown on the left side of Figure 3 before the update.

[0057] The determination unit 14 determines that all processing servers whose current remaining resource amount is greater than the amount of resources consumed (processing servers that have enough remaining resources to accommodate the amount of resources consumed) are available for allocation, and all processing servers whose current remaining resource amount is less than the amount of resources consumed are unavailable for allocation.

[0058] Furthermore, when comparing the current remaining resource amount with the amount of resources consumed, it is desirable to compare whether the consumed resources can be accommodated with the resource amount obtained by subtracting a certain buffer amount from the current remaining resource amount.

[0059] (Regarding steps S105 to S106) The determination unit 14 determines the processing server to be assigned using an online bin packing algorithm or a probabilistic bin packing algorithm. Representative algorithms that can be used for online bin packing include, for example, the following algorithms.

[0060] (1) The First Fit algorithm to assign to the first processing server found.

[0061] (2) The BestFit algorithm assigns resources to the processing server that has the smallest remaining resources after resource allocation.

[0062] (3) The WorstFit algorithm, which assigns resources to the processing server that has the largest remaining resources after resource allocation.

[0063] (4) The sum of squares S of B(h) (=Σ) so that the amount of remaining resources is diversified. ｈ＝１ Ｂ－１ B(h) ２The sum of squares method is used to minimize the allocation (where h is the remaining capacity, B is the bottle capacity, and B(h) is the number of remaining capacities h).

[0064] (Modifications of steps S104 to S109) (Modification 1) After determining the processing server in step S105, if a new amount of remaining resources is notified from each processing server 2 before distributing new connection requests in step S106, the process may return to step S104 and re-determine whether the amount of consumed resources can be allocated to the new amount of remaining resources of each processing server 2.

[0065] (Modification 2) If it is determined in step S104 that it is not possible to allocate the amount of resources to be consumed, instead of sending an error response to the user terminal 4 in step S107, the system may skip to step S109 to add a processing server, return to step S104, and re-determine whether it is possible to allocate the amount of resources to be consumed to the remaining resources of the multiple processing servers, including the added processing server.

[0066] (Modification 3) In Modification 2, a processing server is added when it is determined that the amount of resources to be consumed cannot be allocated, so a waiting period for accepting new connection requests occurs for the duration of the processing server's startup time. To avoid such waiting periods for accepting new connection requests, the system may skip to step S108 immediately after allocating a new connection request to a processing server, determine whether the remaining resource amount indicated in the processing server expansion conditions is met, and if it is met, instruct the system to add a processing server in step S109.

[0067] (Specific examples of expansion conditions) Expansion conditions are set according to the expected degree of sudden increase in connection requests. Specifically, if a relatively sudden increase is expected, such as due to planned congestion, a larger amount of remaining resources should be set as the expansion condition. On the other hand, if a relatively gradual increase is expected, such as due to an increase in the number of contracted users, a smaller amount of remaining resources should be set as the expansion condition.

[0068] [Example of Processing Server Operation] Figure 5 is a flowchart showing an example of the operation of processing server 2.

[0069] In this example, if the processing server to which the data is allocated becomes unused, an instruction will be given to reduce the number of processing servers.

[0070] Step S201; The processing unit 21 executes a stateful application.

[0071] Step S202; The instruction unit 23 determines whether the processing unit 21 has finished executing the application.

[0072] Step S203; When the execution of the application is completed, the instruction unit 23 instructs the management server 3 to remove the processing server 2. Subsequently, the addition / removal unit 31 of the management server 3 removes the processing server 2 that was instructed to be removed.

[0073] Up to this point, the basic operation of processing server 2 has been explained. The instruction unit 23 will be explained in more detail below.

[0074] The instruction unit 23 may be implemented as an instruction device independent of the processing server 2. This instruction device or a single instruction unit 23 monitors the start (in use) and end (unused) of execution of each application on the multiple processing servers 2, and notifies the management server 3 of the ID of the processing server that has finished executing the application.

[0075] [Supplementary information on adding or removing processing servers] Adding or removing processing servers may be done dynamically.

[0076] An alarm can be set on the distribution server 1 when it determines that there is a shortage of overall resources across multiple processing servers 2, and additional processing servers may be added when this alarm is triggered.

[0077] You may set an alarm on distribution server 1 that is triggered when the application status changes from "in use" to "unused," and then reduce the number of processing servers when this alarm is triggered.

[0078] [Effects] According to this embodiment, the distribution server 1 obtains the current remaining resource amount of each processing server from the multiple processing servers 2, and determines which processing server to allocate the connection request to based on the current remaining resource amount of each processing server. This ensures that the resource utilization efficiency of each processing server running a stateful application is reliably improved.

[0079] [Other] This disclosure is not limited to the embodiments described above. This disclosure can be modified in numerous ways within the scope of the gist of this disclosure.

[0080] The distribution server 1 of this embodiment described above can be implemented using a general-purpose computer system, for example, as shown in Figure 6, which includes a CPU 901, memory 902, storage 903, communication device 904, input device 905, and output device 906.

[0081] Memory 902 and storage 903 are storage devices. In this computer system, each function of the distribution server 1 is realized when the CPU 901 executes a predetermined program loaded onto memory 902.

[0082] The distribution server 1 may be implemented on a single computer. The distribution server 1 may be implemented on multiple computers. The distribution server 1 may be a virtual machine implemented on a computer.

[0083] The program for distribution server 1 can be stored on a computer-readable recording medium such as an HDD, SSD, USB memory, CD, or DVD. A computer-readable recording medium is, for example, a non-transitory recording medium. The program for distribution server 1 can also be distributed via a communication network.

[0084] The processing server 2 and the management server 3 can also be implemented using the general-purpose computer system shown in Figure 6.

[0085] 1 Distribution Server 11 Reception Unit 12 Calculation Unit 13 Acquisition Unit 14 Judgment Unit 15 Distribution Unit 2 Processing Server 21 Processing Unit 22 Notification Unit 23 Instruction Unit 3 Management Server 31 Add / Delete Unit 4 User Terminal 901 CPU 902 Memory 903 Storage 904 Communication Device 905 Input Device 906 Output Device

Claims

1. A processing system comprising: a plurality of processing servers that execute a stateful application; and a distribution server that distributes connection requests to the application to any of the plurality of processing servers, wherein the distribution server comprises: a calculation unit that calculates the amount of resources that can be consumed by the processing server due to the connection request; an acquisition unit that obtains the remaining resource amount of each processing server from the plurality of processing servers; a determination unit that determines whether the amount of resources can be allocated to the remaining resource amount of each processing server, and if the amount of resources can be allocated, determines which of the plurality of processing servers to which the amount of resources can be allocated is the processing server to which the resources will be allocated; and a distribution unit that distributes the connection request to the processing server to which the resources will be allocated, wherein each of the plurality of processing servers comprises: a processing unit that executes the application; and a notification unit that notifies the remaining resource amount.

2. The processing system according to claim 1, wherein, if the determination unit can allocate the amount of resources consumed, it determines among a plurality of processing servers to which the amount of resources consumed can be allocated a processing server that improves the efficiency of resource utilization, and selects that processing server as the destination processing server.

3. The processing system according to claim 1, further comprising: a determination unit that, after allocating the amount of resources to be consumed, instructs the management server to add processing servers if the remaining resources of all of the plurality of processing servers meet the remaining resource amount indicated in the processing server addition conditions; and an instruction unit that instructs the management server to remove processing servers if each of the plurality of processing servers has finished executing the application.

4. A distribution method for distributing a connection request to a stateful application to one of several processing servers that execute the application, wherein the distribution server calculates the amount of resources that may be consumed by the processing server due to the connection request, obtains the remaining resources of each processing server from the several processing servers, determines whether the amount of resources to be consumed can be allocated to the remaining resources of each processing server, and if the amount of resources to be consumed can be allocated, determines one of the several processing servers to which the amount of resources to be consumed can be allocated as the destination processing server, and distributes the connection request to the destination processing server.

Images