Resource allocation control device, computer system and resource allocation control method

Abstract

The present invention provides a resource allocation control device, a computer system and a resource allocation control method, which can appropriately allocate HW resources to software. A management node (100) for controlling the amount of hardware resources of storage nodes (120, 130) allocated to software for distributed data storage executed in the storage nodes (120, 130) is configured to have: a disk device that stores a performance model (111) representing the correspondence between the amount of hardware resources and the performance that can be achieved by the software using the hardware with the amount of resources; and a CPU connected to the disk device, the CPU receives the target performance of the distributed data storage (140, 141), determines the amount of hardware resources required to achieve the target performance based on the performance model (111), and sets the hardware so that the determined amount of resources is allocated to the program of the distributed data storage (140, 141).

Description

Technical Field

[0001] The present invention relates to a technology for controlling allocation of hardware resources (HW resources) used for software execution. Background Art

[0002] Data lakes are widely used as storage for AI (Artificial Intelligence) and data analysis. Data lakes are multi-protocol data storage, mainly using distributed data storage such as File storage / Object storage / NoSQL. In a data lake, multiple distributed data storages exist simultaneously, so in each node that constitutes the data lake, there is competition for HW resources between distributed data storages, and there is a problem of unstable performance of distributed data storage. Therefore, QoS (Quality of Service) control is required to solve such problems.

[0003] As a technology related to QoS control, for example, Cgroups of Linux (registered trademark) and a technology for controlling the amount of HW resources allocated to software in a server, such as the technology described in Patent Document 1, are known. When such a technology is used, the administrator of the data lake can set the amount of HW resources allocated to each distributed data storage executed in the data lake. As a result, each distributed data storage can occupy and use the set amount of HW resources even when contention occurs, and a certain performance can be maintained.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: U.S. Patent No. 10,255,217 Summary of the invention

[0007] Problems to be solved by the invention

[0008] In the above-mentioned technique, since the administrator manually calculates the amount of HW resources to be allocated to the distributed data storage, it is difficult to appropriately set the amount of HW resources to be allocated according to the target performance of each distributed data storage.

[0009] For example, if the amount of HW resources allocated to distributed data storage is insufficient, there is a possibility that the distributed data storage cannot achieve the target performance. On the other hand, if the amount of HW resources allocated to distributed data storage is excessive, there is a possibility that sufficient HW resources cannot be allocated to other distributed data storages, and the target performance of other distributed data storages cannot be achieved. Therefore, it is difficult to perform accurate QoS control for each distributed data storage.

[0010] In addition, in the application of data lakes, the relationship between the allocation of HW resources and performance may change due to the increase or decrease in the number of nodes constituting the data lake and the update and error of the distributed data storage program. Even if the administrator has accurately set the allocation of HW resources for distributed data storage at the beginning, it may be difficult to achieve the target performance when the relationship between the allocation of HW resources and performance changes.

[0011] In addition, in distributed data storage, the amount of HW resources required varies depending on the type of operation (IO operation) to be performed. Therefore, if the CPU allocation is increased for an operation that consumes more CPU (Central Processing Unit), but the operation is not actually performed, CPU resources will be excessively consumed.

[0012] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technology capable of appropriately allocating HW resources to software.

[0013] Technical solutions to solve problems

[0014] In order to achieve the above-mentioned purpose, a resource allocation control device in one aspect controls the amount of hardware resources of the node to be allocated to the software running in the specified node, and includes: a storage unit for storing performance resource amount information, wherein the performance resource amount information represents the correspondence between the amount of resources of the hardware and the performance that can be achieved by the software using the hardware with the resource amount; and a processor connected to the storage unit, the processor receiving the target performance of the software, determining the amount of hardware resources required to achieve the target performance based on the performance resource amount information, and setting it so that the hardware with the determined amount of resources is allocated to the software.

[0015] Effects of the Invention

[0016] According to the present invention, HW resources can be appropriately allocated to software. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a diagram showing the overall structure of the computer system according to the first embodiment.

[0018] Figure 2 This is a hardware configuration diagram of components related to the performance model generation process of the first embodiment.

[0019] Figure 3 This is a logical configuration diagram of components related to the performance model generation process of the first embodiment.

[0020] Figure 4 This is a flowchart of the performance model generation process according to the first embodiment.

[0021] Figure 5 This is a diagram showing an overview of the performance model of the first embodiment.

[0022] Figure 6 This is a diagram showing the structure of the performance model management table according to the first embodiment.

[0023] Figure 7 This is a hardware configuration diagram of components related to the resource allocation setting process of the first embodiment.

[0024] Figure 8 This is a logical configuration diagram of components related to the resource allocation setting process of the first embodiment.

[0025] Fig. 9 This is a flowchart of the resource allocation setting process according to the first embodiment.

[0026] Fig.10 This is a diagram showing a GUI (Graphical User Interface) screen for inputting target performance according to the first embodiment.

[0027] Fig.11 It is a diagram showing the structure of the resource allocation setting table according to the first embodiment.

[0028] Fig.12 This is a flowchart of the performance model correction / setting update process according to the first embodiment.

[0029] Fig.13 This is a flowchart of the operation mode learning process of the first embodiment.

[0030] Fig.14 4 is a diagram showing the structure of the operation mode management table according to the first embodiment.

[0031] Fig.15 This is a flowchart of the setting update process based on the operation mode according to the first embodiment.

[0032] Fig.16 This is a diagram for explaining an overview of updating of the resource allocation setting table according to the first embodiment.

[0033] Fig.17 This is a hardware configuration diagram of components related to the performance model generation process in the computer system according to the second embodiment.

[0034] Fig.18 This is a logical configuration diagram of components related to the performance model generation process of the second embodiment.

[0035] Fig.19 This is a flowchart of the performance model generation process according to the second embodiment.

[0036] Fig. 20 This is a diagram showing an overview of a performance model according to the second embodiment.

[0037] Fig.21 It is a diagram showing the structure of the performance model management table according to the second embodiment.

[0038] Fig. 22 This is a hardware configuration diagram of components related to resource allocation setting processing according to the second embodiment.

[0039] Fig.23 This is a logical configuration diagram of components related to resource allocation setting processing according to the second embodiment.

[0040] Fig.24 This is a flowchart of the resource allocation setting process according to the second embodiment.

[0041] Fig.25 It is a diagram showing a target performance input GUI screen according to the second embodiment.

[0042] Fig.26 It is a diagram showing the structure of a resource allocation setting table according to the second embodiment.

[0043] Fig. 27 This is a flowchart of the performance model correction / setting update process according to the second embodiment. DETAILED DESCRIPTION

[0044] The embodiments will be described with reference to the drawings. The embodiments described below do not limit the invention as claimed, and all of the elements and combinations thereof described in the embodiments are not necessarily essential to the solution of the invention.

[0045] Hereinafter, "program" is sometimes used as the subject of action to explain processing, but the program is executed by a processor (such as a CPU) and appropriately uses storage resources (such as memory) and / or communication interface devices (such as NIC (Network Interface Card)) to perform prescribed processing, so the subject of processing can also be regarded as a program. Processing described using a program as the subject of action can also be regarded as processing performed by a processor or a computer having the processor.

[0046] In the following description, information may be described using the term "AAA table", but information may be expressed using any data structure. In other words, in order to indicate that information does not depend on the data structure, "AAA table" may be referred to as "AAA information".

[0047] Figure 1 It is a diagram showing the overall structure of the computer system according to the first embodiment.

[0048] The computer system 1 includes a management node 100 as an example of a resource allocation control device; client nodes 150 and 151 ; storage nodes 120 and 130 ; a management client node 160 ; a performance model generation node 200 , and a benchmark execution node 210 .

[0049] Client nodes 150, 151 and storage nodes 120, 130 are connected via a network such as LAN (Local Area Network) 171. Storage nodes 120, 130, management node 100, management client node 160, performance model generation node 200, and benchmark execution node 210 are connected via a network such as LAN 170.

[0050] Client nodes 150 and 151 execute predetermined processing and, in conjunction with the processing, issue IO requests (read requests, write requests, etc.) to distributed data storage configured in storage nodes 120 and 130. There are two client nodes in the figure, but there may be one or more than one client node.

[0051] Storage nodes 120 and 130 are examples of operating nodes, which collaboratively constitute one or more distributed data storages (distributed data storage 140 (distributed data storage A), distributed data storage 141 (distributed data storage B)). In the figure, there are two storage nodes, but there can be three or more. In addition, in the figure, there are two distributed data storages composed of multiple storage nodes, but one or three or more distributed data storages can also be constituted.

[0052] The distributed data storage 140, 141 composed of the storage nodes 120, 130 receives IO requests from the client nodes 150, 160 and executes IO processing corresponding to the IO requests. In addition, the storage nodes 120, 130 respectively store resource allocation control programs 121, 131. The resource allocation control programs 121, 131 control the amount of HW resources allocated to the programs (software) of each distributed data storage based on the setting of the allocation amount of HW resources (HW resource allocation amount) from the management node 100.

[0053] The management client node 160 receives the target performance of each distributed data storage 140 , 141 configured in the storage node from the manager, and transmits the target performance to the management node 100 .

[0054] The management node 100 stores a QoS control program 110. The QoS control program 110 stores a performance model 111. The performance model 111 is an example of performance resource quantity information indicating the relationship between the resource allocation quantity and the performance of the program. Although only one performance model is shown in the figure, there may be performance models corresponding to the number of distributed data storages configured in the storage node.

[0055] The QoS control program 110 calculates the HW resource allocation amount for the distributed data storages 140 and 141 configured in the storage nodes 120 and 130 based on the performance model 111 and the target performance received from the management client node 160. In addition, the QoS control program 110 sends the calculated HW resource allocation amount to the resource allocation control program 121 of the storage node 120 and the resource allocation control program 131 of the storage node 130.

[0056] The performance model generation node 200 is an example of a generation node, and generates the performance model 111 and stores it in the management node 100. The benchmark execution node 210 measures the performance of the program (e.g., the program constituting the distributed data storage) in the storage nodes 120 and 130. In addition, at least one of the performance model generation node 200 and the benchmark execution node 210 may be embedded in the management node 100.

[0057] Next, QoS control in the data lake formed by the storage nodes 120 and 130 will be described.

[0058] Figure 2 This is a hardware configuration diagram of components related to the performance model generation process of the first embodiment.

[0059] In the present embodiment, the performance model generation process is mainly performed by the performance model generation node 200 , the benchmark execution node 210 , the storage node 120 , and the storage node 130 among the components of the computer system 1 .

[0060] In addition, in this embodiment, in the performance model generation process, storage nodes 120 and 130 that constitute the distributed data storage ultimately provided to the client node are used, but the present invention is not limited to this, and multiple storage nodes other than storage nodes 120 and 130 may also be used.

[0061] The storage node 120 includes a CPU 222 as an example of a processor, a memory 223 as a primary storage device, a disk device 224 as an auxiliary storage device (secondary storage device), and NICs 225 and 226. The CPU 222, memory 223, disk device 224, and NICs 225 and 226 are connected via a bus 221.

[0062] CPU 222 performs various processes by reading programs stored in disk device 224 into memory 223 and executing them. CPU 222 sends and receives data to and from other devices (storage node 130, performance model generation node 200, benchmark execution node 210, etc.) connected to LAN 170 via bus 221 and NIC 226.

[0063] The storage node 130 includes a CPU 232 as an example of a processor, a memory 233 as a primary storage device, a disk device 234 as an auxiliary storage device, and NICs 235 and 236. The CPU 232, the memory 233, the disk device 234, and the NICs 235 and 236 are connected via a bus 231.

[0064] The CPU 232 performs various processes by reading and executing programs stored in the disk device 234 into the memory 233. The CPU 232 sends and receives data to and from other devices (such as the storage node 120, the performance model generation node 200, and the benchmark execution node 210) connected to the LAN 170 via the bus 231 and the NIC 236.

[0065] The performance model generation node 200 includes a CPU 202 as an example of a processor, a memory 203 as a main storage device, a disk device 204 as an auxiliary storage device, and a NIC 205. The CPU 202, the memory 203, the disk device 204, and the NIC 205 are connected via a bus 201.

[0066] CPU 202 performs various processes by reading programs stored in disk device 204 into memory 203 and executing them. CPU 202 sends and receives data to and from other devices (storage nodes 120 and 130, benchmark execution node 210, etc.) connected to LAN 170 via bus 201 and NIC 205.

[0067] The benchmark execution node 210 includes a CPU 212 as an example of a processor, a memory 213 as a main storage device, a disk device 214 as an auxiliary storage device, and a NIC 215. The CPU 212, the memory 213, the disk device 214, and the NIC 215 are connected via a bus 211.

[0068] CPU 212 performs various processes by reading programs stored in disk device 214 into memory 213 and executing them. CPU 212 sends and receives data to and from other devices (storage nodes 120 and 130 , performance model generation node 200 , etc.) connected to LAN 170 via bus 211 and NIC 215 .

[0069] Figure 3This is a logical configuration diagram of components related to the performance model generation process of the first embodiment.

[0070] The storage node 120 stores a data storage program 341 (data storage program A), a data storage program 351 (data storage program B), and a resource allocation control program 121 .

[0071] The storage node 130 stores the data storage program 342 (data storage program A), the data storage program 352 (data storage program B), and the resource allocation control program 131 .

[0072] The data storage program 341 executed by the CPU 222 of the data node 120 and the data storage program 342 executed by the CPU 232 of the storage node 130 work in cooperation to form the distributed data storage 140 (distributed data storage A).

[0073] In addition, the data storage program 351 executed by the CPU 222 of the data node 120 and the data storage program 352 executed by the CPU 232 of the storage node 130 work in cooperation to form the distributed data storage 141 (distributed data storage B).

[0074] When executed by the CPU 222 , the resource allocation control program 121 controls the amount of HW resources (HW resource allocation amount) allocated to each data storage program executed in the storage node 120 .

[0075] When the resource allocation control program 131 is executed by the CPU 232 , it controls the amount of HW resources allocated to each data storage program executed in the storage node 130 .

[0076] The performance model generation node 200 stores a performance model generation program 300. When the performance model generation program 300 is executed by the CPU 202, the performance model of the distributed data storages 140 and 141 configured in the storage nodes 120 and 130 is generated.

[0077] The benchmark execution node 210 stores a benchmark program 310. When the benchmark program 310 is executed by the CPU 212, the performance of the distributed data storages 140 and 141 configured in the storage nodes 120 and 130 is measured.

[0078] Next, the performance model generation process 400 for generating a performance model will be described.

[0079] Figure 4 This is a flowchart of the performance model generation process according to the first embodiment.

[0080] In the performance model generation process of the present embodiment, a performance model is generated for each type of operation of the distributed data storage. Here, the operation refers to the IO operation on the data storage. For example, if the data storage is a File data storage, it is a sequential Read / Write, a random Read / Write, and a metadata operation (such as file creation / deletion, directory creation / deletion, etc.). If the data storage is a NoSQL data storage, it refers to the insertion, deletion, and retrieval of data in the DB (Data Base). In addition, in the present embodiment, a performance model for each type of operation is generated for each HW resource. Here, HW resources are CPU, memory, NIC bandwidth, IO bandwidth of the disk device, etc. Through this performance model generation process, a performance model representing the relationship between the allocation amount of each HW resource and the performance of the distributed data storage (strictly speaking, the data storage program) at the time of the allocation amount is generated for each type of operation.

[0081] First, the performance model generation program 300 of the performance model generation node 200 (strictly speaking, the CPU 202 that runs the performance model generation program 300) checks whether the performance models for all the operation types of the distributed data storage have been generated (step 410). If the performance models have been generated for all the operation types (step 410: Yes), the performance model generation program 300 ends the performance model generation process 400.

[0082] On the other hand, when performance models for all operation types have not yet been generated (step 410: No), the performance model generation program 300 generates performance models for the ungenerated operation types. Here, the target operation types are referred to as target operation types.

[0083] First, the performance model generation program 300 checks whether performance models of all HW resources for which performance models are to be generated have been generated for the target operation type (step 420 ).

[0084] As a result, when performance models of all target HW resources have been generated for the target operation type (step 420 : Yes), the performance model generation program 300 advances the processing to step 410 .

[0085] On the other hand, when the performance models of all the target HW resources have not been generated yet (step 420: No), the performance model generation program 300 generates the performance models of the HW resources that have not been generated (referred to as the target HW resources). Here, when the performance model of the target HW resources is generated, the HW resource allocation amount of the target HW resources is gradually changed by setting the allocation amount of the HW resources other than the target HW resources to a level that does not become a performance bottleneck of the target HW resources, thereby generating the performance model.

[0086] First, the performance model generation program 300 changes the amount of HW resources allocated to the distributed data storage (step 430). The change in the amount of HW resource allocation in the distributed data storage is performed by cooperating with the resource allocation control program of each storage node. Specifically, the performance model generation program 300 sends the amount of HW resource allocation to the data storage program of the distributed data storage to the resource allocation control program of each storage node. Thus, the resource allocation control program receives the amount of HW resource allocation and allocates the HW resources of the received amount of HW resource allocation to the data storage program. The allocation of HW resources in the storage node can be implemented using existing programs and software. For example, if it is a Linux operating system, a resource allocation function called Cgroups can be used. When using this Cgroups, the required amount of HW resources can be allocated to the program running in the Linux operating system.

[0087] Next, the performance model generation program 300 instructs the benchmark program 310 of the benchmark execution node 210 to perform performance measurement on the object operation type of the distributed data storage (step 440). As a result, the benchmark program 310 performs the performance measurement of the distributed data storage (specifically, the distributed data storage program) and sends the performance measurement result to the performance model generation program 300.

[0088] Next, the performance model generation program 300 determines whether the performance model can be generated, specifically, whether the performance measurement has been performed a number of times required for generating the performance model (step 450 ).

[0089] As a result, if the number of measurements required for generating the performance model has not been performed (step 450: No), the performance model generation program 300 advances the process to step 430 to repeat the change of the HW resource allocation amount and the execution of the performance measurement. In addition, the number of times the performance measurement is performed to generate the performance model and the HW resource allocation amount changed for each performance measurement are predetermined.

[0090] On the other hand, when the number of measurements required for generating the performance model has been performed (step 450: Yes), the performance model generating program 300 generates a performance model based on the plurality of measurement results, and registers the generated performance model in the performance model management table 510 (refer to Figure 6 ) (step 460), causing the processing to proceed to step 420.

[0091] Here, generation of a performance model and registration of a performance model are described.

[0092] Figure 5 This is a diagram showing an overview of the performance model of the first embodiment.

[0093] The performance model can be generated, for example, as Figure 5 As shown, a graph of the change in performance relative to the amount of HW resources is generated, and the formula y=f(x) of the approximate curve of the graph is used as a performance model. Here, y represents the performance of the distributed data storage of one node, and x represents the amount of HW resources of one node. y can be calculated by dividing the result of the performance measurement from the benchmark program 310 (the overall performance of the distributed data storage) by the number of nodes of the distributed data storage. In other words, when y is multiplied by the number of nodes of the distributed data storage, the overall performance of the distributed data storage is obtained. The generation of the graph and the derivation of the approximate curve formula in the generation of the performance model can be achieved by using existing spreadsheet software and programs.

[0094] Figure 6 This is a diagram showing the structure of the performance model management table according to the first embodiment.

[0095] The registration of the performance model is performed, for example, on the performance model management table 510. Row 511 of the performance model management table 510 stores information indicating the distributed data storage corresponding to the performance model registered in the performance model management table 510 (distributed data storage name). Column 512 stores the type of operation corresponding to the performance model. Row 513 stores the type of HW resource for which the performance model is an object. In the cells corresponding to the type of each operation in column 512 and the type of HW resource in row 513, the performance model for the type of HW resource when the type of operation is stored by the performance model generation program 300.

[0096] In addition, in the present embodiment, an expression (an example of a performance model) generated based on the performance measurement result is stored, but for example, a plurality of sets of HW resource amounts and corresponding measured performances may be recorded.

[0097] Next, the resource allocation setting process will be described.

[0098] Figure 7 This is a hardware configuration diagram of components related to the resource allocation setting process of the first embodiment.

[0099] In the present embodiment, the storage nodes 120 and 130 , the client nodes 150 and 151 , the management node 100 , and the management client node 160 mainly participate in the resource allocation setting process among the components of the computer system 1 .

[0100] The CPU 222 of the storage node 120 sends and receives data with other devices (storage node 130, client nodes 150, 151, etc.) connected to the LAN 171 via the bus 221 and the NIC 225. In addition, the CPU 222 of the storage node 120 sends and receives data with other devices (management node 100, etc.) connected to the LAN 170 via the bus 221 and the NIC 226.

[0101] The CPU 232 of the storage node 130 sends and receives data with other devices (storage node 130, client nodes 150, 151, etc.) connected to the LAN 171 via the bus 231 and the NIC 235. In addition, the CPU 232 of the storage node 130 sends and receives data with other devices (management node 100, etc.) connected to the LAN 170 via the bus 231 and the NIC 236.

[0102] The client nodes 150 and 151 perform IO for the distributed data storage formed in the storage nodes 120 and 130. Each of the client nodes 150 and 151 has a CPU, a memory, a NIC, and a disk device connected via a bus, which are omitted from the figure. The CPU calls the program stored in the disk device into the memory and executes it. The client nodes 150 and 151 send and receive data with other devices (such as storage nodes 120 and 130) via the LAN 171.

[0103] The management client node 160 has a CPU, memory, NIC, and disk device connected via a bus (not shown). The CPU calls the program stored in the disk device into the memory and executes it. The management client node 160 sends and receives data with other devices (such as the management server 100) via the LAN 170.

[0104] The management node 100 includes a CPU 602 as an example of a processor, a memory 603 as a primary storage device, a disk device 604 as an auxiliary storage device, and a NIC 605. The CPU 602, the memory 603, the disk device 604, and the NIC 605 are connected via a bus 601. Here, at least one of the memory 603 and the disk device 604 is an example of a storage unit.

[0105] The CPU 602 performs various processes by reading programs stored in the disk device 604 into the memory 603 and executing the programs. The CPU 602 transmits and receives data to and from other devices (storage nodes 120 and 130, management client node 160, etc.) connected to the LAN 170 via the bus 601 and the NIC 605.

[0106] Figure 8is a logical structure diagram of components related to resource allocation setting processing in the first embodiment. Figure 3 The same components as those shown are denoted by the same reference numerals, and duplicate descriptions are omitted.

[0107] The storage node 120 stores a data storage program 341 (data storage program A), a data storage program 351 (data storage program B), a QoS control program 720 , and a resource allocation control program 121 .

[0108] The storage node 130 stores the data storage program 342 (data storage program A), the data storage program 352 (data storage program B), the QoS control program 730 , and the resource allocation control program 131 .

[0109] The QoS control program 720 of the storage node 120 includes an IO monitoring unit 721 and a resource monitoring unit 722 , and when executed by the CPU 222 , monitors the IO processing and resource consumption of the data storage programs 341 and 351 .

[0110] The QoS control program 730 of the storage node 130 includes an IO monitoring unit 731 and a resource monitoring unit 732 , and when executed by the CPU 232 , monitors the IO processing and resource consumption of the data storage programs 342 and 352 .

[0111] The client node 150 stores the data analysis program 760. The data analysis program 760 is executed by the client node 150 to perform IO with respect to the distributed data storage configured in the storage nodes 120 and 130.

[0112] The client node 151 stores the data analysis program 770. The data analysis program 770 is executed by the client node 151 to perform IO with respect to the distributed data storage configured in the storage nodes 120 and 130.

[0113] The management client node 160 stores a web browser 710 . The web browser 710 receives input of settings of target performance of distributed data storage from a manager by being executed by the management client node 160 , and transmits the target settings to the management node 100 .

[0114] The management node 100 stores the QoS control program 110. The QoS control program 110 includes a resource allocation setting unit 701, a performance model management unit 702, an operation mode learning unit 703, a performance model management table 510, an operation mode management table 704, and a resource allocation setting table 705. Although omitted in the figure, the QoS control program 110 includes performance model management tables 510 corresponding to the number of distributed data storage executed in the storage node.

[0115] The resource allocation setting unit 701 is executed by the CPU 602 of the management node 100 to receive the target performance of the distributed data storage input via the Web browser 710, updates the resource allocation setting table 705 based on the received target performance and the contents of the performance model management table 510, and sends the set value of the resource allocation amount to each storage node 120, 130.

[0116] The performance model management unit 702 receives the monitoring results of the IO processing and resource consumption of the data storage programs 341, 342, 351, 352 from the QoS control programs 720, 730 executed in the storage nodes 120, 130 by the CPU 602, and modifies the performance model based on the monitoring results. In addition, the performance model management unit 702 updates the resource allocation setting table 705 based on the modified performance model, and sends the setting value of the resource allocation amount to each storage node 120, 130.

[0117] The operation mode learning unit 703 receives the monitoring results of the IO processing of the data storage programs 341, 342, 351, and 352 from the QoS control programs 720 and 730 executed in the storage nodes 120 and 130 by being executed by the CPU 602, learns the operations performed in the distributed data storage based on the monitoring results, and updates the operation mode management table 704. In addition, the operation mode learning unit 703 updates the resource allocation setting table 705 based on the operation mode management table 704, and sends the setting value of the resource allocation amount to each storage node 120 and 130.

[0118] Next, resource allocation setting processing 800 for setting allocation of HW resources to data storage will be described.

[0119] Fig. 9 This is a flowchart of the resource allocation setting process according to the first embodiment.

[0120] The Web browser 710 of the management client node 160 displays a target performance input GUI screen 900 (see Fig.10 ), receive target performance for each distributed data store from the data lake manager (step 810).

[0121] Here, the target performance input GUI screen 900 will be described.

[0122] Fig.10 It is a diagram showing a GUI screen for inputting target performance according to the first embodiment.

[0123] The target performance input GUI screen 900 has input fields (input fields 901, 902) for the target performance of each distributed data storage and a send button 903. Fig.10The target performance input GUI screen 900 shown shows input fields in the case of two distributed data stores, namely distributed data store A and distributed data store B, but input fields corresponding to the number of distributed data stores configured in the storage node are prepared. Input fields 901 and 902 include input fields for inputting target performance for each type of operation in the distributed data store. The administrator can input the target performance for each type of operation of the distributed data store in the input field. The send button 903 is a button for receiving an instruction to send the setting of the target performance input in the input fields 901 and 902 to the management node 100. When the send button 903 is pressed, the management client node 160 sends the setting of the target performance input in the input fields 901 and 902 to the management node 100.

[0124] return Fig. 9 As described above, when the administrator of the data lake presses the send button 903 of the GUI screen 900 for inputting target performance, the web browser 710 sends the target performance settings (target performance settings) inputted in the input fields 901 and 902 to the management node 100 (step 811).

[0125] Next, the resource allocation setting unit 701 of the management node 100 receives the transmitted target performance setting (step 820). The resource allocation setting unit 701 registers the received target performance in the resource allocation setting table 705 (refer to Fig.11 )middle.

[0126] Fig.11 It is a diagram showing the structure of the resource allocation setting table according to the first embodiment.

[0127] The resource allocation setting table 705 is set for each distributed data storage configured in the storage node, and manages the relationship between the target performance of each operation type of the distributed data storage and the required HW resource amount, and the setting of the resource allocation amount for the distributed data storage. Row 1001 of the resource allocation setting table 705 stores information (distributed data storage name) indicating the distributed data storage corresponding to the resource allocation setting table 705. Row 1002 stores the operation type of the distributed data storage. Row 1003 stores the target performance of each operation type. Row 1004 stores the resource amount required for the target performance of each operation type of each HW resource. Column 1005 stores the setting value of the resource allocation amount for the distributed data storage (distributed data storage program).

[0128] return Fig. 9 Next, the resource allocation setting unit 701 calculates the HW resource amount (resource allocation amount) actually allocated to the distributed data storage by referring to the performance model management table 510. Here, the calculated HW resource amount is the HW resource amount allocated to the data storage program of the distributed data storage in each storage node.

[0129] First, the resource allocation setting unit 701 calculates the amount of HW resources required for each type of operation to achieve the target performance based on the performance model (formula) recorded in the performance model management table 510 and the target performance of each type of operation, and registers it in row 1004 of the resource allocation setting table 705 (step 821). Specifically, the resource allocation setting unit 701 divides the target performance of each type of operation by the number of storage nodes constituting the distributed data storage, substitutes the result into y of the performance model formula (y=f(x)), and calculates the required amount of HW resources x. In addition, when recording a plurality of sets of HW resource amounts and measured performance instead of the performance model, the amount of HW resources required for the target performance can be calculated based on the plurality of sets.

[0130] Next, the resource allocation setting unit 701 calculates the resource allocation amount (step 822). Specifically, the resource allocation setting unit 701 refers to row 1004 of the resource allocation table 705 and confirms the required HW resource amount of each HW resource for each operation type. Next, the resource allocation setting unit 701 determines the maximum value of the required HW resource amount as the resource allocation amount and registers it in the column of the corresponding HW resource in column 1005. Here, the maximum value of the required HW resource amount is used as the resource allocation amount in order to ensure that the target performance can be achieved regardless of which operation is executed in the distributed data storage.

[0131] In addition, when multiple types of operations are executed simultaneously and target performance in multiple types of operations needs to be achieved, the total value of the HW resource amounts required for these types of operations can be used as the resource allocation amount. Whether to use the maximum value of the required HW resource amount as the resource allocation amount or the total value of multiple required HW resource amounts as the resource allocation amount can be determined according to the pre-set QoS policy.

[0132] Next, the resource allocation setting unit 701 transmits the content of the column 1005 of the resource allocation setting table 705, that is, the resource allocation amount for each HW resource, to the resource allocation control programs 121 and 131 of each storage node 120 and 130 (step 823).

[0133] The resource allocation control programs 121 and 131 of the storage nodes 120 and 130 receive the content of column 1005 of the resource allocation setting table 705 (step 830), and allocate HW resources to the data storage program of the distributed data storage based on the resource allocation amount for each HW resource in column 1005 (step 831).

[0134] After the allocation setting is completed, the resource allocation control programs 121 and 131 transmit a notification indicating that the setting is completed (setting completion notification) to the resource allocation setting unit 701 of the management node 100 (step 832 ).

[0135] The resource allocation setting unit 701 of the management node 100 receives the setting completion notification (step 824), and checks whether the setting completion notification has been received from all the storage nodes that have sent the resource allocation amount for each HW resource (step 825). If the resource allocation setting unit 701 has not received the setting completion communication from all the storage nodes (step 825: No), the processing proceeds to step 824. On the other hand, if the setting completion communication has been received from all the storage nodes (step 825: Yes), the resource allocation setting processing 800 ends.

[0136] Next, a description will be given of a performance model correction / setting update process 1100 for correcting a performance model and resetting resource allocation based on the corrected performance model.

[0137] Fig.12 This is a flowchart of the performance model correction / setting update process according to the first embodiment.

[0138] The performance model correction / setting update process is performed, for example, periodically. First, the IO monitoring units 721, 731 of the QoS control programs 720, 730 of each storage node 120, 130 obtain the IO operation log of the data storage program of each distributed data storage (step 1110). Here, in the IO operation log, the content of the IO operation performed by the data storage program 341, 342, 351, 352, the timing of performing the IO operation, the processing time, etc. are recorded. Next, the resource monitoring units 722, 732 obtain the log (resource consumption log) of the amount of HW resources consumed by the data storage program 341, 342, 351, 352 of each distributed data storage (step 1111). The resource consumption log can be obtained using the function of the OS (operating system), etc.

[0139] Next, the QoS control programs 720 and 730 send the acquired IO operation logs and resource consumption logs to the performance model management unit 702 of the management node 100 (step 1112 ).

[0140] The performance model management unit 702 of the management node 100 receives the IO operation log and resource consumption log sent from each storage node 120 and 130 (step 1120).

[0141] Next, the performance model management unit 702 obtains the actual performance of the distributed data storage based on the IO operation log, compares the actual performance (actual performance) with the performance (estimated performance) calculated based on the performance model, and determines whether the difference between the actual performance and the estimated performance exceeds a predetermined threshold value (step 1121). As a result, when the difference between the actual performance and the estimated performance does not exceed the predetermined threshold value (step 1121: No), the performance model management unit 702 advances the processing to step 1120. On the other hand, when the difference between the actual performance and the estimated performance exceeds the predetermined threshold value (step 1121: Yes), the performance model management unit 702 performs a process of correcting the performance model and updates the performance model of the performance model management table 510 to the corrected performance model (step 1122). The process of correcting the performance model can be implemented based on the obtained resource consumption log and the performance value obtained based on the IO operation log, using the same process as the performance model generation process 400.

[0142] Next, performance model management unit 702 calculates new resource allocation amounts based on the updated performance model in performance model management table 510 and updates resource allocation setting table 705 (step 1123 ). Calculation of new resource allocation amounts can be implemented in the same manner as resource allocation setting process 800 .

[0143] Next, the performance model management unit 702 transmits the content of the column 1005 of the resource allocation setting table 705, that is, the resource allocation amount for each HW resource, to the resource allocation control programs 121 and 131 of each storage node 120 and 130 (step 1124).

[0144] The resource allocation control programs 121 and 131 of the storage nodes 120 and 130 receive the content of column 1005 of the resource allocation setting table 705 (step 1113), and allocate HW resources to the data storage program of the distributed data storage based on the resource allocation amount for each HW resource in column 1005 (step 1114).

[0145] After the allocation setting is completed, the resource allocation control programs 121 and 131 transmit a notification indicating that the setting is completed (setting completion notification) to the performance model management unit 702 of the management node 100 (step 1115 ).

[0146] The performance model management unit 702 of the management node 100 receives the setting completion notification (step 1125), and checks whether the setting completion notification has been received from all the storage nodes that have sent the resource allocation amount for each HW resource (step 1126). If the performance model management unit 702 has not received the setting completion communication from all the storage nodes (step 1126: No), the processing proceeds to step 1125. On the other hand, if the setting completion communication has been received from all the storage nodes (step 1126: Yes), the performance model correction / setting update processing 1100 ends.

[0147] Through the performance model correction / setting update process 1100, it is possible to update the performance model to an appropriate one according to the actual situation of IO processing of each storage node 120, 130, and to appropriately allocate HW resources according to the actual situation.

[0148] Next, the operation pattern learning process 1200 for learning the execution pattern of the operation in the distributed data storage is described.

[0149] Fig.13 This is a flowchart of the operation mode learning process of the first embodiment.

[0150] First, the IO monitoring units 721 and 731 of the QoS control programs 720 and 730 of each storage node 120 and 130 obtain the IO operation log of the data storage program of each distributed data storage (step 1210). Then, the QoS control programs 720 and 730 send the obtained IO operation log to the operation mode learning unit 703 of the management node 100 (step 1211).

[0151] The operation mode learning unit 703 of the management node 100 receives the IO operation log transmitted from each of the storage nodes 120 and 130 (step 1220 ).

[0152] Next, the operation mode learning unit 703 confirms the received IO operation log and determines whether sufficient data for learning has been collected (step 1221). Here, the basis for judging whether sufficient data for learning has been collected depends on the learning method of the execution mode adopted in step 1222 described later. As a result, when sufficient data for learning has not been collected (step 1221: No), the operation mode learning unit 703 advances the processing to step 1220. On the other hand, when sufficient data for learning has been collected (step 1221: Yes), the operation mode learning unit 703 learns the execution mode of the operation and registers it in the operation mode management table 704 (reference Fig.14 ) (step 1222), ending the operation mode learning process 1200.

[0153] Fig.144 is a diagram showing the structure of the operation mode management table according to the first embodiment.

[0154] The operation mode management table 704 is set for each distributed data storage, and stores the type of operation performed at each time (day of the week, time, etc.) for the distributed data storage. Row 1301 of the operation mode management table 704 stores information (distributed data storage name) indicating the distributed data storage corresponding to the operation mode management table 704. In column 1302, the time (period) during which the operation is performed is stored. In column 1303, the type of operation performed at the corresponding time is recorded. Fig.14 In the example of , column 1303 includes columns corresponding to each day of the week, and records the type of operation performed on each day of the week. Here, in the distributed data storage, as a learning method for the type of operation performed at each opportunity, an existing method such as machine learning can be used.

[0155] Next, the setting update process 1400 based on the operation mode will be described.

[0156] Fig.15 This is a flowchart of the setting update process based on the operation mode according to the first embodiment. Fig.16 This is a diagram for explaining an overview of updating of the resource allocation setting table according to the first embodiment.

[0157] First, the operation mode learning unit 703 of the management node 100 checks the current time (step 1410). The current time can be checked by using a function of the OS, for example.

[0158] Next, the operation mode learning unit 703 compares the current time with the content of row 1302 of the operation mode management table 704 to check whether it is now the time to change the operation execution mode (step 1411). If it is not the time to change the operation execution mode (step 1411: No), the operation mode learning unit 703 advances the processing to step 1410.

[0159] On the other hand, if it is time to change the operation execution mode (step 1411: Yes), the operation mode learning unit 703 updates the resource allocation setting table 705 (step 1412). As a method of updating the resource allocation setting table 705, the HW resource allocation amount is calculated by excluding the operation types that are not executed in the next execution mode. Specifically, Fig.16 As shown, when only the sequential read operation type is executed as the next execution mode, the HW resource amount required for the executed operation type, that is, 0.5 core is calculated as the CPU allocation amount, and the cell 1500 of the resource allocation setting table 705 is updated.

[0160] Next, the operation mode learning unit 703 transmits the content of the updated resource allocation setting table 705 , that is, the resource allocation amount for each HW resource, to the resource allocation control programs 121 and 131 of each storage node 120 and 130 (step 1413 ).

[0161] The resource allocation control programs 121 and 131 of the storage nodes 120 and 130 receive the content of column 1005 of the resource allocation setting table 705 (step 1420), and allocate HW resources to the data storage program of the distributed data storage based on the resource allocation amount for each HW resource in column 1005 (step 1421).

[0162] After the allocation setting is completed, the resource allocation control programs 121 and 131 transmit a notification indicating that the setting is completed (setting completion notification) to the operation mode learning unit 703 of the management node 100 (step 1422).

[0163] The operation mode learning unit 703 of the management node 100 receives the setting completion notification (step 1414), and checks whether the setting completion notification has been received from all the storage nodes that have sent the resource allocation amount for each HW resource (step 1415). If the operation mode learning unit 703 has not received the setting completion communication from all the storage nodes (step 1415: No), the processing proceeds to step 1414. On the other hand, if the setting completion communication has been received from all the storage nodes (step 1415: Yes), the setting update processing 1400 based on the operation mode ends.

[0164] This operation mode-based setting update process 1400 eliminates the need to allocate HW resources required for operation types that are not executed among the types of operations for distributed data storage, and enables appropriate allocation of HW resources.

[0165] Next, a computer system according to a second embodiment will be described.

[0166] In the computer system of the first embodiment, the amount of HW resources is controlled by taking the data storage program constituting the distributed data storage as the object, whereas in the computer system of the second embodiment, the amount of HW resources is controlled by taking an application program (also called an application) as an example of a program (software) executed in one or more nodes as the object.

[0167] The computer system of the second embodiment is Figure 1 In the computer system 1 of the first embodiment shown in the figure, a backup management node 2000 (see Fig.23 ), instead of the client nodes 150, 151, there are client nodes 2040, 2050 (refer to Fig.23), instead of the storage nodes 120 and 130, there are application execution nodes 1620 and 1630 as an example of an execution node (refer to Fig.17 ), a management client node 2010 (reference Fig.23 ), instead of the performance model generation node 200, there is a performance model generation node 1600 (reference Fig.17 ), instead of the reference execution node 210, there is a reference execution node 1610 (reference Fig.17 ).

[0168] Next, QoS control for applications executed in the application execution nodes 1620 and 1630 will be described.

[0169] Fig.17 This is a hardware configuration diagram of components related to the performance model generation process in the computer system according to the second embodiment.

[0170] In this embodiment, the performance model generation process is mainly performed by the components of the computer system including the performance model generation node 1600 , the benchmark execution node 1610 , the application execution node 1620 , and the application execution node 1630 .

[0171] In addition, in this embodiment, in the performance mode generation process, application execution nodes 1620 and 1630 that constitute the application ultimately provided to the client node are used, but the present invention is not limited to this, and multiple application execution nodes other than application execution nodes 1620 and 1630 may also be used.

[0172] The application execution node 1620 includes a CPU 1622 as an example of a processor, a memory 1623 as a primary storage device, a disk device 1624 as an auxiliary storage device, and NICs 1625 and 1626. The CPU 1622, the memory 1623, the disk device 1624, and the NICs 1625 and 1626 are connected via a bus 1621.

[0173] The CPU 1622 performs various processes by reading the program stored in the disk device 1624 into the memory 1623 and executing the program. The CPU 1622 sends and receives data to and from other devices (such as the application execution node 1630, the performance model generation node 1600, and the benchmark execution node 1610) connected to the LAN 170 via the bus 1621 and the NIC 1626.

[0174] The application execution node 1630 includes a CPU 1632 as an example of a processor, a memory 1633 as a primary storage device, a disk device 1634 as an auxiliary storage device, and NICs 1635 and 1636. The CPU 1632, the memory 1633, the disk device 1634, and the NICs 1635 and 1636 are connected via a bus 1631.

[0175] The CPU 1632 performs various processes by reading the program stored in the disk device 1634 into the memory 1633 and executing the program. The CPU 1632 sends and receives data with other devices (such as the application execution node 1620, the performance model generation node 1600, and the benchmark execution node 1610) connected to the LAN 170 via the bus 1631 and the NIC 1636.

[0176] The performance model generation node 1600 includes a CPU 1602 as an example of a processor, a memory 1603 as a main storage device, a disk device 1604 as an auxiliary storage device, and a NIC 1605. The CPU 1602, the memory 1603, the disk device 1604, and the NIC 1605 are connected via a bus 1601.

[0177] The CPU 1602 performs various processes by reading programs stored in the disk device 1604 into the memory 1603 and executing the programs. The CPU 2602 sends and receives data to and from other devices (such as the application execution nodes 1620 and 1630 and the benchmark execution node 1610 ) connected to the LAN 170 via the bus 1601 and the NIC 1605 .

[0178] The benchmark execution node 1610 includes a CPU 1612 as an example of a processor, a memory 1613 as a main storage device, a disk device 1614 as an auxiliary storage device, and a NIC 1615. The CPU 1612, the memory 1613, the disk device 1614, and the NIC 1615 are connected via a bus 1611.

[0179] The CPU 1612 performs various processes by reading programs stored in the disk device 1614 into the memory 1613 and executing the programs. The CPU 1612 transmits and receives data with other devices (such as the application execution nodes 1620 and 1630 and the performance model generation node 1600 ) connected to the LAN 170 via the bus 1611 and the NIC 1615 .

[0180] Fig.18 This is a logical configuration diagram of components related to the performance model generation process of the second embodiment.

[0181] The application execution node 1620 stores a program 1741 for application (program A), a program 1751 for application (program B), and a resource allocation control program 1720 .

[0182] The application execution node 1630 stores a program 1742 for application (program A), a program 1752 for application (program B), and a resource allocation control program 1730 .

[0183] The program 1741 executed by the CPU 1622 of the application execution node 1620 and the program 1742 executed by the CPU 1632 of the application execution node 1630 work in cooperation to form the application program 1740 (application A). In addition, the program 1751 executed by the CPU 1622 of the application execution node 1620 and the program 1752 executed by the CPU 1632 of the application execution node 1630 work in cooperation to form the application program 1750 (application B). In addition, in this figure, an example in which the programs of multiple application execution nodes work in cooperation is shown for the application, but the program of each application execution node may also constitute an application independently.

[0184] When executed by the CPU 1622 , the resource allocation control program 1720 controls the amount of HW resources (HW resource allocation amount) allocated to each program executed in the application execution node 1620 .

[0185] When the resource allocation control program 1730 is executed by the CPU 1632 , it controls the amount of HW resources allocated to each program executed in the storage node 1630 .

[0186] The performance model generation node 1600 stores a performance model generation program 1700 . When the performance model generation program 1700 is executed by the CPU 1602 , the performance models of the applications 1740 and 1750 configured in the application execution nodes 1620 and 1630 are generated.

[0187] The benchmark execution node 1610 stores a benchmark program 1710. When the benchmark program 1710 is executed by the CPU 1612, the performance of the applications 1740 and 1750 configured in the application execution nodes 1620 and 1630 is measured.

[0188] Next, the performance model generation process 1800 for generating a performance model will be described.

[0189] Fig.19 This is a flowchart of the performance model generation process according to the second embodiment.

[0190] In the performance model generation process of this embodiment, a performance model is generated for each HW resource. Here, HW resources are CPU, memory, NIC bandwidth, IO bandwidth of disk device, etc. Through this performance model generation process, a performance model is generated for each HW resource, which shows the relationship between the allocation amount of each HW resource and the performance of the application (strictly speaking, the program for the application) at the time of the allocation amount.

[0191] First, the performance model generation program 1700 of the performance model generation node 1600 (strictly speaking, the CPU 1602 that runs the performance model generation program 1700) checks whether the performance models for all HW resources have been generated (step 1810). If the performance models have been generated for all HW resources (step 1810: Yes), the performance model generation program 1700 ends the performance model generation process 1800.

[0192] On the other hand, when the performance models for all HW resources have not been generated (step 1810: No), the performance model generation program 1700 generates the performance model for the HW resources that have not been generated. Here, the HW resources that are the targets are referred to as target HW resources. In addition, when the performance model of the target HW resources is generated, the HW resource allocation amount of the target HW resources is set to a level that does not become a bottleneck for the performance of the target HW resources, and the HW resource allocation amount of the target HW resources is gradually changed, thereby generating the performance model.

[0193] First, the performance model generation program 1700 sets the amount of HW resources allocated to the application (step 1820). The setting of the amount of HW resource allocation in the application is performed by cooperating with the resource allocation control program of each application running node. Specifically, the performance model generation program 1700 sends the amount of HW resource allocation for the programs 1741, 1742, 1751, and 1752 of the application to the resource allocation control programs 1720 and 1730 of each application running node 1620 and 1630. As a result, the resource allocation control programs 1720 and 1730 receive the amount of HW resource allocation and allocate the HW resources of the received amount of HW resource allocation to the program. The allocation of HW resources in the application running node can be implemented by existing programs and software. For example, if it is a Linux operating system, a resource allocation function called Cgroups can be used. When using this Cgroups, the required amount of HW resources can be allocated to the program running in the Linux operating system.

[0194] Next, the performance model generation program 1700 instructs the benchmark program 1710 of the benchmark execution node 1610 to perform performance measurement on the target HW resource (step 1830). As a result, the benchmark program 1710 performs performance measurement of the application (specifically, the program for the application) and sends the performance measurement result to the performance model generation program 1700.

[0195] Next, the performance model generation program 1700 determines whether the performance model can be generated, and more specifically, determines whether the performance measurement has been performed a number of times required for generating the performance model (step 1840 ).

[0196] As a result, if the number of measurements required for generating the performance model has not been performed (step 1840: No), the performance model generating program 1700 advances the process to step 1820 to repeat the change of the HW resource allocation amount and the execution of the performance measurement. In addition, the number of times the performance measurement is performed to generate the performance model and the HW resource allocation amount changed for each performance measurement are predetermined.

[0197] On the other hand, when the number of measurements required for generating a performance model has been performed (step 1840: Yes), the performance model generating program 1700 generates a performance model based on the plurality of measurement results, and registers the generated performance model in the performance model management table 1910 (reference Fig.21 ) (step 1850), causing the processing to proceed to step 1810.

[0198] Here, generation of a performance model and registration of a performance model are described.

[0199] Fig. 20 This is a diagram showing an overview of the performance mode of the second embodiment.

[0200] The performance model can be generated, for example, as Fig. 20 As shown, a graph of performance changes relative to the amount of HW resources is generated, and the formula y=f(x) of the approximate curve of the graph is used as a performance model. Here, y represents the performance of an application of one node, and x represents the amount of HW resources of one node. y can be calculated by dividing the result of the performance measurement from the benchmark program 1710 (the overall performance of the application) by the number of nodes of the application. In other words, when y is multiplied by the number of nodes of the application, the overall performance of the application is obtained. The generation of the graph and the derivation of the approximate curve formula in the generation of the performance model can be achieved by using existing spreadsheet software and programs.

[0201] Fig.21 It is a diagram showing the structure of the performance model management table according to the second embodiment.

[0202] The registration of the performance model is performed, for example, in the performance model management table 1910. Row 1911 of the performance model management table 1910 stores information (application name) indicating the application corresponding to the performance model registered in the performance model management table 1910. Column 1912 stores the type of HW resource corresponding to the performance model. Row 1913 stores the performance model of each HW resource for the operating performance. In the cell of row 1913, the performance model of the type of the HW resource is stored by the performance model generation program 1700.

[0203] In addition, in the present embodiment, the equation (performance model) generated based on the performance measurement result is stored, but for example, a plurality of sets of HW resource amounts and measured performance may be recorded.

[0204] Next, the resource allocation setting process will be described.

[0205] Fig. 22 This is a hardware configuration diagram of components related to resource allocation setting processing according to the second embodiment.

[0206] In the present embodiment, the resource allocation setting process is mainly participated in by the application execution nodes 1620 and 1630 , the client nodes 2040 and 2050 , the management node 2000 , and the management client node 2010 among the components of the computer system.

[0207] The CPU 1622 of the application execution node 1620 sends and receives data with other devices (the application execution node 1630, the client nodes 2040, 2050, etc.) connected to the LAN 171 via the bus 1621 and the NIC 1625. In addition, the CPU 1622 of the application execution node 1620 sends and receives data with other devices (the management node 2000, etc.) connected to the LAN 170 via the bus 1621 and the NIC 1626.

[0208] The CPU 1632 of the application execution node 1630 sends and receives data with other devices (the application execution node 1630, the client nodes 2040, 2050, etc.) connected to the LAN 171 via the bus 1631 and the NIC 1635. In addition, the CPU 1632 of the application execution node 1630 sends and receives data with other devices (the management node 2000, etc.) connected to the LAN 170 via the bus 1631 and the NIC 1636.

[0209] The client nodes 2040 and 2050 perform IO for the applications configured in the application execution nodes 1620 and 1630. Each of the client nodes 2040 and 2050 includes a CPU, a memory, a NIC, and a disk device which are not shown in the figure and are connected via a bus. The CPU calls the program stored in the disk device into the memory and executes it. The client nodes 2040 and 2050 send and receive data with other devices (such as the application execution nodes 1620 and 1630) via the LAN 171.

[0210] The management client node 2010 has a CPU, a memory, a NIC, and a disk device connected via a bus (not shown). The CPU calls the program stored in the disk device into the memory and executes it. The management client node 2010 sends and receives data with other devices (such as the management server 2000) via the LAN 170.

[0211] The management node 2000 includes a CPU 2002 as an example of a processor, a memory 2003 as a primary storage device, a disk device 2004 as an auxiliary storage device, and a NIC 2005. The CPU 2002, the memory 2003, the disk device 2004, and the NIC 2005 are connected via a bus 2001.

[0212] The CPU 2002 performs various processes by reading programs stored in the disk device 2004 into the memory 2003 and executing the programs. The CPU 2002 transmits and receives data with other devices (application execution nodes 1620 and 1630, management client node 2010, etc.) connected to the LAN 170 via the bus 2001 and the NIC 2005.

[0213] Fig.23 is a logical structure diagram of components related to resource allocation setting processing in the second embodiment. Fig.18 The same components as those shown are denoted by the same reference numerals, and duplicate descriptions are omitted.

[0214] The application execution node 1620 stores a program 1741 (program A), a program 1751 (program B), a QoS control program 2120 , and a resource allocation control program 1720 .

[0215] The application execution node 1630 stores a program 1742 (program A), a program 1752 (program B), a QoS control program 2130 , and a resource allocation control program 1730 .

[0216] The QoS control program 2120 of the application execution node 1620 includes a performance monitoring unit 2121 and a resource monitoring unit 2122 , and when executed by the CPU 1622 , monitors the performance and resource consumption of the programs 1741 and 1751 .

[0217] The QoS control program 2130 of the application execution node 1630 includes a performance monitoring unit 2131 and a resource monitoring unit 2132 , and when executed by the CPU 1632 , monitors the performance and resource consumption of the programs 1742 and 1752 .

[0218] The client node 2040 stores the client application 2160. The client application 2160 is executed by the client node 2040 to send a processing request to the application configured in the application execution nodes 1620 and 1630.

[0219] The client node 2050 stores the client application 2170. The client application 2170 is executed by the client node 2050 to send a processing request to the application configured in the application execution nodes 1620 and 1630.

[0220] The management client node 2010 stores a web browser 2110. The web browser 2110 receives input of target performance settings of an application from a manager by being executed by the management client node 2010, and transmits the target settings to the management node 2000.

[0221] The management node 2000 stores a QoS control program 2100. The QoS control program 2100 includes a resource allocation setting unit 2101, a performance model management unit 2102, a setting execution unit 2103, a performance model management table 1910, and a resource allocation setting table 2105. Although omitted in the figure, the QoS control program 2100 includes performance model management tables 1910 corresponding to the number of application programs executed in the application execution node.

[0222] The resource allocation setting unit 2101 is executed by the CPU 2002 of the management node 2000 to receive the target performance of the application program input via the Web browser 2110, and based on the received target performance and the contents of the performance model management table 1910, updates the resource allocation setting table 2105, and sends the setting value of the resource allocation amount to each application running node 1620, 1630.

[0223] The performance model management unit 2102 receives the monitoring results of the performance and resource consumption of the programs 1741, 1742, 1751, 1752 from the QoS control programs 2120, 2130 executed in the application execution nodes 1620, 1630 by being executed by the CPU 2002, and modifies the performance model based on the monitoring results. In addition, the performance model management unit 2102 updates the resource allocation setting table 2105 based on the modified performance model, and sends the setting value of the resource allocation amount to each application execution node 1620, 1630.

[0224] Next, the resource allocation setting process 2200 for setting the allocation of HW resources to the application will be described.

[0225] Fig.24 This is a flowchart of the resource allocation setting process according to the second embodiment.

[0226] The Web browser 2110 of the management client node 210 displays a target performance input GUI screen 2300 (see Fig.25 ) and receive the target performance of each application from the application manager (step 2210).

[0227] Here, the target performance input GUI screen 2300 will be described.

[0228] Fig.25 It is a diagram showing a target performance input GUI screen according to the second embodiment.

[0229] The target performance input GUI screen 2300 has input fields (input fields 2301, 2302) for the target performance of each application and a send button 2303. Fig.25 The target performance input GUI screen 2300 shown shows input fields for the case of having two applications, application A and application B, but input fields corresponding to the number of applications configured in the application execution node are prepared. Input fields 2301 and 2302 are input fields for inputting target performance for the application. The administrator can input the target performance of each application in the input fields. The send button 2303 is a button for receiving an instruction to send the target performance settings entered in the input fields 2301 and 2302 to the application execution node. When the send button 2303 is pressed, the management client node 2010 sends the target performance settings entered in the input fields 2301 and 2302 to the management node 2000.

[0230] Next, the resource allocation setting unit 2101 of the management node 2000 receives the transmitted target performance setting (step 2220). The resource allocation setting unit 2101 registers the received target performance in the resource allocation setting table 2105 (see Fig.26 )middle.

[0231] Fig.26 It is a diagram showing the structure of a resource allocation setting table according to the second embodiment.

[0232] The resource allocation setting table 2105 is set for each application configured in the application execution node, and manages the relationship between the target performance of the application and the required HW resource amount. Row 2401 of the resource allocation setting table 2105 stores information (application name) indicating the application corresponding to the resource allocation setting table 2105. Row 2402 stores the target performance of the application. Row 2403 stores the resource amount of each HW resource required for the target performance.

[0233] return Fig.24 Next, the resource allocation setting unit 2101 calculates the amount of HW resources actually allocated to the application (resource allocation amount) by referring to the performance model management table 1910. Here, the calculated amount of HW resources is the amount of HW resources allocated to the application program in each application execution node.

[0234] First, the resource allocation setting unit 2101 calculates the amount of HW resources required to achieve the target performance based on the performance model (formula) recorded in the performance model management table 1910 and the target performance of the application, and registers the calculated amount of HW resources as the resource allocation amount in row 2403 of the resource allocation setting table 2105 (step 2221). Specifically, the resource allocation setting unit 2101 divides the target performance of the application by the number of application execution nodes constituting the application, substitutes the result into y of the performance model formula (y=f(x)), and calculates the required amount of HW resources x. In addition, when recording a plurality of sets of HW resource amounts and measured performance instead of the performance model, the amount of HW resources required for the target performance may be calculated based on the plurality of sets.

[0235] Next, the resource allocation setting unit 2101 transmits the content of the column 2403 of the resource allocation setting table 2105 , that is, the resource allocation amount for each HW resource, to the resource allocation control programs 1720 and 1730 of each application execution node 1620 and 1630 (step 2222 ).

[0236] The resource allocation control programs 1720 and 1730 of the application execution nodes 1620 and 1630 receive the content of row 2403 of the resource allocation setting table 2105 (step 2230 ), and set the HW resources allocated to the application program based on the content of row 2403 , i.e., the resource allocation amount for each HW resource (step 2231 ).

[0237] After the allocation setting is completed, the resource allocation control programs 1720 and 1730 transmit a notification indicating that the setting is completed (setting completion notification) to the resource allocation setting unit 2101 of the management node 2000 (step 2232 ).

[0238] The resource allocation setting unit 2101 of the management node 2000 receives the setting completion notification (step 2223), and checks whether the setting completion notification has been received from all the application execution nodes that have sent the resource allocation amount for each HW resource (step 2224). If the resource allocation setting unit 2101 has not received the setting completion communication from all the application execution nodes (step 2224: No), the processing proceeds to step 2223. On the other hand, if the setting completion communication has been received from all the application execution nodes (step 2224: Yes), the resource allocation setting processing 2200 ends.

[0239] Next, a description will be given of a performance model modification / setting update process 2500 for modifying a performance model and reconfiguring resource allocation based on the modified performance model.

[0240] Fig. 27 This is a flowchart of the performance model correction / setting update process according to the second embodiment.

[0241] The performance model correction / setting update process is performed, for example, periodically. First, the performance monitoring units 2121, 2131 of the QoS control programs 2120, 2130 of each application running node 1620, 1630 obtain the running log of the program of each application (step 2510). Here, in the running log, the content of the operation performed by the program 1741, 1742, 1751, 1752, the timing of the operation, the processing time, etc. are recorded. Next, the resource monitoring units 2122, 2132 obtain the log (resource consumption log) of the amount of HW resources consumed by the program 1741, 1742, 1751, 1752 of each application (step 2511). The resource consumption log can be obtained using the function of the OS (operating system), etc.

[0242] Next, the QoS control programs 2120 and 2130 send the acquired operation logs and resource consumption logs to the performance model management unit 2102 of the management node 2000 (step 2512).

[0243] The performance model management unit 2102 of the management node 2000 receives the operation log and resource consumption log sent from each application execution node 1620 and 1630 (step 2520 ).

[0244] Next, the performance model management unit 2102 obtains the actual performance of the application based on the operation log, compares the actual performance (actual performance) with the performance (estimated performance) estimated based on the performance model, and determines whether the difference between the actual performance and the estimated performance exceeds a predetermined threshold value (step 2521). As a result, if the difference between the actual performance and the estimated performance does not exceed the predetermined threshold value (step 2521: No), the performance model management unit 2102 advances the process to step 2520. On the other hand, if the difference between the actual performance and the estimated performance exceeds the predetermined threshold value (step 2521: Yes), the performance model management unit 2102 performs a process of correcting the performance model, and updates the performance model of the performance model management table 1910 to the corrected performance model (step 2522). The process of correcting the performance model can be implemented by the same process as the performance model generation process 1800 based on the acquired resource consumption log and the performance value acquired based on the operation log.

[0245] Next, the performance model management unit 2102 calculates new resource allocation amounts based on the updated performance model in the performance model management table 1910 and updates the resource allocation setting table 2105 (step 2523). The process of calculating new resource allocation amounts can be implemented in the same way as the resource allocation setting process 2200.

[0246] Next, the performance model management unit 2102 sends the content of row 2403 of the resource allocation setting table 2105 , that is, the resource allocation amount for each HW resource, to the resource allocation control programs 1720 and 1730 of each application execution node 1620 and 1630 (step 2524 ).

[0247] The resource allocation control programs 1720 and 1730 of the application execution nodes 1620 and 1630 receive the content of row 2403 of the resource allocation setting table 2105 (step 2513 ), and set the HW resources to be allocated to the application program based on the resource allocation amount for each HW resource in row 2403 (step 2514 ).

[0248] After the allocation setting is completed, the resource allocation control programs 1720 and 1730 send a notification indicating that the setting is completed (setting completion notification) to the performance model management unit 2102 of the management node 2000 (step 2515).

[0249] The performance model management unit 2102 of the management node 2000 receives the setting completion notification (step 2525), and checks whether the setting completion notification has been received from all the application execution nodes that have sent the resource allocation amount for each HW resource (step 2526). If the performance model management unit 2102 has not received the setting completion communication from all the application execution nodes (step 2526: No), the processing proceeds to step 2525. On the other hand, if the setting completion communication has been received from all the application execution nodes (step 2526: Yes), the performance model correction / setting update processing 2500 ends.

[0250] Through the performance model correction / setting update process 2500 , it is possible to update to an appropriate performance model according to the actual status of the processing of each application execution node 1620 , 1630 , and to appropriately allocate HW resources according to the actual status.

[0251] In addition, the present invention is not limited to the above-described embodiment, and can be implemented with appropriate modifications within the scope not departing from the gist of the present invention.

[0252] For example, in the above embodiment, a mathematical formula is used as the performance model, but the present invention is not limited to this. For example, an inference model obtained by machine learning with performance as input and required HW resource amount as output may be used.

[0253] In addition, in the above-mentioned embodiment, a part or all of the processing performed by the CPU can also be performed by a hardware circuit. In addition, the program in the above-mentioned embodiment can be installed from a program source. The program source can be a program distribution server or a recording medium (e.g., a removable non-volatile recording medium).

[0254] Description of Reference Numerals

[0255] 1…computer system, 100…management node, 111…performance model, 120, 130…storage nodes, 140, 141…distributed data storage, 150, 151…client nodes, 160…management client node, 200…performance model generation node, 210…benchmark execution node, 170, 171…LAN.

Claims

1. A resource allocation control device for controlling the amount of hardware resources of a predetermined node to be allocated to software running in the node, characterized in that: include: a storage unit storing a plurality of performance resource amount information, wherein each of the performance resource amount information indicates, for each of a plurality of different operations of the software, a correspondence between the resource amount of the hardware for each of a plurality of hardware resources allocated to each operation and the performance of each operation when the hardware to which the resource amount is allocated; and a processor connected to the storage unit, the processor, accepting the target performance of each of said operations of said software, For each of the hardware resources, the resource amount of the hardware required to implement the different operations is calculated based on each of the plurality of performance resource amount information. For each of the hardware resources used for the plurality of operations of the software, the amount of resources to be allocated to all the operations is determined based on the maximum amount of resources among the calculated amounts of hardware resources required to implement each of the operations, Setting is performed so that the determined resource amount is allocated to the software.

2. The resource allocation control device according to claim 1, characterized in that: the processor, The resource amount of the hardware allocated to the software is changed into a plurality of resource amounts to obtain performance that can be achieved when each of the plurality of resource amounts is allocated, generating the performance resource amount information based on the allocated hardware resource amount and the achievable performance, The generated performance resource amount information is stored in the storage unit.

3. The resource allocation control device according to claim 1, characterized in that: the processor, acquiring, from the node, the performance of the software when the hardware to which the determined amount of resources is allocated is executed; updating the performance resource quantity information based on the acquired performance, Determine the hardware resource amount required to achieve the target performance based on the updated performance resource amount information, The hardware is set so that the software allocates the determined amount of resources.

4. The resource allocation control device according to claim 1, characterized in that: The software is software that is executed in each of a plurality of nodes and constitutes distributed data storage that manages data in a distributed manner in the plurality of nodes.

5. The resource allocation control device according to claim 4, characterized in that: The operations of the software include IO operations.

6. A computer system comprising one or more execution nodes for executing software and a resource allocation control device for controlling the amount of hardware resources of the execution nodes to be allocated to the software, characterized in that: The resource allocation control device, storing a plurality of performance resource amount information, wherein each of the performance resource amount information represents, for each of a plurality of different operations of the software, a correspondence between the resource amount of the hardware for each of a plurality of hardware resources allocated to each operation and the performance of each operation when the hardware to which the resource amount is allocated is used, accepting the target performance of each of said operations of said software, For each of the hardware resources, the resource amount of the hardware required to implement the different operations is calculated based on each of the plurality of performance resource amount information. For each of the hardware resources used for the plurality of operations of the software, the amount of resources to be allocated to all the operations is determined based on the maximum amount of resources among the calculated amounts of hardware resources required to implement each of the operations, The execution node is configured to allocate the determined amount of resources to the software.

7. The computer system according to claim 6, wherein: Also includes a generation node that generates the performance resource quantity information, The generating node, The resource amount of the hardware allocated to the software is changed into a plurality of resource amounts to obtain performance that can be achieved when each of the plurality of resource amounts is allocated, generating the performance resource quantity information based on the allocated hardware resource quantity and the acquired performance, The generated performance resource quantity information is sent to the resource allocation control device.

8. A resource allocation control method, executed by a resource allocation control device for controlling the amount of hardware resources of a specified node to be allocated to software running in the specified node, characterized in that: A plurality of performance resource amount information is stored in the storage unit, wherein each of the performance resource amount information represents, for each of a plurality of different operations of the software, a correspondence between the resource amount of the hardware of each of a plurality of hardware resources allocated to each operation and the performance of each operation when the hardware to which the resource amount is allocated is used, accepting the target performance of each of said operations of said software, For each of the hardware resources, the resource amount of the hardware required to implement the different operations is calculated based on each of the plurality of performance resource amount information. For each of the hardware resources used for the plurality of operations of the software, the amount of resources to be allocated to all the operations is determined based on the maximum amount of resources among the calculated amounts of hardware resources required to implement each of the operations, Setting is performed so that the determined resource amount is allocated to the software.

Images