Method of collecting performance test data of virtual machine instance, and performance test data collection device performing method
The method and device address the lack of standardized performance metrics in public clouds by automating VM instance testing and comparison, ensuring optimal VM instance creation and reducing performance-related losses.
Patent Information
- Application Number
- KR1020200174298
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-12-14
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2040-12-14
AI Technical Summary
Public clouds lack standardized analytical metrics for performance data and actual workloads, forcing users to rely on Cloud Service Providers' performance specifications when selecting VM instances, which can lead to suboptimal choices.
A method and device for collecting performance test data of a VM instance by receiving a performance test request, analyzing it, testing the VM instance's performance, and storing the results in memory or a database, with the option to compare against average performance and send regeneration recommendations.
Enables efficient collection and comparison of VM instance performance data, preventing material and human losses due to performance deviations, and optimizing VM instance creation through automated testing and feedback.
Smart Images

Figure 112020135365380-PAT00003_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a method for collecting performance test data of a VM instance and a device for collecting performance test data that performs the method. Background Technology
[0002] The cloud is a computing system that stores data on a central computer connected to a communication network, such as the internet, allowing users to access the data anytime and anywhere simply by connecting to the network.
[0003] Among the aforementioned cloud computing systems, a public cloud refers to a public infrastructure system that does not restrict service users, allowing anyone to connect to the network and access services solely through payment, with a structure where users pay only for what they use.
[0004] However, public clouds lack standardized analytical metrics that account for performance data and actual workloads regarding the services to be used, which creates a problem where users are forced to rely on performance specification information provided by Cloud Service Providers (CSPs) when selecting products with suitable performance.
[0005] Therefore, when a user creates a VM instance, it is necessary to propose a method to test the performance of the VM instance created by the user and provide it to the user. The problem to be solved
[0006] The problem that the present invention aims to solve is to provide a method for receiving a performance test request for a VM instance created from a user terminal, testing the performance of the VM instance corresponding to the received request, and storing the test result in memory as the result of the test, in order to solve the aforementioned problem.
[0007] However, the problems that the present invention aims to solve are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the description below. means of solving the problem
[0008] A method for collecting the performance of a VM instance created by a user terminal, performed by a performance test data collection device according to an embodiment of the present invention, may include: receiving a performance test request for the VM instance created by the user terminal from the user terminal; analyzing the performance test request to generate information necessary for testing the performance of the VM instance; testing the performance of the VM instance using the information necessary for testing the performance of the VM instance; and storing the results of the test in a memory included in the performance test data collection device or in an external database.
[0009] The above method may further include the step of comparing the result of the above test with the average performance of VM instances having the same environment and specifications as the VM instance, which are stored in the memory or the database; and the step of sending an alarm recommending the regeneration of the VM instance to the user terminal based on the result of the comparison.
[0010] The step of sending an alarm recommending the regeneration of the VM instance may include sending an alarm recommending the regeneration of the VM instance to the user terminal when the result of the test is lower than the average performance of VM instances having the same environment and specifications as the VM instance.
[0011] A performance test data collection device for collecting the performance of a VM instance created by a user terminal according to another embodiment of the present invention may include: a memory; a transceiver that receives a performance test request for the VM instance created by the user terminal from the user terminal; and a processor that analyzes the performance test request to generate information necessary for testing the performance of the VM instance, tests the performance of the VM instance using the information necessary for testing the performance of the VM instance, and stores the results of the test in the memory or an external database.
[0012] A computer-readable recording medium storing a computer program according to another embodiment of the present invention may include a step of receiving a performance test request for a VM instance created by the user terminal from the user terminal; a step of analyzing the performance test request to generate information necessary for testing the performance of the VM instance; a step of testing the performance of the VM instance using the information necessary for testing the performance of the VM instance; and a step of storing the result of the test in a memory included in the performance test data collection device or in an external database, and may include instructions for a processor to perform a method of collecting the performance of a VM instance created by the user terminal, which is performed by the performance test data collection device. Effects of the invention
[0013] According to an embodiment of the present invention, the performance of a VM instance can be easily collected by automatically saving the results of a performance test as the user terminal creates a VM instance.
[0014] In addition, according to an embodiment of the present invention, by comparing the performance test results of a VM instance created by a user terminal with the average performance of a VM instance having the same environment and specifications collected previously, material and human losses caused by performance deviations of VM instances depending on the creation environment can be prevented. Brief explanation of the drawing
[0015] FIG. 1 is a block diagram showing a performance test data collection system according to one embodiment of the present invention. FIG. 2 is a block diagram showing the detailed configuration of a performance test data collection device according to one embodiment of the present invention. FIG. 3 is a block diagram conceptually illustrating the functions of a performance test data collection program according to one embodiment of the present invention. FIG. 4 is a flowchart illustrating a method for a performance test data collection device to collect data that tests the performance of a VM instance according to an embodiment of the present invention. Specific details for implementing the invention
[0016] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.
[0017] In describing the embodiments of the present invention, specific descriptions of known functions or configurations will be omitted if it is determined that such detailed descriptions could unnecessarily obscure the essence of the invention. Furthermore, the terms described below are defined considering their functions in the embodiments of the present invention, and these definitions may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification.
[0018] FIG. 1 is a block diagram showing a performance test data collection system according to one embodiment of the present invention, and FIG. 2 is a block diagram showing a detailed configuration of a performance test data collection device according to one embodiment of the present invention.
[0019] Referring to FIG. 1, the performance test data collection system (10) may include a user terminal (20) and a performance test data collection device (100).
[0020] The user terminal (20) can create a VM instance using a pre-generated VM template image. Here, the VM template image may be an image that has been created and distributed by an operator of a performance test data collection device (100), an operator of a public cloud, etc., and may be an image configured to create a VM instance having specific functions and performance.
[0021] When the above VM instance is created, the user terminal (20) can send a performance test request for the created VM instance to the performance test data collection device (100).
[0022] According to an embodiment, the performance test request can be transmitted to a performance test data collection device (100) without control by a user using the user terminal (20) (i.e., automatically upon creation of a VM instance).
[0023] The user terminal (20) can receive the performance of the created VM instance (or the performance of the created VM instance and a regeneration recommendation alarm) from the performance test data collection device (100) in response to the performance test request.
[0024] The performance test data collection device (100) can receive a performance test request for a VM instance created from a user terminal (20).
[0025] To this end, the user terminal (20) may include a transceiver that transmits a performance test request to a performance test data collection device (100) and receives the performance of a generated VM instance (or the performance of a generated VM instance and a regeneration recommendation alarm) from the performance test data collection device (100), and a processor that generates a VM instance from a VM template image.
[0026] A performance test data collection device (100) receives a performance test request for a VM instance created by a user terminal (20) from a user terminal (20), tests the performance of the VM instance corresponding to the received request, stores the test result in memory (130) or an external database (not shown) as a result of the test, and transmits the performance of the created VM instance (or the performance of the created VM instance and a regeneration recommendation alarm) to the user terminal (20).
[0027] Referring further to FIG. 2, for this purpose, the performance test data collection device (100) may include a processor (110), a transceiver (120), and a memory (130).
[0028] The processor (110) can control the overall function of the performance test data collection device (100).
[0029] The transceiver (120) receives a request for a performance test for a VM instance created by the user terminal (20) from the user terminal (20), and can transmit the performance of the created VM instance (or the performance of the created VM instance and a regeneration recommendation alarm) as a result of the performance test to the user terminal (20).
[0030] In this specification, for convenience of explanation, it is described that a transceiver (120) receives a performance test request for a VM instance created by a user terminal (20) and transmits the performance of the created VM instance (or the performance of the created VM instance and a regeneration recommendation alarm) to the user terminal (20) as a result of the performance test, but is not limited thereto. That is, according to an embodiment, the performance test data collection device (100) may include an input / output device (not shown), and the performance test data collection device (100) may receive a performance test request for a VM instance created by a user terminal (20) using the input / output device (not shown) and output the performance of the created VM instance (or the performance of the created VM instance and a regeneration recommendation alarm) as a result of the performance test.
[0031] The memory (130) may store information necessary for the execution of the performance test data collection program (200) and the performance test data collection program (200).
[0032] In the present specification, the performance test data collection program (200) may refer to software that includes instructions programmed to test the performance of a VM instance in accordance with a performance test request received from a user terminal (20).
[0033] The processor (110) can load the performance test data collection program (200) and information necessary for the execution of the performance test data collection program (200) from memory (130) in order to execute the performance test data collection program (200).
[0034] The processor (110) can execute a performance test data collection program (200) to test the performance of a VM instance created by a user terminal (20) in accordance with a performance test request received from a user terminal (20), and as a test result, transmit the performance of the created VM instance (or the performance of the created VM instance and a regeneration recommendation alarm) to the user terminal (20).
[0035] The function and / or operation of the performance test data collection program (200) will be examined in detail through FIG. 3.
[0036] Additionally, information regarding the performance of a VM instance can be stored in the memory (130) as a test result of the VM instance. That is, when a performance test request is received from a user terminal (20), the processor (110) can test the performance of the VM instance that received the performance test request and store the test result in the memory (130).
[0037] Meanwhile, according to the embodiment, the test results may be stored in an external database (DB) (not shown) rather than in memory (130).
[0038] FIG. 3 is a block diagram conceptually illustrating the functions of a performance test data collection program according to one embodiment of the present invention.
[0039] Referring to FIG. 3, the performance test data collection program (200) may include a receiving unit (210), a performance test unit (220), and a test result processing unit (230).
[0040] The receiving unit (210), performance test unit (220), and test result processing unit (230) illustrated in FIG. 3 are conceptual divisions of the functions of the performance test data collection program (200) to easily explain the functions of the performance test data collection program (200), but are not limited thereto. According to embodiments, the functions of the receiving unit (210), performance test unit (220), and test result processing unit (230) may be merged or separated and may be implemented as a series of instructions included in one or more programs.
[0041] The receiving unit (210) can receive a performance test request for a VM instance created by the user terminal (20) from the user terminal (20).
[0042] Additionally, the receiving unit (210) can generate information necessary to test the performance of the VM instance for which a performance test has been requested.
[0043] A request for a performance test for a VM instance is automatically transmitted to a performance test data collection device (100) as the user terminal (20) creates a VM instance, and the information required to test the performance of the VM instance may not be predetermined information, but information determined by the creation of the VM instance.
[0044] Accordingly, the receiving unit (210) can analyze a performance test request received from a user terminal (20) and generate information necessary to test the performance of a VM instance.
[0045] Here, the information required to test the performance of a VM instance includes the IP address of the VM instance, and if the VM instance is a DB, the information required to test the performance of the VM instance may include the DB connection account, IP address, etc.
[0046] The performance test unit (220) can test the performance of a VM instance using information necessary to test the performance of a VM instance generated by the receiving unit (210).
[0047] More specifically, the performance test unit (220) can test the performance of the VM instance created by the user terminal (20) through the following process.
[0048] First, the performance test unit (220) can calculate the size of resources required for the public cloud based on the required resource information for each predefined test scenario. Here, the predefined test scenario may include a test scenario for at least one of a load balancer test, a file storage test, an object storage test, an application test, a server test, and a database test.
[0049] Here, the resource size required for the test target cloud calculated by the processing unit (130) is shown in Table 1 below as an example.
[0050] Test scenario By role Required resource quantity Basis for calculation Load Balancer 1 Controller An instance for the master for nGrinder process control 6 Members Servers for nGrinder load generation, 6 servers allocated for maximum and stable load generation ※ Target load: 100,000 TPS (Transactions Per Second) or higher 10 Targets Estimated 10 servers for maximum and stable load handling ※ Target load: 100,000 TPS or more File Storage 1 Controller Instance for master for vdbench process control 9 Members Instances for generating Disk IO load; 9 servers allocated for maximum or stable IO load generation. ※ Load conditions: Create / delete up to 1.28 million files (1MB), Block size 4KB, total 240 Threads 1 Target One File Storage product from the test target CSP Object Storage 1 Controller Instance for master for cosbench process control 3 Members 3 servers allocated for maximum and stable load generation ※ Load conditions: Object File (1 MB), Worker (1,000 ea), GET / PUT ratio adjustment (100:0 / 70:30 / 0:100) 1 Target One Object Storage product from the test target CSP Application 1 Controller, An instance for the master for nGrinder process control 2 Members Server for nGrinder load generation, 2 servers allocated for maximum and stable load generation※ Target load: vUser 6000 [process 30 * thread 100 * agent 2], Network Bandwidth 10Gbps or higher※ vUser: nGrinder load unit 1 Target Create one resource with the specifications selected by the user Server 1 Create one resource with the specifications selected by the user Instance / PaaS DB 1 Create one resource with the specifications selected by the user
[0051] The performance test unit (220) can classify VM instances based on the estimated resource size and predefined test types. Here, the predefined test types may include at least one of instance type, storage type, and application type.
[0052] For example, the performance test unit (220) can classify a VM instance that is either an instance server or an instance database as an instance type, classify a VM instance that is either file storage or object storage as a storage type, and classify a VM instance that is either a load balancer, a PaaS DB, or an application as an application type.
[0053] Subsequently, the performance test unit (220) can transmit information regarding the estimated resource size to a cloud server (not shown) and request the creation of cloud resources corresponding to the required resource size. Here, the information regarding the estimated resource size may include at least one of CSP (Cloud Service Provider) information, cloud region information, network information, resource specification information, resource quantity information, and storage space information.
[0054] The performance test unit (220) can determine whether the type of the VM instance is an instance type, and depending on the determination result, perform either a local test or a remote test on the VM instance and generate test result data.
[0055] According to an embodiment, the performance test unit (220) can perform a local test on the VM instance when the type of the VM instance is an instance type, and perform a remote test on the VM instance when the type of the VM instance is not an instance type (i.e., when it is one of a storage type and an application type).
[0056] The above local tests may include POPS computation, Cache Latency, Memory Bandwidth, DB QPS / Latency, Disk IOPS, etc., and a description of each local test is as follows.
[0057] - POPS Calculation: Refers to POPS (Prime Operations Per Second), a CPU performance metric. It measures CPU computation speed and serves as a measure of the feasibility of timely resource allocation when applications request CPU resources. It calculates the number of times decimals ranging from 1 to 1,800,000 are measured per second. The tool used for this is sysbench.
[0058] - Cache Latency: Compared to Main Memory, CPU Cache Memory has a smaller capacity but significantly superior performance. By measuring CPU Cache Memory latency, the degree of application performance improvement is predicted. Data transfer latency is measured by increasing the matrix size in increments of 512 up to 256 MB. The tool used is LMbench.
[0059] - Memory Bandwidth: Data in memory is transferred to the CPU for actual processing, and the bandwidth of the bus channel used for this is related to latency. The bandwidth of the bus connecting memory and the CPU is measured to verify the expected speed of the application's actual data processing. The tool used is STREAM.
[0060] - DB QPS / Latency: DBMS performance is calculated as a combination of various factors such as Disk, CPU, and Memory. This item applies to DBMSs running in an instance environment. Among the final performance outputs of a DBMS, the performance of the DBMS is verified for each Read / Write case using QPS (queries processed per second) and Latency (response speed), which are the most intuitive performance indicators.
[0061] - Disk IOPS: Measures disk read / write performance and is used as an estimated disk performance indicator for each CSP / product. Since disk performance exhibits significant variability based on block size, sequential / random operation, and read / write performance, measuring performance for each case serves as the basis for making comprehensive judgments.
[0062] In addition, the above remote tests may include Remote Disk IOPS / Latency, TPS / MTT / Error, etc., and a description of each remote test is as follows.
[0063] - Remote Disk IOPS / Latency: Data processing speed is verified by measuring Throughput, which refers to the size of data transferred when an application attempts disk I / O on a storage product, and IOPS, which refers to Input / Output Operations Per Second. IOPS and Throughput have a direct impact on the application's data processing speed. The tools used are vdbench and cosbench.
[0064] - TPS / MTT / Error: Measures performance values that can be handled by assuming user requests at the application level. TPS, which is similar to a user's service request, and MTT (Mean Time to Stop), which is the response time directly experienced by the user, are selected as standard metrics. The tool used is nGrinder.
[0065] We will examine, by example, public cloud resources or specifications capable of performing performance tests in the performance test section (220) and the process of performing performance tests on them. For each cloud resource or specification, we examined the test purpose, test tool, test logic, and test items in that order.
[0066] 1. Server - CPU
[0067] 1) Purpose
[0068] - Adopted Prime Operations Per Second (POPS) metric, which is closely related to CPU processing speed
[0069] - Measures CPU processing speed and uses it as a measure of the feasibility of timely resource allocation when an application requests CPU resources.
[0070] 2) Measurement tools: sysbench, perf
[0071] 3) Measurement Logic
[0072] - Set to find prime numbers from 1 to 1,800,000 (selected as the standard because current commercial CPUs find prime numbers from 1 to 1,800,000 once per second)
[0073] - Measured for 10 seconds per vCPU
[0074] - Record the average excluding maximum and minimum values after 5 repeated tests
[0075] 4) Measurement items
[0076] - POPS (Prime Operation Per Second): Number of Prime Operations per second
[0077] - IPC (Instruction Per Cycle): Number of instructions processed per cycle
[0078] - Clock: CPU Clock
[0080] 2. Server - Memory
[0081] 1) Purpose
[0082] - Data in memory is transferred to the CPU for actual processing, and the bandwidth of the bus channel used at this time is related to latency.
[0083] - Measure the bandwidth of the bus connecting memory and CPU to verify the expected speed of the application's actual data processing.
[0084] 2) Measurement Tool: Triad Operation Item within STREAM
[0085] 3) Measurement Logic
[0086] - Bandwidth measurement per vCPU based on STREAM tool
[0087] - Record the average excluding maximum and minimum values after 5 repeated tests
[0088] 4) Measurement items
[0089] - Throughput (Bandwidth)
[0091] 3. Server - Cache Latency
[0092] 1) Purpose
[0093] - Compared to Main Memory, CPU Cache Memory has a smaller capacity but superior performance.
[0094] - Predict application performance improvement by measuring CPU cache memory latency
[0095] 2) Measurement tool: LMBench
[0096] 3) Measurement Logic
[0097] - Measure data transfer latency by increasing the matrix size in increments of 512 up to 256 MB using the LMBench tool.
[0098] - Record the average excluding maximum and minimum values after 5 repeated tests
[0099] 4) Measurement items
[0100] - Cache, Memory Latency
[0102] 4. Server - Disk
[0103] 1) Purpose
[0104] - Measure disk read / write performance to utilize as an estimated disk performance indicator for each CSP / product / Since disk performance varies significantly depending on block size, sequential / random operation, and read / write speeds, measuring performance for each case establishes the basis for comprehensive judgment.
[0105] - Select metrics by comparing application characteristics between types suitable for specific cases and general types.
[0106] 2) Measurement tool: sysbench
[0107] 3) Measurement Logic
[0108] - Measure the performance of the target disk using the sysbench tool
[0109] - Block Size: 8k, File Size: Read / Write measurements for a total of 300G divided into 128 parts
[0110] - Record the average excluding maximum and minimum values after 5 repeated tests
[0111] 4) Measurement items
[0112] - IOPS (Input / Output Operations Per Second), Throughput (MiB / s) of Sequential Read, Sequential Write, Sequential Read / Write items
[0113] -IOPS (Input / Output Operations Per Second), Throughput (MiB / s) of Random Read, Random Write, Random Read / Wirte items
[0115] 5. DBMS - IaaS / PaaS
[0116] 1) Purpose
[0117] - The performance of a DBMS is calculated as a combination of the performance of various elements such as disk, CPU, and memory.
[0118] - Verify DBMS performance in separate Read / Write cases using QPS (Query Per Second) and Latency (Response Speed), which are the most intuitive performance indicators among the final performance deliverables of a DBMS.
[0119] 2) Measurement tools: sysbench, perf
[0120] 3) Measurement Logic
[0121] - Measure QPS, Latency, IPC, and Clock speeds for Read Only, Write Only, and Read / Write modes while increasing in multiples of 2 from 1 to 512 Threads (1, 2, 4, 8, 16, …).
[0122] - Record the average excluding maximum and minimum values after 5 repeated tests
[0123] 4) Measurement items
[0124] - Measurement of QPS (Query Per Second), 95thLatency, Clock, and IPC (Instructions Per Cycle) during Read Only, Write Only, and Read / Write
[0126] 6. Load Balancer
[0127] 1) Purpose
[0128] - Measure the direct load generation section from Client to Server and the load handling level when routed through the LB.
[0129] - Selected TPS metrics similar to actual user service requests and MTT (Mean Time to Stop), which users directly experience, as standard metrics.
[0130] 2) Measurement Tool: nGrinder (Based on The Grinder, an open-source project for server performance measurement)
[0131] 3) Basis for Adoption: Verification of application-level performance metrics (tps, response time) and ability to adjust load (vuser) according to the situation
[0132] 4) Measurement Logic: When measuring using the tool, the following result values can be verified as a typical example.
[0133] Among these, information on performance trend characteristics by type is provided by collecting and analyzing the trends of change in each metric—TPS, Errors, and Mean_Test_Time (MTT)—during the test period.
[0134] - vuser: virtual user, number of virtual users (threads) configured when a load occurs
[0135] - Errors: Number of errors occurring during the test period
[0136] - Mean_Test_Time: Average test time (used as response time)
[0137] - TPS: Number of transactions that can be processed per second
[0138] - Mean_response_length: Average size of HTTP Response (Bytes)
[0139] - Mean_time_to_resolve_host: Average connection time from Domain Name to Target Server / Application
[0140] - Mean_time_to_establish_connection: Average time for a TCP connection to establish a connection with a Target Server / Application
[0142] 7. Application Performance
[0143] 1) Purpose
[0144] - Measures the performance value that can be processed by assuming application-level user requests.
[0145] - Selected TPS metrics similar to actual user service requests and MTT (Mean Time to Stop), which users directly experience, as standard metrics.
[0146] 2) Measurement Tool: nGrinder (Based on The Grinder, an open-source project for server performance measurement)
[0147] 3) Basis for Adoption: Verification of application-level performance metrics (tps, response time) and ability to adjust load (vuser) according to the situation
[0148] 4) Measurement Logic: When measuring using the tool, the following result values can be verified as a typical example.
[0149] Among these, information on performance trend characteristics by type is provided by collecting and visualizing the trends of change in each metric during the test period, focusing on TPS / Errors / Mean_Test_Time (MTT) metrics.
[0150] - vuser: virtual user, number of virtual users (threads) configured when a load occurs
[0151] - Errors: Number of errors occurring during the test period
[0152] - Mean_Test_Time: Average test time (used as response time)
[0153] - TPS: Number of transactions that can be processed per second
[0154] - Mean_response_length: Average size of HTTP Response (Bytes)
[0155] - Mean_time_to_resolve_host: Average connection time from Domain Name to Target Server / Application
[0156] - Mean_time_to_establish_connection: Average time for a TCP connection to establish a connection with a Target Server / Application
[0158] 8. File Storage
[0159] 1) Purpose
[0160] - Verify data processing speed by measuring Throughput, which refers to the size of data transferred when an application attempts disk I / O, and IOPS, which refers to Input / Output Operations Per Second.
[0161] - IOPS and Throughput have a direct impact on the application's data processing speed.
[0162] 2) Measurement Tool: vdbench (I / O generation tool for measuring storage performance, provided by Oracle)
[0163] 3) Basis for Adoption: Enables the acquisition of necessary information by allowing adjustment of file system environments based on situation (number of directories, number of files) and providing comprehensive support for storage performance metrics.
[0164] 4) Measurement Logic: When measuring using the tool, the following result values can typically be verified.
[0165] Among these, measurement results were collected for ReqstdOps (rate / response) and mb / sec (read / write) metrics. Throughput and IOPS performance data information for Object Storage by CSP is provided.
[0166] - ReqstdOps(rate / response): Measures Total I / O rate and response time when I / O load occurs
[0167] - Read(rate / response): Measured values of I / O rate and response time when Read I / O load occurs
[0168] - write(rate / response): Measures I / O rate and response time when write I / O load occurs
[0169] - read(pct): Read job ratio when I / O occurs
[0170] - mb / sec(read / write): Amount of data processed when performing read or write jobs
[0171] - cpu(%): Server-side CPU usage during load
[0173] 9. Object Storage
[0174] 1) Purpose
[0175] - Measure the error rate during Object Storage Read / Write operations to reduce costs associated with retries
[0176] Throughput, representing data transmission volume, and response time, representing data processing speed, were selected as standard metrics.
[0177] 2) Measurement Tool: cosbench (Open source tool for measuring Cloud Object Storage performance)
[0178] 3) Basis for Adoption: Solution supporting Object Storage performance measurement and ability to secure performance metrics required for data collection (Success Rate, Throughput, etc.)
[0179] 4) Measurement Logic: When measuring using the tool, the following result values can typically be verified.
[0180] Among these, measurement results are collected for Avg-ResTime, Throughput, Succ-Ratio, and Bandwidth metrics, and information on storage performance trends is provided through data and visualization.
[0181] - Op-Count(ea): Total number of operations
[0182] - Byte-Count(byte): Total data transfer amount
[0183] - Avg-ResTime(ms): Average Response Time
[0184] - Throughput (Op / s): Number of completed operations per second
[0185] - Succ-Ratio(%): Operation Success Rate
[0186] - Bandwidth (MB / s): Amount of data transmitted per second
[0188] The test result processing unit (230) can process the performance test results of the VM instance performed by the performance test unit (220).
[0189] More specifically, the test result processing unit (230) can store the performance test results of the VM instance in memory (130) (or an external database (not shown)).
[0190] Additionally, the test result processing unit (230) can compare the performance test result of the VM instance with the average performance of a VM instance of the same environment and specifications, and if the comparison result shows that the performance of the VM instance is less than the average performance of a VM instance of the same environment and specifications, it can send an alarm recommending the regeneration of the VM instance to the user terminal (20).
[0191] On the other hand, if the performance of the VM instance is greater than the average performance of the VM instance with the same environment and specifications as a result of comparison, the test result processing unit (230) can terminate the performance test of the VM instance.
[0192] Here, the term "environment" refers to the location, time, or situation in which a VM instance is created, and the term "specification" refers to the spec of the created VM instance. The average performance c of a VM instance with environment a and specification b may mean that when a VM instance with specification b is created in environment a, it has an average performance of c.
[0193] That is, the performance of VM instances by physical environment and specifications is stored in memory (130) or an external database (not shown), and the test result processing unit (230) can calculate the average performance of VM instances that have the same environment and specifications as the VM instance created by the user terminal (20) among the VM instances stored in memory (130) or an external database (not shown), and compare the average with the performance of the VM instance created by the user terminal (20) to determine whether to send an alarm recommending the regeneration of the VM instance.
[0194] FIG. 4 is a flowchart illustrating a method for a performance test data collection device to collect data that tests the performance of a VM instance according to an embodiment of the present invention.
[0195] Referring to FIGS. 3 and 4, the receiving unit (210) receives a request for a performance test for a VM instance created by the user terminal (20) from the user terminal (20) (S400), and can generate information necessary to test the performance of the VM instance for which the performance test was requested (S410).
[0196] The performance test unit (220) can test the performance of a VM instance using information necessary to test the performance of a VM instance (S420).
[0197] Afterwards, the test result processing unit (230) stores the performance test results of the VM instance in memory (130) (S430), and based on the result of comparing the performance test results of the VM instance with the average performance of a VM instance of the same environment and specifications, it can decide whether to send an alarm recommending the regeneration of the VM instance to the user terminal (20) (S440).
[0198] That is, if the performance of the VM instance is greater than the average performance of VM instances of the same environment and specifications ('Yes' in S440), the test result processing unit (230) can terminate the performance test of the VM instance (S450).
[0199] On the other hand, if the performance of the VM instance is lower than the average performance of VM instances of the same environment and specifications ('No' in S440), the test result processing unit (230) may send an alarm recommending the regeneration of the VM instance to the user terminal (20) (S460).
[0200] Combinations of each block of the block diagram attached to the present invention and each step of the flowchart may be performed by computer program instructions. Since these computer program instructions may be loaded into an encoding processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions performed through the encoding processor of the computer or other programmable data processing equipment create means for performing the functions described in each block of the block diagram or each step of the flowchart. Since these computer program instructions may also be stored in computer-available or computer-readable memory that can be directed toward the computer or other programmable data processing equipment to implement the function in a specific way, the instructions stored in computer-available or computer-readable memory may also produce a manufactured item containing instruction means for performing the function described in each block of the block diagram or each step of the flowchart. Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that execute a computer or other programmable data processing equipment by performing a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.
[0201] Additionally, each block or each step may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). Also, it should be noted that in some alternative embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps described in succession may actually be performed substantially simultaneously, or the blocks or steps may be performed in reverse order according to the corresponding function.
[0202] The above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential quality of the present invention. Accordingly, the embodiments disclosed in the present invention are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the present invention. Explanation of the symbols
[0203] 10: Performance Test Data Collection System 20: User terminal 100: Performance test data acquisition device 200: Performance Test Data Collection Program 210: Receiver 220: Performance Test Department 230: Test Result Processing Unit
Claims
Claim 1 A method for collecting the performance of a VM instance created by a user terminal, performed by a performance test data collection device, comprising: receiving a performance test request for the VM instance created by the user terminal from the user terminal; analyzing the performance test request to generate information necessary for testing the performance of the VM instance; testing the performance of the VM instance using the information necessary for testing the performance of the VM instance; and storing the results of the test in memory included in the performance test data collection device or in an external database, wherein the testing step includes performing a comparison between the average performance of VM instances having the same environment and specifications as the VM instance and the performance of the VM instance. Claim 2 A method for collecting the performance of a VM instance created by a user terminal, comprising: a step of comparing the result of the test with the average performance of VM instances having the same environment and specifications as the VM instance, which are stored in the memory or the database; and a step of sending an alarm recommending the recreation of the VM instance to the user terminal based on the result of the comparison. Claim 3 In claim 2, the step of transmitting an alarm recommending the regeneration of the VM instance is a method for collecting the performance of a VM instance created by a user terminal, wherein if the result of the test is less than the average performance of VM instances having the same environment and specifications as the VM instance, the user terminal transmits an alarm recommending the regeneration of the VM instance. Claim 4 A performance test data collection device for collecting the performance of a VM instance created by a user terminal, comprising: a memory; a transceiver receiving a performance test request for the VM instance created by the user terminal from the user terminal; and a processor that analyzes the performance test request to generate information necessary for testing the performance of the VM instance, tests the performance of the VM instance using the information necessary for testing the performance of the VM instance, and stores the results of the test in the memory or an external database, wherein the processor performs a comparison of the average performance of VM instances having the same environment and specifications as the VM instance with the performance of the VM instance. Claim 5 A performance test data collection device according to claim 4, wherein the processor compares the result of the test with the average performance of VM instances having the same environment and specifications as the VM instance stored in the memory or the database, and transmits an alarm recommending the regeneration of the VM instance to the user terminal using the transceiver according to the result of the comparison. Claim 6 In claim 5, the processor is a performance test data collection device that transmits an alarm recommending the recreation of the VM instance to the user terminal when the result of the test is less than the average performance of VM instances having the same environment and specifications as the VM instance. Claim 7 A computer program stored on a computer-readable recording medium, wherein the computer program comprises instructions for a processor to perform a method according to any one of claims 1 to 3.
Citation Information
Patent Citations
System, method and server for looking up last result data of deactivated cloud virtual machine instance
KR1020160041272A
