A virtualization performance evaluation method based on loss ratio

Abstract

The present invention belongs to cloud computing and virtualization technology, and relates to a virtualization performance evaluation method based on loss ratio. The present invention adopts a unified software and hardware testing environment, eliminates the differences between software and hardware environments of different manufacturers, and ensures the comparability of evaluation results. At the same time, by comparing the performance of the client and the host, the comprehensive loss ratio is calculated, which is used as the basis for evaluating the virtualization performance level, normalizing the test data of different dimensions and units, and effectively supporting the evaluation and selection of virtualization platforms. In addition, the evaluation method also has the characteristics of automated execution, which reduces the testing workload, avoids human errors, and supports rapid reproduction and re-verification of the evaluation process.

Description

Technical Field

[0001] The invention belongs to cloud computing and virtualization technology, and relates to a virtualization performance evaluation method based on loss ratio. Background Art

[0002] Virtualization platform is an important infrastructure in the field of modern computer science. It allows multiple complete virtual computer (client) hardware environments to be simulated on the same physical computer (host) and multiple independent operating system instances to improve overall resource utilization and computing efficiency. Virtualization platform is widely used in cloud computing, data center virtualization and other scenarios. Through virtualization technology, it realizes the pool management of computing, storage and network resources, which can flexibly meet the resource requirements of different user applications, improve equipment utilization and reduce maintenance complexity.

[0003] Due to the implementation principle of virtualization technology, the virtualization platform needs to add a virtualization layer on top of the host hardware to convert the client's instructions into instructions that the host operating system and hardware can understand and execute. Therefore, the client's performance is often lower than that of the host. Virtualization platform manufacturers usually adopt various strategies such as improving instruction conversion efficiency, optimizing resource management algorithms, and increasing hardware acceleration support to improve the platform's virtualization performance, minimize the performance overhead brought by the virtualization layer, and reduce the impact on client performance.

[0004] The virtualization performance of the virtualization platform is a key factor in determining the client performance. Therefore, virtualization performance evaluation is an essential part of the virtualization platform users' platform evaluation and selection process.

[0005] Currently, in the process of evaluating and selecting virtualization platforms, virtualization performance evaluation mainly relies on performance data provided by manufacturers or POC (proof of concept) tests organized by users.

[0006] The performance data provided by manufacturers are usually the results of tests based on their own hardware and software environments. Different hardware and software environments, including specific hardware configurations, software versions, operating systems, network conditions, etc., lead to obvious differences and lack of direct comparability between the performance data provided by manufacturers; in addition, performance data involves multiple different dimensions, and its dimensions and units are also different, so it is difficult to give a unified evaluation result.

[0007] POC testing is usually user-led and provides a unified software and hardware environment. In this environment, the virtualization platform manufacturer is responsible for deploying its platform software to verify whether the software can effectively support upper-level user applications. However, since user application software usually does not have professional performance testing functions, although POC testing can quickly verify whether the virtualization platform meets user business needs, it cannot make a horizontal comparison of multiple virtualization platforms that all meet the needs, making it difficult to make the best selection decision. In addition, due to the lack of automated evaluation methods, the efficiency of the deployment test process of multiple manufacturers is low.

[0008] Flight simulation is a high-real-time application that has high performance requirements for basic hardware and software such as computers and operating systems. The virtualization platform is located in the infrastructure layer of the cloud computing resource architecture. As an operating system-level software, it abstracts real, physical hardware resources into virtual, logical resource pools and provides resource services to upper-level cloud applications. In the application scenario of flight simulation on the cloud, the virtualization performance of the virtualization platform is a key factor affecting the performance of cloud simulation.

[0009] However, in the current evaluation and selection process of virtualization platforms, virtualization performance evaluation mainly relies on performance data provided by manufacturers or POC (proof of concept) tests organized by users. There are problems such as poor comparability of test data and lack of comprehensive and objective basis for evaluating virtualization performance levels. Summary of the invention

[0010] The purpose of the present invention is to solve the problems encountered in the evaluation and selection process of virtualization platforms, especially the current situation of lack of effective and objective automated evaluation methods. The present invention is based on the performance overhead analysis of the client and the host, and the proposed technical solution has the advantages of strong comparability of test results, comprehensive and objective evaluation basis, and high degree of automation of the execution process. It can effectively support the evaluation and selection of virtualization platforms, while reducing the testing workload and avoiding human errors. The scope of application of the present invention is not limited to the field of flight simulation, but is also applicable to the evaluation and selection scenarios of virtualization platforms in other industry fields, and is used to realize the automated evaluation of virtualization performance.

[0011] The technical solution of the present invention is a virtualization performance evaluation method based on loss ratio, comprising the following steps:

[0012] Step 1: Prepare a host machine with unified configuration, install the operating system, the virtualization platform to be evaluated, and a unified test application on the host machine;

[0013] Step 2: Prepare a client with unified configuration, install a unified operating system and a unified test application on the client, and create a client image file in a unified format;

[0014] Step 3: Measure the hardware performance indicators of the host machine;

[0015] Step 4: Create and start the client and measure the hardware performance indicators of the client;

[0016] Step 5: Compare the hardware performance index measurement results of the host machine and the client machine, and calculate the loss ratio;

[0017] Step 6: Calculate the average loss ratio for multiple test cases in the same type of hardware performance indicators;

[0018] Step 7: Calculate the comprehensive loss ratio of the virtualization platform to be evaluated based on the weights of different types of hardware performance indicators.

[0019] The step 1 prepares a host machine with uniform configuration, specifically: for different virtualization platforms to be evaluated, the host machine adopts a uniform computer hardware configuration, including CPU resources, memory space, hard disk type and capacity, network card type and capacity.

[0020] The step 1 installs the operating system and the virtualization platform to be evaluated on the host machine, specifically: installing the virtualization platform to be evaluated and the operating system matched therewith on the host machine.

[0021] The step 1 installs a unified test application on the host machine, specifically: for different virtualization platforms to be evaluated, a unified test application software is installed on the host machine, including CPU performance test software, memory performance test software, network performance test software and storage performance test software.

[0022] The step 2 prepares a client with a uniform configuration, specifically: for different virtualization platforms to be evaluated, the client adopts a uniform computer virtual hardware configuration, including CPU resources, memory space, hard disk type and capacity, network card type and capacity.

[0023] The step 2 installs a unified operating system and a unified test application on the client, specifically: for different virtualization platforms to be evaluated, a unified operating system is installed on the client, and test application software consistent with that of the host is installed.

[0024] The step 2 is to prepare a client image file in a unified format, specifically: prepare a client image file in a unified format for different virtualization platforms to be evaluated; and import the client image file in a unified format into different virtualization platforms to be evaluated so as to generate a client with a unified configuration.

[0025] The step 3 measures the hardware performance indicators of the host machine, specifically: using a unified test application software to measure the hardware performance indicators of the host machine, the indicator types include CPU, memory, network and storage; or different types of hardware performance indicators contain multiple test cases; running a performance test script on the host machine, automatically executing multiple test cases in each type of hardware performance indicator, and saving the measurement results.

[0026] The step 4 creates and starts the client, specifically: running the client creation script on the host machine to automatically complete uploading the client image file, creating the client and starting the client.

[0027] The step 4 measures the hardware performance indicators of the client, specifically: using the test application software consistent with the host machine to measure the hardware performance indicators of the client, the indicator type is consistent with the host machine, and the test cases of different types of hardware performance indicators are consistent with the host machine; the host machine connects to the client and remotely runs the performance test script, automatically executes multiple test cases in each type of hardware performance indicator, and saves the measurement results.

[0028] The step 5 compares the hardware performance index measurement results of the host machine and the client machine, and calculates the loss ratio, specifically: compares the hardware performance index measurement results of the host machine and the client machine, and calculates the loss ratio for each test case in each type of hardware performance index, Run the loss ratio calculation script on the host machine to automatically compare the hardware performance indicator measurement results of the host machine and the client machine and calculate the loss ratio.

[0029] The step 6 calculates the average loss ratio for multiple test cases in the same type of hardware performance indicators, specifically: averaging the loss ratios of all test cases in the same type of hardware performance indicators to obtain the average loss ratio of the type of hardware performance indicators; running the loss ratio calculation script on the host machine to automatically calculate the average loss ratio.

[0030] The step 7 calculates the comprehensive loss ratio of the virtualization platform to be evaluated according to the weights of different types of hardware performance indicators, specifically: according to the weights of different types of hardware performance indicators, based on the average loss ratio, the comprehensive loss ratio of the virtualization platform to be evaluated is calculated; the loss ratio calculation script is run on the host machine to automatically calculate the comprehensive loss ratio; the comprehensive loss ratio is used as the virtualization performance evaluation result of the virtualization platform for horizontal comparison between different virtualization platforms.

[0031] The steps 3, 4, 5, 6 and 7 also include: running a batch script to sequentially complete running a performance test script on the host, running a client creation script on the host, connecting the host to the client and remotely running a performance test script, and running a loss ratio calculation script on the host, thereby automatically completing all measurement and calculation processes.

[0032] Beneficial effects of the present invention: In response to the problems existing in traditional virtualization performance evaluation solutions, this solution proposes a virtualization performance evaluation method based on loss ratio. This method adopts a unified software and hardware testing environment, eliminates the differences between software and hardware environments of different manufacturers, and ensures the comparability of the evaluation results. At the same time, by comparing the performance of the client and the host, the comprehensive loss ratio is calculated, which is used as the basis for evaluating the virtualization performance level, normalizing the test data of different dimensions and units, and effectively supporting the evaluation and selection of the virtualization platform. In addition, the evaluation method also has the characteristics of automated execution, which reduces the testing workload, avoids human errors, and supports rapid reproduction and re-verification of the evaluation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The difference between the host and client CPU processing processes;

[0034] Figure 2 The difference between the host and client memory access processes;

[0035] Figure 3 The differences between the host and client network communication processes;

[0036] Figure 4 Differences in the read and write processes for host and client storage;

[0037] Figure 5 Measuring the performance overhead of the process for the host and the client;

[0038] Figure 6 Automated execution of virtualization performance assessment process. DETAILED DESCRIPTION

[0039] The following uses a virtualization platform based on an X86 architecture CPU and hardware-assisted virtualization technology as an example to illustrate the specific implementation method of virtualization performance evaluation.

[0040] 1.1 Step 1: Prepare a host machine with unified configuration, install the operating system, the virtualization platform to be evaluated, and a unified test application on the host machine

[0041] 1.1.1 Prepare a host with unified configuration

[0042] For different virtualization platforms to be evaluated, the host machines all use a unified computer hardware configuration, including CPU resources, memory space, hard disk type and capacity, and network card type and capacity.

[0043] In this embodiment, the host machine is uniformly configured with 2 Intel Xeon Gold CPUs (28C@2.6GHz), 256GB of memory, 16TB of SATA HDD hard disk, and 1000BAST-T Ethernet card.

[0044] 1.1.2 Install the operating system and the virtualization platform to be evaluated on the host machine

[0045] Install the virtualization platform to be evaluated and its supporting operating system on the host. In the current data center and cloud computing usage scenarios, virtualization platform manufacturers usually customize and develop host operating systems that are compatible with their own virtualization platforms based on the open source Linux operating system, and package the host operating system and virtualization platform together to provide to users.

[0046] 1.1.3 Install a unified test application on the host machine

[0047] For different virtualization platforms to be evaluated, unified test application software is installed on the host machine, including CPU performance test software, memory performance test software, network performance test software and storage performance test software.

[0048] In this embodiment, sysbench is used as the CPU and memory performance test software, iperf is used as the network performance test software, and fio is used as the storage performance test software. 1.2 Step 2: Prepare a client with unified configuration, install a unified operating system and a unified test application on the client, and create a client image file in a unified format

[0049] 1.2.1 Prepare the client with unified configuration

[0050] For different virtualization platforms to be evaluated, the client computers all use a unified computer virtual hardware configuration, including CPU resources, memory space, hard disk type and capacity, and network card type and capacity.

[0051] In this embodiment, a client is created by VMWare or any other virtualization software. The client is uniformly configured with 1 virtual CPU (single core 2.6 GHz), 4 GB memory, 500 GB system disk, and Gigabit Ethernet card.

[0052] 1.2.2 Install a unified operating system and unified test application on the client

[0053] For different virtualization platforms to be evaluated, a unified operating system is installed on the client machine, and the test application software consistent with the host machine is installed.

[0054] In this embodiment, CentOS 7.0 is used as the client operating system, and the test application software consistent with the host machine is installed.

[0055] 1.2.3 Create a client image file in a unified format

[0056] Create client image files in a unified format for different virtualization platforms to be evaluated.

[0057] In this embodiment, an image creation tool (such as mkisofs, genisoimage) is used, the client mechanism is used as a custom image, and the image format is unified as qcow2.

[0058] The client image file in a unified format can be imported into different virtualization platforms to be evaluated to generate clients with unified configurations.

[0059] 1.3 Step 3: Measure the hardware performance indicators of the host machine

[0060] Use a unified test application to measure the host's hardware performance indicators, including CPU, memory, network, and storage.

[0061] 1.3.1 Host hardware performance index test cases and measurement results

[0062] The same type of hardware performance indicator measurement will use multiple test cases. In this embodiment, the test cases and measurement results used for measuring various types of indicators of the host machine are as follows:

[0063] (1) CPU test cases and measurement results

[0064] Test prime number calculation, maximum value 10000, 1 thread, 20 seconds eps: the measured result of the host machine is 2209.5493.

[0065] (2) Memory test cases and measurement results

[0066] Testing the 4K random read speed, the host machine's measurement result was 1629.15MiB / s.

[0067] Testing the 4K random write speed, the host machine's measurement result was 1612.9MiB / s.

[0068] Testing the 128K sequential read speed, the host machine's measurement result is 19631.88MiB / s.

[0069] Testing the 128K sequential write speed, the host machine's measurement result was 29342.73MiB / s.

[0070] (3) Network test cases and measurement results

[0071] Test the sending bandwidth of 1GB of data: the host machine's measurement result is 941639kbps.

[0072] Test the receiving bandwidth of sending 1GB of data: the measurement result of the host machine is 941214kbps.

[0073] Test time to send 1GB of data: The host machine measured 84.96s.

[0074] (4)Store test cases and measurement results

[0075] Test the random write latency of 4K and 1 queue: the measurement result of the host machine is 909.34us.

[0076] Test the random read latency of 4K and 1 queue: the measurement result of the host machine is 5376.77us.

[0077] Test the sequential write bandwidth of 128K and 32 queues: the measurement result of the host machine is 273062.46KiB / s.

[0078] Test the sequential read bandwidth of 128K and 32 queues: the measurement result of the host machine is 276919.37KiB / s.

[0079] 1.3.2 Automating the measurement process through performance test scripts

[0080] Run the performance test script on the host machine to automatically execute multiple test cases in various types of hardware performance indicators and save the measurement results.

[0081] The performance test script is used to perform specific hardware performance tests, including all test cases, and needs to be run on both the host and the client. The script running results are saved in the test result file for subsequent evaluation. A separate test result file is saved for each hardware performance, which facilitates the subsequent loss ratio calculation script to automatically parse the calculation result file.

[0082] In this embodiment, since both the host and client operating systems are Linux, the scripts all use Linux shell commands. The performance test scripts for each type of indicator are as follows:

[0083] (1) CPU: Call sysbench through the script to execute the test case and save the measurement results in the retCpu file. Specifically:

[0084] sysbench cpu--cpu-max-prime=10000--threads=1--time=20run>>retCpu

[0085] (2) Memory: Call sysbench through the script to execute the test case and save the measurement results in the retMem file. Specifically:

[0086] sysbench memory--memory-block-size=4k

[0087] --memory-total-size=100G--memory-oper=read

[0088] --memory-access-mode=seq run>>retMem

[0089] (3) Network: Call iperf through the script to execute the test case and save the measurement results in the retNet file. Specifically:

[0090] iperf3-c 192.168.2.67-f kn 10000000000>>retNet

[0091] (4) Storage: Call fio through the script to execute the test case and save the measurement results in the retStore file. Specifically:

[0092] fio--filename= / root / fiotest / test--direct=1

[0093] --ioengine=libaio--iodepth=1--rw=randwrite--bs=4k

[0094] --size=10G--numjobs=1--runtime=300--time_based

[0095] --group_reporting--lockmem=1g--norandommap--name=fio-rw--refill_buffers--randrepeat=0>>retStore

[0096] 1.4 Step 4: Create and start the client and measure the client's hardware performance indicators 1.4.1 Create and start the client

[0097] Run the client creation script on the host machine to automatically upload the client image file, create the client, and start the client.

[0098] In this embodiment, the client creation script uses the Restful API interface provided by the virtualization platform to automatically upload the client image file, create the client, and start the client. The client creation script is as follows:

[0099] (1) Send a POST request to upload the specified image file. After the upload is successful, obtain the image ID, specifically:

[0100] IMAGE_ID=$(curl-sX POST "$BASE_URL / images"-H"$HEADERS"-H"$CONTENT_TYPE"-d"{\"file\":\"$IMAGE_BASE64\"}"|jq-r'.id')

[0101] (2) Send a POST request to create a client based on the specified image. After the creation is successful, obtain the client ID, specifically:

[0102] INSTANCE_ID=$(curl-sX POST"$BASE_URL / instances"-H"$HEADERS"-H"$CONTENT_TYPE"-d

[0103] "{\"image_id\":\"$IMAGE_ID\"}"|jq-r'.id')

[0104] (3) Send a POST request to start the specified client, specifically:

[0105] curl-sX POST"$BASE_URL / instances / $INSTANCE_ID / start"-H"$HEADERS"-H"$CONTENT_TYPE"

[0106] 1.4.2 Measuring the client's hardware performance indicators

[0107] Client hardware performance indicator test cases and measurement results

[0108] The hardware performance indicators of the client are measured using the test application software consistent with the host. The indicator type is consistent with the host, and the test cases of different types of hardware performance indicators are consistent with the host. In this embodiment, the test cases used for measuring various types of indicators of the client and the measurement results are as follows:

[0109] (1) CPU test cases and measurement results

[0110] Test prime number calculation, maximum value 10000, 1 thread, duration 20 seconds EPS: the client measured result is 2199.411.

[0111] (2) Memory test cases and measurement results

[0112] Test 4K random read speed: the client measured result was 1609.43MiB / s.

[0113] Test 4K random write speed: the client measurement result is 1579.76MiB / s.

[0114] Test 128K sequential read speed: the client measured result was 19206.52MiB / s.

[0115] Test 128K sequential write speed: the client measured result was 29007.47MiB / s.

[0116] (3) Network test cases and measurement results

[0117] Test the sending bandwidth of 1GB of data: the client measured the result as 913991kbps.

[0118] Test the receiving bandwidth of sending 1GB of data: the client measured 913765kbps.

[0119] Test the time to send 1GB of data: The client measured 87.53s.

[0120] (4)Store test cases and measurement results

[0121] Test the random write latency of 4K and 1 queue: the measurement result of the client is 1010.66us.

[0122] Test the random read latency of 4K and 1 queue: the measurement result of the client is 5465.09us.

[0123] Test the sequential write bandwidth of 128K and 32 queues: the measurement result of the client is 265930.95KiB / s.

[0124] Test the sequential read bandwidth of 128K and 32 queues: the measurement result of the client is 276522.13KiB / s.

[0125] Automate the measurement process through performance test scripts

[0126] The host connects to the client and runs the performance test script remotely, automatically executing multiple test cases in various types of hardware performance indicators and saving the measurement results.

[0127] In this embodiment, the host machine connects to the client machine via ssh and remotely controls the client machine to run the performance test script, specifically:

[0128] ssh"${REMOTE_USER}@${REMOTE_HOST}""${SCRIPT_PATH}"

[0129] 1.5 Step 5: Compare the hardware performance index measurement results of the host and client, and calculate the loss ratio

[0130] 1.5.1 Analysis of the reasons for the difference in hardware performance index measurement results between the host and the client

[0131] The reason why the hardware performance index measurement results of the host and the client are different is that the processing process of the real hardware of the host and the virtual hardware of the client is different. In this embodiment, the processing process of different types of hardware on the host and the client is different as follows:

[0132] (1) Differences in CPU processing (see Figure 1 ) Mainly, the privileged / sensitive instructions of the client operating system need to be trapped in the Hypervisor layer (virtualization platform) for execution.

[0133] (2) Differences in memory access processes (see Figure 2 ) Mainly, the client needs to complete two-level memory mapping through MMU virtualization.

[0134] (3) Host-resident applications and client-resident applications communicate with the same external physical machine through the same physical network card and physical link. The differences in network communication processes (see Figure 3 ) Mainly, the client's network traffic needs to flow through the virtual network card (paravirtualized or fully virtualized network card) and the virtual switch.

[0135] (4) Host-resident applications and client-resident applications access the same local storage device through the same SATA driver. The differences in storage read and write processes (see Figure 4 ) Mainly, the client's storage traffic needs to flow through the client's file system, block device partition, paravirtualized disk driver, and host backend driver.

[0136] 1.5.2 Loss ratio calculation formula

[0137] The measurement results of hardware performance indicators are directly related to the performance overhead of the measurement process, so the loss ratio is calculated as follows:

[0138]

[0139] Performance overhead of host and client measurement process ( Figure 5)as follows:

[0140] (1) The performance overhead of the host measurement process = test application overhead + host operating system overhead + real hardware resource overhead.

[0141] (2) The performance overhead of the client measurement process = test application overhead + client operating system overhead + virtualization platform overhead + host operating system overhead + real hardware resource overhead.

[0142] Substituting the performance overhead of the host and client measurement process into the loss ratio calculation formula, the new calculation formula is as follows:

[0143]

[0144] This solution unifies the software and hardware test environment, including uniformly configured test applications, client operating systems, and hardware environments. Therefore, the test application overhead, client operating system overhead, and real hardware resource overhead of different virtualization platforms are the same. In addition, in the current data center and cloud computing usage scenarios, the host operating systems provided by different virtualization platform manufacturers are generally limited customized modifications to the open source version of Linux, so the differences in host operating system overhead can be ignored.

[0145] According to the loss ratio calculation formula, in the test process of different virtualization platforms, all parameters except the virtualization platform overhead are constants. Therefore, the larger the loss ratio, the greater the virtualization platform overhead, that is, the lower the virtualization performance.

[0146] 1.5.3 Loss ratio calculation results

[0147] In this embodiment, the hardware performance index measurement results of the host machine and the client machine are compared, and for each test case in each type of hardware performance index, the loss ratio is calculated as follows:

[0148] (1) CPU loss ratio

[0149] Test prime number calculation, maximum value 10000, 1 thread, duration 20 seconds eps: loss ratio is 0.45%.

[0150] (2) Memory loss ratio

[0151] Test 4K random read speed: the loss ratio is 1.21%.

[0152] Test 4K random write speed: the loss ratio is 2.05%.

[0153] Test 128K sequential read speed: the loss ratio is 2.16%.

[0154] Test 128K sequential write speed: the loss ratio is 1.14%.

[0155] (3) Network loss ratio

[0156] The test shows that the transmission bandwidth: loss ratio of 1GB data is 2.93%.

[0157] The test sent 1GB of data and the receiving bandwidth: loss ratio was 2.91%.

[0158] The test time for sending 1GB of data: the loss ratio is 3.02%.

[0159] (4) Storage loss ratio

[0160] Test the random write latency of 4K and 1 queue: the loss ratio is 11.14%.

[0161] Test the random read latency of 4K and 1 queue: the loss ratio is 1.64%.

[0162] Test the sequential write bandwidth of 128K and 32 queues: the loss ratio is 2.61%.

[0163] Test the sequential read bandwidth of 128K and 32 queues: the loss ratio is 0.14%.

[0164] 1.5.4 Automatically execute the calculation process through the loss ratio calculation script

[0165] Run the loss ratio calculation script on the host machine to automatically compare the hardware performance indicator measurement results of the host machine and the client machine and calculate the loss ratio.

[0166] The loss ratio calculation script is used to automatically complete the calculation process of loss ratio, average loss ratio, and comprehensive loss ratio, and only needs to be run on the host machine. In this embodiment, the loss ratio calculation script reads the host and client test result files, calculates the loss ratio of each test case, and retains 2 significant digits, specifically:

[0167] paste-d,"$HOST_RESULTS""$GUEST_RESULTS"|while IFS=','read-r host_result guest_result; do

[0168] diff=$(echo"scale=2;($guest_result-$host_result) / $host_result"|bc)

[0169] done

[0170] 1.6 Step 6: Calculate the average loss ratio for multiple test cases in the same type of hardware performance indicators

[0171] 1.6.1 Average loss ratio calculation results

[0172] The loss ratios of all test cases in the same type of hardware performance indicator are averaged to calculate the average loss ratio of the hardware performance indicator of this type. In this embodiment, the average loss ratio calculation result is as follows:

[0173] (1) The average CPU loss ratio is 0.45%.

[0174] (2) The average memory loss ratio is 1.64%.

[0175] (3) The average network loss ratio is 2.95%.

[0176] (4) The average storage loss ratio is 3.88%.

[0177] 1.6.2 Automatically execute the calculation process through the loss ratio calculation script

[0178] Run the loss ratio calculation script on the host machine to automatically calculate the average loss ratio. In this embodiment, the loss ratio calculation script calculates the loss ratio of each test case, accumulates the loss ratio and the number of test cases, and finally calculates the average loss ratio, specifically:

[0179] paste-d,"$HOST_RESULTS""$GUEST_RESULTS"|while IFS=','read-r host_result guest_result; do

[0180] diff=$(echo"scale=2;($guest_result-$host_result) / $host_result"|bc)

[0181] total_diff=$(echo"$total_diff+$diff"|bc)

[0182] test_count=$((test_count+1))

[0183] done

[0184] avg_diff=$(echo"scale=2;$total_diff / $test_count"|bc)1.7Step 7: Calculate the comprehensive loss ratio of the virtualization platform to be evaluated based on the weights of different types of hardware performance indicators

[0185] 1.7.1 Calculation results of comprehensive loss ratio

[0186] According to the weights of different types of hardware performance indicators and based on the average loss ratio, the comprehensive loss ratio of the virtualization platform to be evaluated is calculated. In this embodiment, the weights of different types of hardware performance indicators are: CPU weight 40%, memory weight 40%, network weight 10%, storage weight 10%. The calculation result of the comprehensive loss ratio is 1.51%.

[0187] 1.7.2 Automatically execute the calculation process through the loss ratio calculation script

[0188] Run the loss ratio calculation script on the host machine to automatically calculate the comprehensive loss ratio. In this embodiment, the loss ratio calculation script calculates the comprehensive loss ratio based on the average loss ratio and weight of each type of hardware performance indicator, and retains 2 as a valid number. Specifically:

[0189] total_loss=$(echo"scale=2; ($cpu_loss*$cpu_weight)+($memory_loss*$memory_weight)+

[0190] ($network_loss*$network_weight)+

[0191] ($storage_loss*$storage_weight)"|bc)

[0192] 1.8 Batch processing steps 3 to 7

[0193] Run the batch script to complete the performance test script on the host, the client creation script on the host, the host connecting to the client and remotely running the performance test script, and the loss ratio calculation script on the host, and automatically complete all measurement and calculation processes ( Figure 6 ).

[0194] In this embodiment, the batch processing script is specifically:

[0195] $performance_test_script

[0196] vm_id=$($vm_creation_script)

[0197] $vm_start_script$vm_id

[0198] ssh$vm_user@$vm_ip"$performance_test_script"

[0199] $calculate_loss_ratio_script.

Claims

1. A virtualization performance evaluation method based on loss ratio, characterized in that: The following steps are involved: Step 1: Prepare a host machine with unified configuration, install the operating system, the virtualization platform to be evaluated, and a unified test application on the host machine; Step 2: Prepare a client with unified configuration, install a unified operating system and a unified test application on the client, and create a client image file in a unified format; Step 3: Measure the hardware performance indicators of the host machine; Step 4: Create and start the client and measure the hardware performance indicators of the client; Step 5: Compare the hardware performance index measurement results of the host machine and the client machine, and calculate the loss ratio; Step 6: Calculate the average loss ratio for multiple test cases in the same type of hardware performance indicators; Step 7: Calculate the comprehensive loss ratio of the virtualization platform to be evaluated based on the weights of different types of hardware performance indicators.

2. The virtualization performance evaluation method based on loss ratio according to claim 1 is characterized in that: The step 1 prepares a host machine with uniform configuration, specifically: for different virtualization platforms to be evaluated, the host machine adopts a uniform computer hardware configuration, including CPU resources, memory space, hard disk type and capacity, network card type and capacity.

3. The virtualization performance evaluation method based on loss ratio according to claim 1 is characterized in that: The step 1 installs the operating system and the virtualization platform to be evaluated on the host machine, specifically: installing the virtualization platform to be evaluated and the operating system matched therewith on the host machine.

4. The virtualization performance evaluation method based on loss ratio according to claim 1 is characterized in that: The step 1 installs a unified test application on the host machine, specifically: for different virtualization platforms to be evaluated, a unified test application software is installed on the host machine, including CPU performance test software, memory performance test software, network performance test software and storage performance test software.

5. The virtualization performance evaluation method based on loss ratio according to claim 1 is characterized in that: The step 2 prepares a client with a uniform configuration, specifically: for different virtualization platforms to be evaluated, the client adopts a uniform computer virtual hardware configuration, including CPU resources, memory space, hard disk type and capacity, network card type and capacity.

6. The virtualization performance evaluation method based on loss ratio according to claim 1 is characterized in that: The step 2 installs a unified operating system and a unified test application on the client, specifically: for different virtualization platforms to be evaluated, a unified operating system is installed on the client, and test application software consistent with that of the host is installed.

7. The virtualization performance evaluation method based on loss ratio according to claim 1 is characterized in that: The step 2 is to prepare a client image file in a unified format, specifically: prepare a client image file in a unified format for different virtualization platforms to be evaluated; and import the client image file in a unified format into different virtualization platforms to be evaluated so as to generate a client with a unified configuration.

8. The virtualization performance evaluation method based on loss ratio according to claim 1, characterized in that: The step 3 measures the hardware performance indicators of the host machine, specifically: using a unified test application software to measure the hardware performance indicators of the host machine, the indicator types include CPU, memory, network and storage; different types of hardware performance indicators contain multiple test cases; or running a performance test script on the host machine, automatically executing multiple test cases in each type of hardware performance indicator, and saving the measurement results.

9. The virtualization performance evaluation method based on loss ratio according to claim 1, characterized in that: The step 4 creates and starts the client, specifically: running the client creation script on the host machine to automatically complete uploading the client image file, creating the client and starting the client.

10. The virtualization performance evaluation method based on loss ratio according to claim 1, characterized in that: The step 4 measures the hardware performance index of the client, specifically: using the test application software consistent with the host to measure the hardware performance index of the client, the index type is consistent with the host, and the test cases of different types of hardware performance indexes are consistent with the host; The host connects to the client and runs the performance test script remotely, automatically executing multiple test cases in various types of hardware performance indicators and saving the measurement results.

11. The virtualization performance evaluation method based on loss ratio according to claim 1, characterized in that: The step 5 compares the hardware performance index measurement results of the host machine and the client machine, and calculates the loss ratio, specifically: compares the hardware performance index measurement results of the host machine and the client machine, and calculates the loss ratio for each test case in each type of hardware performance index, Alternatively, you can run a loss ratio calculation script on the host machine to automatically compare the hardware performance indicator measurement results of the host machine and the client machine and calculate the loss ratio.

12. The virtualization performance evaluation method based on loss ratio according to claim 1, characterized in that: The step 6 calculates the average loss ratio for multiple test cases in the same type of hardware performance indicators, specifically: averaging the loss ratios of all test cases in the same type of hardware performance indicators to obtain the average loss ratio of the type of hardware performance indicators; running the loss ratio calculation script on the host machine to automatically calculate the average loss ratio.

13. The virtualization performance evaluation method based on loss ratio according to claim 1, characterized in that: The step 7 calculates the comprehensive loss ratio of the virtualization platform to be evaluated according to the weights of different types of hardware performance indicators, specifically: according to the weights of different types of hardware performance indicators, based on the average loss ratio, the comprehensive loss ratio of the virtualization platform to be evaluated is calculated; the loss ratio calculation script is run on the host machine to automatically calculate the comprehensive loss ratio; the comprehensive loss ratio is used as the virtualization performance evaluation result of the virtualization platform for horizontal comparison between different virtualization platforms.

14. The virtualization performance evaluation method based on loss ratio according to claim 1, characterized in that: The steps 3, 4, 5, 6 and 7 also include: running a batch script to sequentially complete running a performance test script on the host, running a client creation script on the host, connecting the host to the client and remotely running a performance test script, and running a loss ratio calculation script on the host, thereby automatically completing all measurement and calculation processes.

Images