An FIO-based block storage performance automatic testing method and system, and a storage medium

Abstract

The application discloses a kind of block storage performance automation test method, system, storage medium based on FIO, it is related to cloud computing and storage performance test technical field, it is applied to the cloud platform environment including management node and multiple test nodes.By running environment deployment script on management node, automatically complete test node IP management, SSH password-free configuration, firewall processing and according to X86 / ARM architecture adaptive deployment FIO;Run test execution script, precheck is carried out to test disk and time synchronization, start FIO service on multiple nodes according to preset task queue, in turn complete the stress test of covering 4K / 32K / 128K / 1M block size and random, order, 7:3 mixed read-write scene;Run result arrangement script, automatically parse FIO log, extract and uniformly output the IOPS, bandwidth and latency index of each scene.The method reduces manual operation while improving the consistency and result reliability of block storage performance test.

Description

Technical Field

[0001] This invention belongs to the field of cloud computing and storage performance testing technology, and in particular relates to an automated testing method, system and storage medium for block storage performance based on FIO. Background Technology

[0002] With the widespread deployment of cloud platforms, especially private cloud platforms, in production environments, verifying block storage performance before platform delivery has become a necessary step. A common practice is to use FIO tools to perform stress tests on the mounted data disks on multiple test cloud hosts, including random read / write, sequential read / write, and mixed read / write scenarios, and extract key indicators such as IOPS, bandwidth, and latency from the results to evaluate whether the block storage performance meets delivery requirements.

[0003] In existing technologies, block storage performance testing still relies heavily on manual operations: test engineers need to manually bind data disks, install FIO, handle firewalls and network access for each test cloud host, and manually start load testing tasks on multiple nodes. In order to cover different block sizes such as 4K, 32K, 128K, and 1M, as well as various modes such as random / sequential read / write and 7:3 mixed read / write, multiple rounds of load testing are often required, and each test takes a long time.

[0004] This approach has several drawbacks: during the deployment phase, multiple nodes repeatedly install and configure FIO, which is inefficient and prone to omissions; during the test execution phase, each node relies on manual control of its startup time, making it difficult to guarantee second-level synchronization. Once the time is out of sync, especially in mixed read / write scenarios, the actual read / write ratio may deviate from expectations, affecting the authenticity of the results; throughout the entire testing process, there are many manual input commands and steps, which can easily lead to problems such as missed test cases or incorrect parameters. After the test is forced to be interrupted, manual troubleshooting and retrying are required, further lengthening the testing cycle.

[0005] During the results processing phase, test engineers need to manually extract metrics such as IOPS, bandwidth, and latency from each FIO log. For mixed read / write scenarios, they also need to separate the performance data for read and write operations. As the number of scenarios and nodes increases, this process becomes extremely tedious and prone to statistical errors due to visual fatigue, misreading fields, etc. Furthermore, existing processes generally lack automated cleanup mechanisms. When tests are interrupted or not promptly shut down the FIO services on each node after completion, residual processes continue to consume disk I / O and system resources, interfering with subsequent tests and even affecting the normal operation of the nodes.

[0006] Therefore, there is an urgent need for an automated block storage performance testing solution that can automatically complete environment deployment, uniformly schedule load testing tasks, ensure time synchronization, and automatically extract and organize test results in a multi-node environment, while also having test protection and resource cleanup mechanisms, in order to improve testing efficiency and the reliability of results. Summary of the Invention

[0007] In view of this, the present invention aims to propose an automated testing method for block storage performance based on FIO, so as to at least solve one of the problems in the background art.

[0008] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0009] Firstly, this solution discloses an automated testing method for block storage performance based on FIO, applied to a cloud platform testing environment including a management node and multiple test nodes communicatively connected to the management node, comprising:

[0010] The environment deployment steps are as follows: The management node receives the IP addresses of multiple test nodes input by the user, performs format verification on the IP addresses, and generates a node list file; it checks whether the management node has configured an SSH key pair; if not, it generates a key pair and configures passwordless login for the multiple test nodes based on the key pair; it disables the firewall service of each test node via remote command; it automatically selects the corresponding FIO installation package according to the hardware architecture type of each test node, distributes the FIO installation package to each test node, and completes the installation verification.

[0011] The test execution steps are as follows: Load the IP addresses of each test node according to the node list file; check if the FIO tool is available on each test node and if the preset test task file exists; parse the disk path of the block storage device to be tested from the test task file, and verify the existence and capacity validity of the disk on each test node; obtain the system time of each test node and calculate the time difference; when the time difference exceeds a preset threshold, terminate the test process; when the time difference meets the requirements and the user confirms that the test will only be performed in the test environment, start the FIO service on each test node at a predetermined startup time, and trigger stress tests including multiple block sizes and multiple read / write modes sequentially according to the preset task queue; save the test logs of each test scenario to a result directory named with a time identifier; during the test, automatically shut down the FIO service on each test node when an interrupt signal is detected through a signal capture mechanism.

[0012] The test results processing steps involve traversing the test log files in the results directory on the management node, parsing the block size, read / write mode, queue depth, and concurrency for each test scenario based on the file name and log content, extracting IOPS, bandwidth, and latency metrics from the log files, normalizing the bandwidth and latency, extracting read and write operation metrics for mixed read / write test scenarios, and outputting the metrics for each test scenario in tabular form.

[0013] Furthermore, in the environment deployment step, the format validation of the IP addresses of the multiple test nodes input by the user includes: using regular expressions in combination with system network commands to perform a validity check on each IP address, generating a node list file only when all IP addresses pass the validation, otherwise outputting an error message and terminating the deployment process.

[0014] Furthermore, the automatic selection of the corresponding FIO installation package based on the hardware architecture type of each test node includes: obtaining the processor architecture information of each test node by remotely executing system commands on the management node, selecting the FIO installation package that matches the X86 architecture or ARM architecture based on the processor architecture information, and adapting the package management methods of different operating systems such as CentOS, Kylin, and Euler to complete the deployment of the FIO tool.

[0015] Furthermore, in the test execution steps, detecting whether the FIO tool is available on each test node and whether the preset test task file exists includes: executing the FIO version query command on each test node to confirm that the FIO tool is executable, and detecting multiple test task files based on a predefined path. When the FIO tool or test task file does not exist, outputting missing information to the user and terminating the test process.

[0016] Furthermore, the step of parsing the disk path of the block storage device to be tested from the test task file and verifying the existence and capacity validity of the disk on each test node includes: parsing the disk path parameters in each test task file on the management node to obtain a set of disk paths to be tested; performing device query and capacity query on each disk in the set of disk paths on each test node, and proceeding to the subsequent test process only when all disks exist and the capacity meets the preset test requirements.

[0017] Furthermore, the step of obtaining the system time of each test node and calculating the time difference includes: obtaining the current system time of each test node in batches on the management node, calculating the difference between the maximum and minimum values ​​of the current system time as the time difference, outputting a time synchronization prompt and terminating the test when the time difference is greater than a threshold of 5 seconds, and calculating the waiting time according to the test start time set by the user when the time difference is less than or equal to the threshold, and starting the FIO service on each test node uniformly when the waiting time ends.

[0018] Furthermore, the step of uniformly starting the FIO service on each test node at a predetermined startup time includes: starting the FIO tool in server mode on each test node and listening on a unified port; when the management node captures an interrupt signal or a termination signal, sending a cleanup command to each test node to shut down the corresponding FIO service process; and after the cleanup is completed, checking again for any residual FIO processes, and if any exist, forcibly terminating them.

[0019] Furthermore, the stress test, which is triggered sequentially according to the preset task queue and includes multiple block sizes and multiple read / write modes, includes: executing at least twelve standard test tasks in sequence, covering block sizes of 4K, 32K, 128K and 1M, random read, random write, sequential read, sequential write and mixed read / write with a read / write ratio of 7:3, and saving the execution log of each test task as an independent log file.

[0020] Furthermore, the step of extracting IOPS, bandwidth, and latency metrics from the log file and normalizing the bandwidth and latency includes: firstly extracting completion latency-related statistical information from the log file; when completion latency information is missing, extracting total latency statistics and converting microsecond-level latency to millisecond-level latency; converting bandwidth in KiB per second or MiB per second to bandwidth in MB per second; and for mixed read / write test scenarios, extracting and recording the IOPS, bandwidth, and latency metrics for read and write operations respectively.

[0021] Furthermore, the test execution steps also include: outputting test risk warning information on the management node before starting the test, limiting the method to be used only in the test environment, requiring the user to confirm that the current environment is the test environment by entering a confirmation command before continuing to execute the test; automatically cleaning up the FIO service process and related temporary files on each test node after the test is interrupted or all test tasks are completed, releasing disk I / O and system resources.

[0022] Furthermore, before traversing the test log files in the result directory on the management node, the method further includes: enabling strict shell mode when running the test result processing script, so that the script execution is terminated when encountering undefined variables, command execution failure, or failure of any subcommand in the pipeline, in order to prevent the continued output of erroneous test results in abnormal states.

[0023] Secondly, this solution discloses an automated testing system for block storage performance based on FIO, including a management node and multiple test nodes communicatively connected to the management node.

[0024] The management node is equipped with an environment deployment module, which is used to receive the IP addresses of multiple test nodes input by the user, perform format verification on the IP addresses and generate a node list file, configure passwordless login between the management node and each test node, disable the firewall service of each test node, and automatically select and deploy the corresponding FIO installation package according to the hardware architecture type of each test node.

[0025] The management node is equipped with a test execution module, which is used to load each test node based on the node list file, verify the existence of the FIO tool and test task file on each test node, parse the disk path of the block storage device to be tested from the test task file and verify the existence and capacity validity of the disk, obtain the system time of each test node and verify the time synchronization, and start the FIO service on each test node at a predetermined start time after the user confirms that the current environment is a test environment. The module executes stress tests including multiple block sizes and multiple read and write modes according to the preset task queue, and manages the shutdown and cleanup of the FIO service process on each test node after the test is interrupted or completed.

[0026] The management node is equipped with a test result processing module, which is used to traverse the test log files in the test result directory, parse the block size, read / write mode, queue depth and concurrency of each test scenario, extract IOPS, bandwidth and latency indicators from the test log files, perform unit normalization on the indicators, and output the indicators of each test scenario in tabular form.

[0027] Thirdly, this solution discloses an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement an automated testing method for block storage performance based on FIO.

[0028] Fourthly, this solution discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an automated testing method for block storage performance based on FIO.

[0029] Compared with existing technologies, the FIO-based automated block storage performance testing method described in this invention has the following advantages:

[0030] (1) The FIO-based block storage performance automated testing method described in this invention automatically completes test node IP management, format verification, SSH passwordless configuration, firewall shutdown, and automatically selects the FIO installation package corresponding to the X86 / ARM architecture based on uname-m and operating system type and completes the installation verification. It realizes architecture adaptive deployment for mainstream Linux distributions such as CentOS, Kylin, and Euler, avoids manual node-by-node installation and configuration, and significantly improves deployment efficiency.

[0031] (2) The FIO-based automated block storage performance testing method described in this invention divides environment deployment, test execution, and result processing into three script modules. By associating node list files and result directories, a complete automated closed loop is achieved from environment preparation, test task queue scheduling to log analysis and output. Test engineers only need to run the scripts sequentially on the management node and perform a few confirmation operations to complete the entire test, reducing manual login and repetitive command input, and lowering the probability of human error.

[0032] (3) The FIO-based automated block storage performance testing method described in this invention automatically obtains the system time of each test node and calculates the time difference before the test is executed. When the time difference exceeds a preset threshold (e.g., 5 seconds), the test is actively terminated and a synchronization prompt is given. The FIO service is only started on each node after the time difference meets the requirements and the user confirms. With the unified scheduling of the task queue, multiple nodes can maintain good synchronization in scenarios such as random read / write, sequential read / write, and 7:3 mixed read / write, reducing the impact of time deviation on the read / write ratio and performance results from the source, and improving the consistency and repeatability of the test results.

[0033] (4) The FIO-based block storage performance automated testing method described in this invention uses tools such as grep and sed to automatically parse FIO logs, prioritize matching the clat field, and fall back to the lat field when missing, and perform unit conversion from microseconds to milliseconds; at the same time, bandwidths such as KiB / s and MiB / s are uniformly converted to MB / s. For the 7:3 mixed read and write scenario, the script extracts the IOPS, bandwidth and latency indicators of read and write operations respectively, and displays the block size, read and write mode, queue depth, concurrency and performance indicators of each scenario in a structured table, avoiding manual line-by-line log review and manual extraction, and greatly reducing the workload and error rate of result processing;

[0034] (5) The FIO-based automated block storage performance testing method described in this invention, before test execution, forces the script to output risk warning information, requiring the user to explicitly confirm that the current environment is the test environment, to prevent the stress test from being run in the production environment by mistake; during execution, interruption and termination signals are uniformly processed through a signal capture mechanism, and the FIO service on each test node is automatically shut down and residual processes are detected when the test is interrupted or completed, and forced termination is performed when necessary to avoid long-term resource occupation and reduce interference with subsequent tests and business. Combined with the pre-test checks on disk existence, capacity and FIO tool availability, the safety and robustness of the entire test process are improved;

[0035] (6) The FIO-based automated block storage performance testing method described in this invention, through script automation, shortens the traditional multi-node block storage performance test, which takes about 2.5 hours and relies entirely on manual labor, to about 10 minutes, significantly improving efficiency; time synchronization control and automated indicator extraction reduce human operation errors and improve the accuracy and repeatability of performance results; at the same time, the test process reduces the dependence on the operator's command line experience, and non-professionals can also complete complex multi-scenario tests by following the prompts, improving the ease of use and scalability of the solution. Attached Figure Description

[0036] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0037] Figure 1 This is a flowchart of the environment deployment script described in an embodiment of the present invention;

[0038] Figure 2 This is a flowchart of the FIO automated test script according to an embodiment of the present invention;

[0039] Figure 3 A flowchart for organizing the test results described in the embodiments of the present invention. Detailed Implementation

[0040] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0041] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0042] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0043] like Figure 1 The diagram illustrates the process of the environment deployment script deploy_fio.sh in this embodiment of the invention. After starting the deployment process, the management node first initializes the logs and collects a list of IP addresses of the nodes to be tested. It then verifies the validity of the user-inputted node IP addresses, generating a node list file named node_list upon successful verification. Subsequently, the script configures SSH passwordless login for each test node according to the node_list and preprocesses the test environment, including batch processing firewall policies such as stopping the firewalld service to prevent port access from being blocked. Next, the script completes the deployment and installation package check of the FIO tools on each test node, distinguishing between X86 and ARM based on the node architecture information, and transmitting and installing the corresponding version of the tools to nodes with different architectures. After installation, it waits for the tools to be ready and performs installation verification. If verification fails, the error is recorded and the program automatically exits; if verification succeeds, the interface displays a message indicating deployment completion and outputs the log location, ending the environment deployment process.

[0044] like Figure 2 The diagram illustrates the workflow of the FIO automated test execution script fiocsnew.sh in this embodiment of the invention. After startup, the script first checks the node_list file and records the prompt information. Upon successful verification, it proceeds to the pre-test preparation and verification process: the script sets up and enables signal capture processing logic, triggering FIO service cleanup on all test nodes upon receiving a termination signal to prevent residual processes from affecting subsequent tests. Subsequently, it performs pre-test checks, including checking the existence of FIO tools and test task files, checking the validity of the disk under test, and completing necessary time synchronization verification. After the pre-test checks pass, the script enters the FIO service management phase, starts the service, and performs port connectivity checks. Under normal service conditions, the script automatically executes multiple sets of FIO test tasks according to a preset queue and generates corresponding result logs. Once all test tasks are completed, it outputs test process completion information and terminates.

[0045] like Figure 3The diagram illustrates the process of the test result processing script `make_result.sh` in this embodiment of the invention. After startup, the script first outputs the header of the result table and defines sub-functions for parsing logs, including the `extract_latency` function for extracting latency metrics and the `analyze_log` function for analyzing individual logs and extracting key metrics. Then, it defines a list of log files to be analyzed (`file_list`) and iterates through each file. When a log file exists, `analyze_log` is called to parse it, extracting parameter information corresponding to the test scenario and outputting metrics such as block size, read / write mode, queue depth, concurrency, IOPS, bandwidth, and latency. For specific mixed read / write scenarios, such as 1M random read / write 7:3, the script extracts both read and write statistics separately and outputs them on separate lines, ensuring that the results match the log structure of different scenarios. After the iteration is complete, it outputs a message indicating that the script has finished executing and ends the result processing.

[0046] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0047] It should be noted that the embodiments of the present invention are illustrated using the Tsinghua Unigroup Cloud private cloud platform as a typical application scenario, but the present invention is not limited to this platform and is also applicable to other cloud platforms or data center environments that use block storage.

[0048] Example 1: Automated Testing Method for Block Storage Performance Based on FIO

[0049] In this embodiment, the FIO-based automated block storage performance testing method is deployed in a private cloud platform. The platform includes at least one cloud host as a management node and multiple cloud hosts as test nodes. Each test node is equipped with a block storage data disk to be tested, such as a cloud hard drive or a local data disk.

[0050] The management node runs a Linux operating system with a built-in shell script execution environment and establishes network connectivity with each test node; each test node also runs a compatible Linux distribution, such as CentOS, Kylin, Euler, etc., and supports the installation of FIO tools.

[0051] The method in this embodiment is implemented collaboratively by three Shell scripts: the environment deployment script deploy_fio.sh, the test execution script fiocsnew.sh, and the test result processing script make_result.sh, which correspond to the environment deployment steps, test execution steps, and test result processing steps mentioned in this solution.

[0052] I. Environmental Deployment Steps

[0053] In the environment deployment step, the management node executes the environment deployment script deploy_fio.sh to complete the unified initialization configuration for multiple test nodes, specifically including:

[0054] 1. Test Node IP Management and Verification

[0055] The management node receives multiple test node IP addresses input by the user, separated by commas, for example, "10.128.9.245,10.128.9.246,10.128.9.247".

[0056] The script uses regular expressions combined with system network commands to perform format validity checks on each IP address. Only if all IP addresses pass the checks will the IP address be written to the node list file node_list. If an invalid IP address is found, an error message will be output, and the subsequent deployment process will be terminated to avoid performing remote operations on the wrong node.

[0057] 2. Passwordless login configuration

[0058] The management node automatically checks whether an SSH key pair has been generated locally; if not, it calls ssh-keygen to generate a new key pair.

[0059] Subsequently, the script uses the ssh-copy-id command to distribute the public key of the management node to each test node in the node_list file in batches, and automatically performs remote login verification to ensure that the management node can log in to each test node without a password, laying the foundation for subsequent batch remote command execution.

[0060] 3. Firewall preprocessing

[0061] To avoid firewall policies blocking network access to FIO service ports, the environment deployment script remotely logs into each test node via SSH and executes commands to disable the firewall (e.g., systemctl stop firewalld) and disable the firewall at startup, ensuring that port access is not blocked during testing.

[0062] In some operating systems, the script can also automatically select the appropriate firewall management command based on the system version to improve compatibility.

[0063] 4. Automated deployment and adaptive architecture of FIO tools

[0064] The management node obtains the processor architecture information of each test node by remotely executing system commands such as uname -m, thus distinguishing between x86 architecture nodes and ARM architecture nodes.

[0065] For x86 architecture nodes, the script automatically selects the appropriate installation package, such as fio.tar, and uses the scp command to transfer it to the specified directory on each test node before decompressing and installing it.

[0066] For ARM architecture nodes, the script automatically selects the fioarm.tar.gz installation package and executes the corresponding installation steps.

[0067] During the installation process, the script can automatically select the appropriate dependency installation method and command based on the target operating system, such as CentOS, Kylin, or Euler.

[0068] After installation, the script executes commands such as "fio -v" on each test node to verify whether the FIO tool is usable, and outputs the installation results and possible error information to the log file fio_deploy.log for easy troubleshooting later.

[0069] Through the above steps, this embodiment realizes the batch, architecture-adaptive deployment of FIO tools and the unified configuration of the basic environment in a multi-node environment, which greatly reduces the workload of manual repetitive installation and configuration on each node.

[0070] II. Test Execution Steps

[0071] During the test execution process, the management node executes the test execution script fiocsnew.sh, which automatically schedules and controls the multi-node block storage performance stress test, specifically including:

[0072] 1. Node pre-reading and environment pre-verification

[0073] The test execution script first checks if the node_list file exists and then loads the IP addresses of each test node from it.

[0074] Subsequently, the script executes an FIO version query command on each test node to verify whether the FIO tool is correctly installed; it also checks whether the pre-configured test task files (.job) exist, such as a set of task files containing scenarios like 4K random write, 4K random read, and 32K sequential write. If any node is missing FIO tools or task files, the script outputs the missing information and terminates the test process to prevent testing from being performed in an incomplete environment.

[0075] 2. Disk check

[0076] The management node parses the filename parameter from each test task file, which corresponds to the block storage disk path to be tested, such as / dev / vdb.

[0077] The script performs device and capacity queries on each parsed disk path at each test node to confirm that the disk actually exists and its capacity meets the preset test requirements. If a disk is found to be missing or has an abnormal capacity, the script outputs an error message and interrupts the test to avoid distorting test results due to incorrect devices.

[0078] 3. Time synchronization control

[0079] To ensure that the start and end times of multi-node concurrent testing are as consistent as possible, the script retrieves the current system time of each test node in batches on the management node and calculates the difference between the maximum and minimum values ​​as the time difference.

[0080] When the time difference is greater than a preset threshold (e.g., 5 seconds), the script outputs a time synchronization error message and terminates the test, prompting the user to configure time synchronization first; when the time difference is less than or equal to the threshold, the script calculates the waiting time based on the test start time set by the user (which can be immediate execution or a future time point), and starts the FIO service on each test node when the waiting time ends.

[0081] 4. Test risk warning and execution confirmation

[0082] Before the test is officially started, the test execution script outputs a clear risk warning message on the management node terminal, indicating that this script is only allowed to be executed in the test environment and is strictly prohibited from running in the production business environment; the script will only continue to execute when the user clearly confirms that the current environment is a test environment by entering a confirmation command such as "Y".

[0083] This mandatory interactive confirmation mechanism reduces the risk of erroneous operations affecting production business systems from the source.

[0084] 5. FIO service management and signal capture mechanism

[0085] Provided that user confirmation and time synchronization conditions are met, the script starts FIO in service mode on each test node (e.g., by executing the "fio-server-daemon" command), listens on port 8765, and confirms that the service is in listening state through the port detection command.

[0086] The test execution script sets up a signal capture mechanism on the management node side. When an interrupt signal (such as SIGINT) or a termination signal (such as SIGTERM) is detected, it automatically sends a cleanup command to each test node to shut down the corresponding FIO service process, so as to avoid leaving processes that occupy disk I / O and system resources after the test is interrupted. After the cleanup is completed, the status of the FIO process on each node is checked again, and any remaining processes are forcibly terminated.

[0087] 6. Automated task execution

[0088] After the FIO service starts and is in a ready state, the test execution script initiates test requests sequentially according to the preset task queue.

[0089] In this embodiment, the preset task queue includes at least 12 standard test scenarios, covering different block sizes such as 4K, 32K, 128K and 1M, as well as various IO modes such as random read, random write, sequential read, sequential write and mixed read and write with a read-write ratio of 7:3.

[0090] The script concurrently issues test commands to all test nodes in the FIO client / server mode according to the queue order and the corresponding task file, runs for a specified duration (e.g., 10 minutes for a single scenario), and saves the execution logs of each test task to the results directory named with the current time, such as "fio_results_20240101103000".

[0091] After all tasks are completed, the script outputs the directory path where the test results are stored and executes the FIO service cleanup process again to ensure that there are no residual test processes on each test node.

[0092] Through the above-described automated test execution steps, this embodiment avoids the complex operations of manually logging in to each machine and executing commands in each scenario, significantly shortens the overall test execution time, and eliminates problems such as missed executions and repeated executions caused by manual operations.

[0093] III. Steps for Processing Test Results

[0094] In the test result processing step, the management node executes the test result processing script `make_result.sh` to perform batch parsing and structured output of multi-scenario test logs in the aforementioned results directory, specifically including:

[0095] 1. Initialization and Strict Mode Settings

[0096] Enable bash strict mode when the script starts, for example, by enabling it simultaneously via "set-euo pipefail":

[0097] Accessing an undefined variable will result in an error and the program will exit.

[0098] The script terminates if any command fails to execute.

[0099] The failure of any subcommand in the pipeline will also be considered a failure of the entire pipeline.

[0100] Strict mode can prevent the output of incomplete or incorrect statistical results in the event of missing logs or abnormal command execution.

[0101] The script then defines the header fields of the output table, such as BlockSize, RWMode, QueueDepth, NumJobs, IOPS, Bandwidth (MB / s), Latency (ms), etc., and prints the headers using formatted methods such as printf, laying the foundation for the subsequent standardized output of results.

[0102] 2. Design of the delay extraction sub-function

[0103] To accommodate latency statistics recorded in different formats in the FIO log, this embodiment defines the extract_latency function in the script. This function takes an anchor point (e.g., "read:" or "All clients:") and the entire log content as parameters.

[0104] The function prioritizes matching statistical rows related to completion latency (clat). If a match is found, the average latency value is extracted. If no clat statistics are found in the log, the function then matches the total latency (lat) field and extracts the average latency value.

[0105] For latency data in microseconds, the function automatically converts it to milliseconds to standardize the table output format.

[0106] 3. Core Logic of Single Log Analysis

[0107] The script defines the analyze_log function to process a single FIO log file.

[0108] The function first parses the test case name from the log file name, then further parses the block size (e.g., 4K, 32K, 128K, 1M) and read / write mode (e.g., randread, randwrite, read, write, rand-RW7:3, etc.) from the name, and obtains parameters such as queue depth and concurrency based on the task file or log content.

[0109] For non-hybrid scenarios (such as pure read or pure write), the analyze_log function directly extracts the overall IOPS, bandwidth, and latency metrics from the log;

[0110] For the mixed read / write 7:3 scenario, the analyze_log function calls the latency extraction sub-function with "read:" and "write:" as anchor points respectively, and parses the IOPS and bandwidth data corresponding to read and write respectively, and outputs the metrics of read operation and write operation respectively, so that testers can accurately evaluate the performance of read and write operations in the mixed scenario.

[0111] 4. Log list definition and batch traversal analysis

[0112] The script pre-lists 12 target FIO log files using the file_list array, corresponding to the aforementioned 12 standard test scenarios, covering four block sizes (4K / 32K / 128K / 1M) and three core scenarios (random read / write, sequential read / write, and 7:3 mixed read / write).

[0113] In actual execution, the script iterates through each log file in file_list using a for loop to check if the file exists. If it exists, the analyze_log function is called to complete single-file analysis and table output. If a log file is missing, a message is output, but this does not affect the parsing of other logs.

[0114] 5. Structured Result Output

[0115] In summary, the test result processing script outputs all available test log parsing results in a table in a unified format. Each row corresponds to a specific test scenario, including core metrics such as block size, read / write mode, queue depth, concurrency, IOPS, bandwidth (MB / s), and latency (ms).

[0116] Testers can use this table to quickly compare performance differences under different block sizes and read / write modes, enabling efficient evaluation of cloud platform block storage performance.

[0117] The test result processing steps in this embodiment completely automate the tedious work that originally required manually reviewing logs and extracting metrics, thus avoiding omissions and errors in the manual data extraction process.

[0118] Example 2: Typical Three-Node Test Scenario

[0119] In this embodiment, a specific test scenario based on the above method is given to illustrate the actual application process of the present invention.

[0120] 1. Test Environment Description

[0121] The test environment includes three test nodes with IP addresses of 10.128.9.245, 10.128.9.246, and 10.128.9.247, respectively. The management node is interconnected with the three test nodes.

[0122] Each test node is equipped with a block storage disk / dev / vdb to be tested, and the disk capacity is sufficient for continuous stress testing of 12 scenarios.

[0123] 2. Environment Deployment Process

[0124] Run the environment deployment script deploy_fio.sh on the management node, and enter the IP address as "10.128.9.245,10.128.9.246,10.128.9.247".

[0125] The script first validates the input IP format. If the validation is successful, it generates a node_list file. Then, it automatically configures SSH passwordless login, disables and blocks firewalls on each node, selects the appropriate FIO installation package according to the architecture type of each node (fio.tar for x86 nodes and fioarm.tar.gz for ARM nodes), completes the FIO deployment, and outputs the detailed deployment process and results to fio_deploy.log.

[0126] Operations personnel can check fio_deploy.log to confirm that FIO has been successfully installed on all nodes and can run normally.

[0127] 3. Test execution process

[0128] After the environment deployment is complete, run the test execution script fiocsnew.sh on the management node.

[0129] The script automatically detects and loads the node IPs in node_list, prompts the user to confirm that the current environment is only for testing and asks for "Y" to confirm; then it checks the existence of the FIO tool and test task files on each node, verifies that the disk / dev / vdb exists and has valid capacity, and checks that the system time difference between the three nodes is less than 5 seconds.

[0130] When the user selects the "Execute Now" mode, the script immediately starts the FIO service on the three test nodes and executes 12 standard test tasks in the order of the preset queue, covering block sizes of 4K, 32K, 128K, and 1M, as well as random / sequential read / write and 7:3 mixed read / write modes.

[0131] All test result logs are saved in the "fio_results_current time" directory. After all tests are completed, the script automatically shuts down the FIO service processes on the three nodes and checks again for any residual processes to ensure that the test environment is restored to an idle state.

[0132] 4. Test Result Processing

[0133] After the test is completed, specify the corresponding results directory on the management node and run the test results processing script make_result.sh.

[0134] The script enables bash strict mode, defines the output header, and analyzes the 12 FIO log files in the file_list array one by one. For each scenario, it extracts metrics such as block size, read / write mode, queue depth, concurrency, IOPS, MB / s bandwidth, and millisecond latency.

[0135] For a 1M mixed read / write 7:3 scenario, the script outputs the IOPS, bandwidth, and latency for both read and write operations. Finally, the results of all test scenarios are displayed in a table format on the terminal or in the output file, making it convenient for test engineers to directly compare and evaluate.

[0136] Through this typical three-node test scenario, the present invention compresses the original test process, which took about 2.5 hours and required full manual intervention, into about 10 minutes, and ensures that the process is repeatable and the results are easy to verify through automated scripts.

[0137] Example 3: Automated Testing System Based on Modular Architecture

[0138] In this embodiment, the invention is described from a system perspective. The FIO-based automated block storage performance testing system of the present invention includes:

[0139] Environment Deployment Module: Deployed on the management node, it is used to implement the functions described in Example 1, such as IP management and verification, SSH passwordless configuration, firewall preprocessing, FIO automated deployment and architecture adaptive processing.

[0140] Test execution module: Deployed on the management node, it is used to implement functions such as test node environment pre-verification, disk check, time synchronization verification, test risk warning and confirmation, FIO service management, signal capture and automatic execution of test task queue.

[0141] Test result processing module: Deployed on the management node, it is used to implement functions such as test log traversal, latency extraction sub-function, single log analysis function, log list definition and traversal, and structured table output.

[0142] The modules described above can be implemented using scripts, packaged into graphical user interface tools, or integrated into a unified testing platform. The modules interact with each other by sharing configuration files, node list files, and result directories, enabling fully automated workflow from environment deployment and test execution to result processing.

[0143] Example 4: Electronic device and computer-readable storage medium

[0144] In this embodiment, the FIO-based automated block storage performance testing method of the present invention can also be deployed in software form on an electronic device. The electronic device includes a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, it completes the steps described in Embodiment 1 or Embodiment 2.

[0145] Similarly, the present invention can also be provided in the form of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the FIO-based automated block storage performance testing method as described in the foregoing embodiments. Those skilled in the art will understand that the computer-readable storage medium can be a USB flash drive, external hard drive, read-only memory, random access memory, optical disc, or other media capable of storing program code.

[0146] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An automated testing method for block storage performance based on FIO, applied to a cloud platform testing environment including a management node and multiple test nodes communicatively connected to the management node, characterized in that, include: Environment deployment: The management node receives the IP addresses of multiple test nodes input by the user, performs format verification on the IP addresses, and generates a node list file. Check whether the management node has been configured with an SSH key pair. If not, generate a key pair and configure passwordless login for the multiple test nodes based on the key pair. The firewall service of each test node is shut down via remote command; the corresponding FIO installation package is automatically selected according to the hardware architecture type of each test node, the FIO installation package is distributed to each test node and the installation verification is completed; The test execution involves loading the IP addresses of each test node according to the node list file, checking whether the FIO tool is available on each test node and whether the preset test task file exists; parsing the disk path of the block storage device to be tested from the test task file, and verifying the existence and capacity validity of the disk on each test node; obtaining the system time of each test node and calculating the time difference; stopping the test process when the time difference exceeds a preset threshold; and starting the FIO service on each test node at a predetermined startup time when the time difference meets the requirements and the user confirms that the test will only be performed in the test environment. Stress tests including multiple block sizes and multiple read / write modes are triggered sequentially according to the preset task queue. Test logs for each test scenario are saved to a result directory named with a time identifier. During the test, the FIO service on each test node is automatically shut down when an interrupt signal is detected through a signal capture mechanism. The test results are compiled by traversing the test log files in the results directory on the management node. The block size, read / write mode, queue depth, and concurrency of each test scenario are obtained by parsing the file name and log content. IOPS, bandwidth, and latency metrics are extracted from the log files. The bandwidth and latency are normalized. For mixed read / write test scenarios, the metrics for read and write operations are extracted separately, and the metrics for each test scenario are output in tabular form.

2. The automated testing method for block storage performance based on FIO according to claim 1, characterized in that, In the environment deployment step, the format validation of the IP addresses of multiple test nodes input by the user includes: using regular expressions in combination with system network commands to perform a validity check on each IP address, generating a node list file only when all IP addresses pass the validation, otherwise outputting an error message and terminating the deployment process.

3. The automated testing method for block storage performance based on FIO according to claim 1, characterized in that, The test execution steps include checking whether the FIO tool is available on each test node and whether the preset test task files exist. This includes executing the FIO version query command on each test node to confirm that the FIO tool is executable, and checking multiple test task files based on the predefined path. If the FIO tool or test task files do not exist, the missing information is output to the user and the test process is terminated.

4. The automated testing method for block storage performance based on FIO according to claim 1, characterized in that, The step of parsing the disk path of the block storage device to be tested from the test task file and verifying the existence and capacity validity of the disk on each test node includes: parsing the disk path parameters in each test task file on the management node to obtain a set of disk paths to be tested; performing device query and capacity query on each disk in the set of disk paths on each test node, and proceeding to the subsequent test process only when all disks exist and the capacity meets the preset test requirements.

5. The automated testing method for block storage performance based on FIO according to claim 1, characterized in that, The process of obtaining the system time of each test node and calculating the time difference includes: obtaining the current system time of each test node in batches on the management node, calculating the difference between the maximum and minimum values ​​of the current system time as the time difference, outputting a time synchronization prompt and terminating the test when the time difference is greater than a threshold of n seconds, calculating the waiting time according to the test start time set by the user when the time difference is less than or equal to the threshold, and starting the FIO service on each test node uniformly when the waiting time ends.

6. The automated testing method for block storage performance based on FIO according to claim 1, characterized in that, The step of uniformly starting the FIO service on each test node at a predetermined startup time includes: starting the FIO tool in server mode on each test node and listening on a unified port; when the management node captures an interrupt signal or a termination signal, sending a cleanup command to each test node to shut down the corresponding FIO service process; and after the cleanup is completed, checking again for any residual FIO processes, and if any exist, forcibly terminating them.

7. The automated testing method for block storage performance based on FIO according to claim 1, characterized in that, The test execution steps also include: outputting test risk warning information on the management node before starting the test, limiting the method to be used only in the test environment, requiring the user to confirm that the current environment is the test environment by entering a confirmation command before continuing to execute the test; automatically cleaning up the FIO service process and related temporary files on each test node after the test is interrupted or all test tasks are completed, releasing disk I / O and system resources.

8. The automated testing method for block storage performance based on FIO according to claim 1, characterized in that, Before traversing the test log files in the result directory on the management node, the method further includes: enabling strict Shell mode when running the test result processing script, so that the script execution is terminated when encountering an undefined variable, command execution failure, or failure of any subcommand in the pipeline, in order to prevent the continued output of erroneous test results in abnormal states.

9. An automated performance testing system for block storage based on FIO, comprising a management node and multiple test nodes communicatively connected to the management node, characterized in that: The management node is equipped with an environment deployment module, which is used to receive the IP addresses of multiple test nodes input by the user, perform format verification on the IP addresses and generate a node list file, configure passwordless login between the management node and each test node, disable the firewall service of each test node, and automatically select and deploy the corresponding FIO installation package according to the hardware architecture type of each test node. The management node is equipped with a test execution module, which is used to load each test node based on the node list file, verify the existence of the FIO tool and test task file on each test node, parse the disk path of the block storage device to be tested from the test task file and verify the existence and capacity validity of the disk, obtain the system time of each test node and verify the time synchronization, and start the FIO service on each test node at a predetermined start time after the user confirms that the current environment is a test environment. The module executes stress tests including multiple block sizes and multiple read and write modes according to the preset task queue, and manages the shutdown and cleanup of the FIO service process on each test node after the test is interrupted or completed. The management node is equipped with a test result processing module, which is used to traverse the test log files in the test result directory, parse the block size, read / write mode, queue depth and concurrency of each test scenario, extract IOPS, bandwidth and latency indicators from the test log files, perform unit normalization on the indicators, and output the indicators of each test scenario in tabular form.

10. A computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the FIO-based automated testing method for block storage performance as described in any one of claims 1 to 8.

Images