Storage equipment performance test method and device and computer equipment

Abstract

The invention provides a storage equipment performance testing method and device and computer equipment, and belongs to the technical field of storage equipment. The method comprises the following steps: acquiring test parameters input by a user, and generating at least one test parameter combination according to the test parameters; generating test configuration data corresponding to each test parameter combination, wherein the test configuration data comprises global parameters and test task parameters; based on the test configuration data corresponding to each test parameter combination, respectively calling a preset test tool to execute a test action, and continuously monitoring a test process corresponding to each test parameter combination to determine a test result and test error information corresponding to each test parameter combination; and generating and displaying a test report according to the test configuration data, the test result and the test error information corresponding to each test parameter combination. According to the method, the effects of simplifying the performance testing process of the storage device, supporting parallel testing of multiple storage devices and visually analyzing the testing result of the storage device can be achieved.

Description

Technical Field

[0001] This application relates to the field of storage device technology, and more specifically, to a storage device performance testing method, apparatus, and computer equipment. Background Technology

[0002] Storage device performance is a key factor determining the overall performance of a computer system, especially in applications highly sensitive to input / output (I / O) throughput and response latency, such as big data processing, cloud computing, artificial intelligence, and high-performance computing. To accurately evaluate storage device performance, the industry widely adopts the Flexible I / O Tester (FIO) tool as the standard testing tool. FIO is a powerful and highly configurable open-source storage performance testing tool that supports various combinations of parameters such as I / O engines, read / write modes, data block sizes, queue depths, and concurrent tasks, enabling it to simulate complex tests under real-world application scenarios.

[0003] However, when using FIO tools to professionally evaluate the performance of storage devices, there are several drawbacks that severely restrict the efficiency and depth of storage performance testing. Specifically: First, the testing process is cumbersome, requiring not only a comprehensive understanding of the storage device's performance characteristics but also manual combination of test parameters. Second, there is a lack of support for multi-device testing, making it difficult to efficiently organize and execute parallel comparative tests of multiple storage devices, and also hindering the unified analysis of parallel test results from multiple storage devices. Third, there is a lack of visualization capabilities, which prevents performance data from automatically generating intuitive trend charts, making it difficult to quickly locate performance inflection points, bottlenecks, and optimal configurations. Summary of the Invention

[0004] The purpose of this application is to provide a storage device performance testing method, apparatus, and computer equipment, which can simplify the performance testing process of storage devices, support parallel testing of multiple storage devices, and visualize the test results of storage devices.

[0005] The embodiments of this application are implemented as follows: A first aspect of this application provides a method for testing the performance of a storage device, the method comprising: Obtain the test parameters input by the user, including: storage device path, test run time, queue depth, number of threads, test mode, and block size; Based on the test parameters, at least one combination of test parameters is generated, wherein each combination of test parameters corresponds to a test task of at least one test device in a test scenario with a test mode, a queue depth, a number of threads, and a block size. Generate test configuration data corresponding to each combination of test parameters. The test configuration data includes global parameters and test task parameters. Based on the test configuration data corresponding to each test parameter combination, the preset test tools are called to execute test actions, and the test process corresponding to each test parameter combination is continuously monitored to determine the test results and test error information corresponding to each test parameter combination. Based on the test configuration data, test results, and test error information corresponding to each combination of test parameters, a test report is generated and displayed.

[0006] As one possible implementation, the above method also includes: When the user-inputted test parameters are not obtained, the pre-stored reference test parameters are retrieved. The reference test parameters include: a set of reference values ​​for queue depth, a set of reference values ​​for the number of threads, and a set of reference values ​​for block size.

[0007] As one possible implementation, at least one combination of test parameters is generated based on the test parameters, including: Parse the test parameters to obtain the corresponding sets of queue depth, thread count, test mode, and block size values; Perform a Cartesian product on the set of queue depth values, the set of thread count values, the set of test mode values, and the set of block size values ​​to obtain at least one combination of test parameters.

[0008] As one possible implementation, test configuration data corresponding to each combination of test parameters is generated, including: Based on the combination of various test parameters, construct the corresponding standard test configuration template, and write global parameters into the standard test configuration template. The global parameters include: input / output engine, direct input / output test mode, and test data size. Based on each combination of test parameters, determine whether each combination of test parameters includes multiple storage device paths; If yes, add the corresponding test path configuration item for each storage device path in the standard test configuration template; if no, add a test path configuration item in the standard test configuration template, determine the test task parameters corresponding to each test parameter combination based on each test parameter combination, and embed the test task parameters into the standard test configuration template to obtain the test configuration data corresponding to each test parameter combination.

[0009] As one possible implementation, a test report is generated and displayed based on the test configuration data, test results, and test error information corresponding to each combination of test parameters. This report includes: The test results corresponding to each combination of test parameters are analyzed to obtain the key performance indicators corresponding to each combination of test parameters. The key performance indicators include: number of input and output operations, bandwidth, and latency. Based on the analytical test results and key performance parameters corresponding to each combination of test parameters, generate performance charts and summary tables for each storage device. A test report is generated based on the performance charts and summary tables for each storage device, as well as the test configuration data, test results, and test error information for each combination of test parameters.

[0010] As one possible implementation, the test results corresponding to each combination of test parameters are analyzed to obtain the key performance indicators corresponding to each combination of test parameters, including: Analyze the test results corresponding to the test parameter combinations to determine whether the test mode corresponding to the test parameter combinations is a mixed read / write mode. If so, extract the read performance metrics, write performance metrics, bandwidth, and number of input / output operations from the parsed test results corresponding to the test parameter combinations. The read performance metrics include read operation latency and number of read input / output operations, and the write performance metrics include write operation latency and number of write input / output operations. Based on read and write performance metrics, determine the latency corresponding to the test parameter combination, and based on latency, bandwidth, and number of input / output operations, determine the key performance metrics corresponding to the test parameter combination.

[0011] As one possible implementation, the latency corresponding to the combination of test parameters is determined based on read performance metrics and write performance metrics, including: Calculate the first product between the read operation delay time and the number of read input / output operations; Calculate the second product between the write operation latency and the number of write input / output operations; Calculate the first sum between the first product and the second product; Calculate the second sum between the number of read input / output operations and the number of write input / output operations; Calculate the ratio between the first sum and the second sum to determine the delay time corresponding to the combination of test parameters.

[0012] As one possible implementation, based on the parsed test results corresponding to each combination of test parameters and key performance parameters, performance charts and summary tables for each storage device are generated, including: Based on the parsed test results corresponding to each combination of test parameters, determine the storage device, test mode, and block size corresponding to each combination of test parameters; Based on the storage device, test mode, and block size corresponding to each combination of test parameters, the parsed test results and key performance parameters corresponding to each combination of test parameters are grouped to obtain the parsed test results and key performance parameters corresponding to each storage device. Based on the parsed test results and key performance parameters of each storage device, performance charts and summary tables are generated for each storage device. The performance charts include: input / output operation count - queue depth change curve, bandwidth - queue depth change curve, and latency - queue depth change curve. The summary table is used to record the key performance parameters corresponding to each combination of thread count and queue depth.

[0013] Based on the test configuration data, test results, and test error information corresponding to each combination of test parameters, a test report is generated and displayed.

[0014] A second aspect of this application provides a storage device performance testing apparatus, the apparatus comprising: The parameter acquisition module is used to acquire the test parameters input by the user. The test parameters include: storage device path, test run time, queue depth, number of threads, test mode, and block size. The parameter combination module is used to generate at least one combination of test parameters based on the test parameters. Each combination of test parameters corresponds to a test mode, a queue depth, a number of threads, and a block size for a storage device. The configuration generation module is used to generate test configuration data corresponding to each combination of test parameters. The test configuration data includes global parameters and test task parameters. The test execution module is used to call preset test tools to execute test actions based on the test configuration data corresponding to each combination of test parameters, and continuously monitor the test process corresponding to each combination of test parameters to determine the test results and test error information corresponding to each combination of test parameters. The results parsing module is used to generate and display test reports based on the test configuration data, test results, and test error information corresponding to each combination of test parameters.

[0015] A third aspect of this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the storage device performance testing method described in the first aspect above.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the storage device performance testing method described in the first aspect.

[0017] The beneficial effects of the embodiments of this application include: This application provides a storage device performance testing method. It acquires user-inputted test parameters and intelligently combines them to obtain multiple test parameter combinations that comprehensively cover the user's input. For each test parameter combination, it constructs corresponding test configuration data, calls testing tools based on the test configuration data to execute test operations, and monitors the test process corresponding to each test parameter combination in real time to obtain test results for each parameter and record test error information during the test process. A comprehensive test report is generated based on the test configuration data, test results, and test error information corresponding to each test parameter combination. Specifically, after the user inputs test parameters, the method intelligently combines them to obtain multiple comprehensive test parameter combinations without manual intervention. Each test parameter performs performance testing on multiple storage devices in parallel using key parameters such as thread count and queue depth. It can also analyze the test process and results of multiple storage devices in parallel and convert complex test result text into user-readable charts and data. This simplifies the storage device performance testing process, supports parallel testing of multiple storage devices, and provides visualized analysis of storage device test results. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A structural diagram of a storage device performance testing system provided in this application embodiment; Figure 2 A flowchart illustrating the first storage device performance testing method provided in this application embodiment; Figure 3 A flowchart illustrating the second storage device performance testing method provided in this application embodiment; Figure 4 A flowchart illustrating the third storage device performance testing method provided in this application embodiment; Figure 5 A flowchart illustrating the fourth storage device performance testing method provided in this application embodiment; Figure 6 A flowchart illustrating the fifth storage device performance testing method provided in this application embodiment; Figure 7 A flowchart illustrating the sixth storage device performance testing method provided in this application embodiment; Figure 8 A delay time-queue depth variation curve is provided for an embodiment of this application; Figure 9 A schematic diagram of a summary table provided for an embodiment of this application; Figure 10 This is a schematic diagram of the structure of a storage device performance testing apparatus provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0023] Currently, FIO tools are frequently used to perform performance testing on storage devices to obtain performance evaluation results. However, this approach requires testers to have a comprehensive understanding of the storage device's performance characteristics and to manually combine test parameters to simulate various test scenarios, which significantly increases the complexity of storage device performance testing. Furthermore, this approach does not support parallel comparative testing of multiple storage devices, making it impossible to perform unified analysis of test results from multiple devices. Moreover, this approach lacks visualization capabilities, resulting in the inability to automatically generate intuitive trend charts from test performance data, hindering the rapid identification of performance inflection points, bottlenecks, and optimal configurations.

[0024] To address this, this application provides a storage device performance testing method. This method involves acquiring user-inputted test parameters or calling pre-stored reference test parameters, and automatically generating multiple test parameter combinations based on a preset algorithm. It then iterates through each test parameter combination, dynamically generating corresponding test configuration data. Based on the test configuration data for each test parameter combination, preset testing tools are invoked to perform test actions on at least one storage device, and the test process for each test parameter combination is continuously monitored to obtain test results and test error information. Finally, a visual test analysis report is generated based on the test configuration data, test results, and test error information for each test parameter combination, for tester reference. This simplifies the storage device performance testing process, supports parallel testing of multiple storage devices, and provides visual analysis of the storage device test results.

[0025] The test utilized an Intel server platform, configured with dual Intel processors and 1TB of memory, and incorporated a large-capacity storage module to build the hardware testing resources. Simultaneously, a CentOS 8.5.2111 x86 operating system environment was used, with the fio testing tool installed and a Python environment deployed. Performance testing of the storage devices was conducted within this system combining hardware and software testing resources.

[0026] The storage device performance testing method provided in the embodiments of this application will be explained in detail below with reference to the accompanying drawings.

[0027] Figure 1 This application provides a storage device performance testing system architecture diagram, with reference to... Figure 1 The system used for storage device performance testing provided in this application includes a user interaction layer, a core processing layer, and a data storage layer. The user interaction layer includes a command-line parameter interface, a configuration file manager, and a result display interface. The core processing layer includes a parameter parsing and configuration generation module, a dynamic test configuration generation module, a parallel test execution module, an intelligent result analysis module, a visualization analysis module, and a report generation module. The data storage layer includes an FIO configuration file storage space, a test result data storage space, a chart file storage space, and a report storage space.

[0028] Specifically, the command-line argument interface supports receiving test parameters input by the user through the human-computer interaction interface. It is responsible for receiving all test parameters (such as storage device path, queue depth, number of threads, test mode, and block size) and control parameters (such as output directory) entered by the user via the command line. Furthermore, it can automatically apply pre-stored reference test parameters when the user does not specify certain non-critical parameters, which can greatly simplify the testing process. In addition, the command-line argument interface can parse and convert the scattered, string-based test parameters input by the user into structured data objects (such as dictionaries or command spaces in Python) that the program can process internally, to generate reliable test parameter input.

[0029] In addition, the configuration file manager acts as an automated scheduling center, precisely translating high-level user test intentions into specific instructions that the FIO testing tools can recognize and execute. The configuration file manager contains standardized FIO configuration templates, which include a global configuration framework and a test task configuration framework. When the configuration file manager reads a combination of test parameters, it dynamically populates the configuration template based on that combination to obtain the corresponding FIO configuration text. The configuration file manager provides a bridge between upper-layer business logic and the lower-layer execution engine, obtaining the test intentions from the upper-layer business logic and relaying them to the lower-layer execution engine, enabling the FIO tools to execute tests according to the user's test intentions.

[0030] It's worth noting that the configuration file manager creates a unique FIO configuration text for each combination of test parameters obtained through intelligent combination, ensuring that each specific combination of test parameters can be executed accurately. The configuration file manager can intelligently handle performance testing of multiple storage devices, generating corresponding device path configuration items for multiple storage devices within a single FIO configuration text, thereby enabling parallel testing of multiple storage devices.

[0031] It should be noted that the prerequisite for the configuration file manager to create device path configuration items for multiple storage devices in a certain FIO configuration text is that the test parameter combination corresponding to the FIO configuration text contains storage device paths for multiple storage devices.

[0032] Furthermore, the results display interface, serving as the user interaction layer's presentation window, transforms the massive amounts of test data generated by the FIO tool into information easily understood and used for decision-making, enabling intelligent analysis of storage device performance test results. Specifically, the results display interface can generate trend curves for key performance parameters, presenting abstract performance indicators in an intuitive graphical format. It plots multiple trend curves for different thread counts, marking key data points on each curve so users can immediately see the trend of performance indicators changing with queue depth, bottlenecks, and optimal operating points. In addition, the results display interface can generate data summary tables, arranging key performance parameters in a two-dimensional format according to thread count and queue depth to form a clear performance matrix. This facilitates precise horizontal and vertical comparisons, providing data support for test report generation.

[0033] Optionally, the results display interface can also generate a comprehensive test report. This report compiles all information from the storage device performance testing process, such as test start and end times, test duration, test configuration information, test success rate, performance charts, and summary tables. This comprehensive test report can be directly used for reporting and archiving without manual intervention. Furthermore, automatic unit conversion and error log integration lower the barrier for users to interpret test results and improve the efficiency of test result analysis.

[0034] Optionally, the parameter parsing and configuration generation module is responsible for semantic parsing and structured processing of the received raw test requests (such as API calls, user-inputted test commands, etc.), extracting key test parameters (such as the path of the storage device under test, test mode, protocol type, etc.), and automatically generating standardized test configuration files according to preset rules or strategy templates. The parameter parsing and configuration generation module supports unified parsing of multi-format test parameters and can perform test parameter validation and default value filling to ensure the completeness of the generated FIO configuration text. The module can also call the test mode library or scenario template library in the knowledge base to map the user's high-level requirements to low-level executable configuration data. Furthermore, the module can output structured test plans for use by the FIO configuration template.

[0035] Furthermore, the dynamic test configuration generation module can dynamically adjust and optimize test configurations based on the current runtime environment status of the FIO tool, historical test data, and real-time feedback from the storage device under test, thereby improving test coverage and effectiveness. Specifically, the dynamic test configuration generation module can automatically adjust concurrency, request count, and latency according to system load capacity. It can also combine AI models to predict high-risk test paths and generate targeted performance test scenarios. In addition, the dynamic test configuration generation module supports differentiated configuration generation under complex deployment modes such as canary deployments and A / B testing. It can also link with service discovery mechanisms to automatically identify microservice topologies and generate end-to-end test plans.

[0036] Optionally, the parallel test execution module schedules multiple test agents or virtual users to concurrently execute multiple test tasks synchronously or asynchronously in a distributed environment to maximize resource utilization. The parallel test execution module is primarily used to manage the test node cluster to achieve load balancing and failover, while also controlling the test pace. Furthermore, the parallel test execution module collects key performance parameters of each test node in real time, supporting the parallel execution of various test types, thereby shortening the testing cycle of storage devices.

[0037] Optionally, the intelligent results analysis module cleans, aggregates, models, and intelligently diagnoses the massive amounts of raw test results data collected, automatically identifying performance bottlenecks, abnormal behaviors, and potential defects. The intelligent results analysis module evaluates system stability using statistical methods, discovers hidden problems using machine learning algorithms, judges performance regression by comparing with baseline data, and can also correlate logs, link tracing, and monitoring indicators for root cause localization, thereby achieving high-value data analysis.

[0038] Optionally, the visualization analysis module is used to present complex testing processes and results to users in an intuitive and interactive graphical interface, helping users understand the performance of storage devices. Specifically, the visualization analysis module can not only display trend graphs of key performance parameters and queue depth, but also provide graphical representations by highlighting chain topology, dependencies, and bottleneck nodes.

[0039] Optionally, the report generation module can integrate data and analysis conclusions from all testing phases to automatically generate standard, auditable, multi-level test reports. These reports typically include: test environment configuration, test start and end times, test runtime, test error logs, performance charts, data summary tables, a list of identified issues, and improvement suggestions.

[0040] Optionally, the FIO configuration file storage space is used to store test configuration data, centrally saving input / output workload templates for new tests on storage devices; recording core test parameters such as read / write test mode, block size, queue depth, runtime, and number of threads; supporting storage according to application scenarios (such as database load, log writing load, data scanning load, etc.); providing version management so that historical configuration data can be reused during regression performance testing; and also providing basic template library support for dynamically generated templates.

[0041] Optionally, the test result data storage space is used to store structured and semi-structured key performance parameters collected during the original test execution, and persistently saves the raw performance data stream output from each test run (such as throughput, latency distribution, memory space, network utilization, and timeout count). It can also provide timestamp indexes to support data comparison and analysis across time periods. It is worth noting that the test result data storage space can be used to automatically identify performance degradation of storage devices.

[0042] Optionally, the chart file storage space is used to store static images or dynamic chart files generated by the visualization analysis module. Specifically, it is used to store key performance trend charts, comparison bar charts, heat maps, call chain topology diagrams, etc. automatically generated during the testing process, which can be embedded into comprehensive test reports to enhance the intuitiveness of the test reports.

[0043] In addition, the chart file storage space provides a caching mechanism to accelerate the loading speed of the front-end dashboard and avoid repeated rendering. It can also be tagged and organized according to test identifier, test time, test task name, etc., to facilitate subsequent retrieval.

[0044] Optionally, the report storage space is used to store the fully packaged test report, uniformly archiving the final test results of each test task, including the test background, test objectives, test configuration data, key performance parameters, problem findings, and optimization suggestions for each test task.

[0045] Figure 2 A flowchart of a storage device performance testing method provided in this application is provided. This method can be applied to... Figure 1 The storage device performance testing system shown. See also... Figure 2 This application provides a method for testing the performance of a storage device, including: S201. Obtain the test parameters input by the user, including: storage device path, test run time, queue depth, number of threads, test mode, and block size.

[0046] Optionally, the user inputs test parameters through the human-machine interface provided by the test equipment or terminal server to inform the test equipment what performance tests they want to perform on the current batch of storage devices or the current storage device. The test parameters are the test conditions set through the human-machine interface, defining what performance tests the test tool will perform on the current batch of storage devices or the current storage device.

[0047] Specifically, the test parameters include: storage device path, test runtime, queue depth, number of threads, test mode, and block size. The storage device path refers to the hardware address or unique identifier of the storage device, used to inform the testing tool which storage device to test; that is, the storage device path specifies the target object for performance testing. Test runtime refers to the duration of a single test task (e.g., 60 seconds), used to ensure stable and reliable test results, avoid the influence of accidental factors due to excessively short runtime, and control the total test cycle to prevent excessively long test times. Queue depth refers to the maximum number of I / O requests simultaneously initiated to the storage device, used to measure the concurrent processing capacity of the storage device. By testing different queue depths, the performance inflection point of the storage device can be determined; the number of threads is used to simulate multiple concurrent applications or users simultaneously initiating I / O requests to the storage device, and is used to test the performance of the storage device in multi-tasking and multi-user scenarios. Combined with queue depth, it can comprehensively evaluate the performance of the storage device under high concurrency pressure; the test mode is used to define the types and mixing methods of read and write operations in the test, and is used to simulate different application scenarios, such as sequential read and write, random read and write, etc.; the block size is used to indicate the size of the data block operated by each independent I / O request (e.g., bs=4k), and is used to determine whether the performance test of the storage device is biased towards the number of input / output operations per second or the amount of data transferred per second.

[0048] S202. Based on the test parameters, generate at least one combination of test parameters, wherein each combination of test parameters corresponds to a test task of at least one test device in a test scenario with a test mode, a queue depth, a number of threads, and a block size.

[0049] Optionally, user-inputted test parameters can be combined to simulate all test scenarios that these parameters can indicate, thereby achieving comprehensive performance testing. Each combination of test parameters indicates a test application scenario, and each combination of test parameters indicates a test task for at least one storage device in a test environment with a test mode, a queue depth, a number of threads, and a block size.

[0050] S203. Generate test configuration data corresponding to each combination of test parameters. The test configuration data includes global parameters and test task parameters.

[0051] Optionally, based on the parameter values ​​such as test mode, queue depth, number of threads, block size, and storage device path included in each test parameter combination, the standard test template of the testing tool is populated to obtain the test configuration data corresponding to the test parameter combination. The test configuration data contains all the instructions and test parameters required by the testing tool to execute performance tests. The test configuration data consists of different configuration sections and is used to standardize the test environment.

[0052] Specifically, test configuration data ensures that each test by the testing tool can be executed under the same underlying conditions, and can also convert user-defined test scenarios into test instructions that the testing tool can recognize; and by saving test configuration data, it ensures the reproducibility and verifiability of test results.

[0053] In addition, global parameters define the basic operating environment and underlying mechanism of the testing tool. Global parameters include: I / O engine, direct I / O mode, total amount of test data, single test run time, etc. Global parameters are effective for all test tasks, establishing a unified, reliable and professional benchmark testing environment for all test tasks.

[0054] Furthermore, the test task parameters define the characteristics and behaviors of a specific workload. These parameters include test mode, block size, queue depth, and thread depth, and are used to simulate real-world application scenarios.

[0055] S204. Based on the test configuration data corresponding to each test parameter combination, call the preset test tools to execute test actions, and continuously monitor the test process corresponding to each test parameter combination to determine the test results and test error information corresponding to each test parameter combination.

[0056] The preset test tool can be implemented by the FIO tool, and this application does not make any specific restrictions on it.

[0057] Optionally, based on the test configuration data corresponding to each test parameter combination, preset test tools are invoked to perform performance tests on at least one storage device included in each test parameter combination, and the test process corresponding to each test parameter combination is continuously monitored to determine the test results and test error information corresponding to each test parameter combination. Here, one test parameter combination corresponds to one test application scenario; the test results are used to indicate the performance performance of each storage device when the test tool simulates the test application scenario indicated by the test parameter combination to perform performance tests on at least one storage device included in the test parameter combination; the test error information is used to indicate whether the test was successful or not when the test tool simulates the test application scenario indicated by the test parameter combination to perform performance tests on at least one storage device included in the test parameter combination.

[0058] S205. Generate and display a test report based on the test configuration data, test results, and test error information corresponding to each combination of test parameters.

[0059] Optionally, a comprehensive test report can be generated based on the test configuration data, test results, and test error information corresponding to each combination of test parameters.

[0060] In this embodiment, user-inputted test parameters are acquired and intelligently combined to obtain multiple test parameter combinations that comprehensively cover the user's input. For each test parameter combination, corresponding test configuration data is constructed, and test tools are invoked to execute test operations based on the test configuration data. The test process corresponding to each test parameter combination is monitored in real time to obtain the test results for each test parameter and record test error information during the test process. A comprehensive test report is generated based on the test configuration data, test results, and test error information corresponding to each test parameter combination. Specifically, after the user inputs test parameters, the test parameters are intelligently combined to obtain multiple comprehensive test parameter combinations without manual intervention. Each test parameter performs performance tests on multiple storage devices in parallel using key parameters such as thread count and queue depth. The test process and results of multiple storage devices can also be analyzed in parallel, and the complex test result text is converted into user-readable charts and data. This simplifies the performance testing process for storage devices, supports parallel testing of multiple storage devices, and provides visualized analysis of the test results.

[0061] In an optional implementation, the storage device performance testing method provided in this application further includes: When the user-inputted test parameters are not obtained, the pre-stored reference test parameters are retrieved. The reference test parameters include: a set of reference values ​​for queue depth, a set of reference values ​​for the number of threads, and a set of reference values ​​for block size.

[0062] Optionally, when the user does not input test parameters, the reference test parameters pre-stored in the user layer are directly invoked. These reference test parameters include: a queue depth reference value set, a thread count parameter value set, and a block size reference value set. Specifically, the queue depth reference value set can be [1, 8, 64], the thread count parameter value set can be [1, 4, 16], the test mode reference value set can be ['randread'], and the block size reference value set can be ['4k']. This application does not impose specific limitations on these values.

[0063] In one alternative implementation, refer to Figure 3 The specific operation of step S202 above can be as follows: S301. Parse the test parameters to obtain the corresponding set of queue depth values, thread count values, test mode values, and block size values.

[0064] Optionally, the user-inputted test parameters are parsed to determine the corresponding sets of queue depth values, thread count values, test mode values, and block size values. The queue depth value set includes a full range of queue depths, such as queue depth 1, queue depth 8, and queue depth 64; the thread count value set includes a full range of thread counts, such as thread count 1, thread count 4, and thread count 16; the test mode value set includes a full range of test modes, such as test mode read, test mode write, and test mode mixed read / write; the block size value set includes a full range of block sizes, such as block size 4k, etc., and this application does not impose specific limitations on these.

[0065] S302. Perform Cartesian product calculation on the set of queue depth values, the set of thread count values, the set of test mode values, and the set of block size values ​​to obtain at least one combination of test parameters.

[0066] Optionally, the Cartesian product calculation method is used to combine the set of queue depth values, the set of thread count values, the set of test mode values, and the set of block size values ​​to obtain multiple test parameter combinations that fully cover the test application scenario.

[0067] In one alternative implementation, see [link to implementation details]. Figure 4 The specific operation of step S203 above can be as follows: S401. Based on the combination of various test parameters, construct the corresponding standard test configuration template and write global parameters into the standard test configuration template. The global parameters include: input / output engine, direct input / output test mode, and test data size.

[0068] Optionally, based on the parameter values ​​included in each test parameter combination, a standard test configuration template corresponding to each test parameter combination is constructed, and baseline global parameters are written into the standard test configuration template. These global parameters include: an input / output engine to define an I / O mechanism to fully utilize the concurrency capabilities of the storage device; a direct input / output test mode to bypass the operating system's page cache, measure the actual performance of the storage device, and eliminate memory cache interference; and a test data size to define the total amount of test data, ensuring that the test data is large enough to avoid the storage device cache affecting the final test results.

[0069] S402. Based on each combination of test parameters, determine whether each combination of test parameters includes multiple storage device paths.

[0070] Optionally, it can be determined whether each combination of test parameters includes a number of storage device paths, with each storage device path corresponding to a uniquely identified storage device.

[0071] S403. If so, add the corresponding test path configuration item for each storage device path in the standard test configuration template.

[0072] Optionally, if the test parameter combination includes multiple storage device paths, it means that multiple storage devices are being tested and compared in parallel under the test application environment corresponding to the test parameter combination. In this case, it is necessary to add a corresponding test path configuration item for each storage device.

[0073] S404. If not, add a test path configuration item to the standard test configuration template, and determine the test task parameters corresponding to each test parameter combination based on the combination of test parameters.

[0074] Optionally, if the test parameter combination contains only one storage device path, it means that the test application environment corresponding to the test parameter combination is to perform parallel testing and comparison on one storage device, and only the corresponding test path configuration item needs to be added for this one storage device.

[0075] S405. Embed the test task parameters into the standard test configuration template to obtain the test configuration data corresponding to each combination of test parameters.

[0076] Optionally, key parameters from the user-input test parameters can be embedded as test task parameters into a standard test configuration template. The test task parameters can specifically be test mode, block size, queue depth, number of threads, etc., and this application does not impose specific limitations on them.

[0077] In one alternative implementation, see [link to implementation details]. Figure 5 The specific operation of step S205 above can be as follows: S501. Analyze the test results corresponding to each combination of test parameters to obtain the key performance indicators corresponding to each combination of test parameters. The key performance indicators include: number of input / output operations, bandwidth, and latency.

[0078] Optionally, the test results corresponding to each combination of test parameters are analyzed, and key performance indicators are extracted from the analyzed test results. These key performance indicators are critical parameters used to indicate the performance of the storage device. These key performance indicators include: number of input / output operations, bandwidth, and latency. The number of input / output operations refers to the number of read / write commands that the storage device can process per second; bandwidth refers to the total amount of data successfully transferred per second; and latency refers to the time required for a single I / O request to be sent and completed.

[0079] S502. Based on the analytical test results and key performance parameters corresponding to each combination of test parameters, generate performance charts and summary tables for each storage device.

[0080] Optionally, performance charts and summary tables are generated based on the analyzed test results and key performance parameters corresponding to each combination of test parameters. The performance charts indicate the mapping relationship between key performance indicators and queue depth, while the summary tables are used for horizontal and vertical comparisons of key performance indicators.

[0081] S503. Generate a test report based on the performance charts and summary tables for each storage device, as well as the test configuration data, test results, and test error information for each combination of test parameters.

[0082] Optionally, the performance charts and summary tables corresponding to each storage device, as well as the test configuration data, test results, and test error information corresponding to each combination of test parameters, can be correlated to generate a comprehensive test report.

[0083] In one alternative implementation, see [link to implementation details]. Figure 6 The specific operation of step S501 above can be as follows: S601. Analyze the test results corresponding to the test parameter combination to determine whether the test mode corresponding to the test parameter combination is a mixed read / write mode.

[0084] Optionally, the test results corresponding to the test parameter combination are parsed to determine whether the test mode in the test parameter combination is a test mode that mixes read and write commands.

[0085] S602. If so, extract the read performance index, write performance index, bandwidth, and number of input / output operations from the parsed test results corresponding to the test parameter combination. The read performance index includes: read operation latency time and number of read input / output operations. The write performance index includes: write operation latency time and number of write input / output operations.

[0086] Optionally, read performance metrics are used to measure the performance of read commands on the storage device, and write performance metrics are used to measure the performance of write commands on the storage device. Specifically, read operation latency is the average time elapsed from when the system issues a read request to when it successfully receives the requested data, used to measure the speed at which the storage device responds to read commands; read I / O operation count refers to the number of read operations the storage device can complete per second, used to measure the storage device's concurrent processing capability and speed in handling random read requests; write operation latency is the average time elapsed from when the system issues a write request to when the storage device confirms that the data has been safely written, used to measure the storage device's speed in responding to write commands; and write I / O operation count is the number of write operations the storage device can complete per second, used to measure the storage device's concurrent processing capability and speed in handling random write requests.

[0087] S603. Based on the read performance indicators and write performance indicators, determine the latency corresponding to the test parameter combination, and based on the latency, bandwidth, and number of input / output operations, determine the key performance indicators corresponding to the test parameter combination.

[0088] In an optional implementation, the operation of "determining the latency time corresponding to the test parameter combination based on the read performance index and the write performance index" in step S603 above can specifically be as follows: Calculate the first product between the read operation delay time and the number of read input / output operations; Calculate the second product between the write operation latency and the number of write input / output operations; Calculate the first sum between the first product and the second product; Calculate the second sum between the number of read input / output operations and the number of write input / output operations; Calculate the ratio between the first sum and the second sum to determine the delay time corresponding to the combination of test parameters.

[0089] Optionally, the latency time corresponding to the combination of test parameters under the read-write mixed test mode can be calculated according to the following formula (1), which is as follows: T=(Rt×Ri+Wt×Wi) / (Ri+ Wi)(1) Where T refers to the delay time, Rt refers to the read operation delay time, Ri refers to the number of read input / output operations, Wt refers to the write operation delay time, and Wi refers to the number of write input / output operations.

[0090] In one alternative implementation, see [link to implementation details]. Figure 7 The specific operation of step S502 above can be as follows: S701. Based on the parsed test results corresponding to each combination of test parameters, determine the storage device, test mode, and block size corresponding to each combination of test parameters.

[0091] Optionally, based on the parsed test results corresponding to each combination of test parameters, it is possible to determine which storage devices are specified by each combination of test parameters, what the test mode is, and the block size.

[0092] S702. Based on the storage device, test mode, and block size corresponding to each combination of test parameters, group the parsed test results and key performance parameters corresponding to each combination of test parameters to obtain the parsed test results and key performance parameters corresponding to each storage device.

[0093] Optionally, the parsed test results and key performance parameters can be grouped according to the storage device, test mode and block size corresponding to each combination of test parameters. This will yield the parsed test results and key performance parameters for each storage device. Cluster analysis can then be performed on the parsed test results and key performance parameters for each storage device to obtain the performance charts and summary tables for each storage device.

[0094] S703. Based on the parsed test results and key performance parameters of each storage device, generate performance charts and summary tables for each storage device. The performance charts include: input / output operation count - queue depth change curve, bandwidth - queue depth change curve, and latency - queue depth change curve. The summary table is used to record the key performance parameters corresponding to each combination of thread count and queue depth.

[0095] Optionally, the input / output operation count-queue depth variation curve is used to indicate the mapping relationship between the input / output operation count and the queue depth; the bandwidth-queue depth variation curve is used to indicate the mapping relationship between bandwidth and queue depth; and the latency-queue depth variation curve is used to indicate the mapping relationship between read / write operation latency and queue depth. See also Figure 8 This is used to characterize the mapping relationship between the read / write operation latency of a storage device and the queue depth. The horizontal axis represents the change in queue depth, and the vertical axis represents the change in read / write operation latency. The latency points corresponding to each queue depth are marked on the change curve.

[0096] Additionally, see Figure 9 The summary table is used to record the read / write latency of each storage device for each number of threads.

[0097] S704. Generate and display a test report based on the test configuration data, test results, and test error information corresponding to each combination of test parameters.

[0098] The following describes the apparatus, equipment, and computer-readable storage medium used to implement the storage device performance testing method provided in this application. The specific implementation process and technical effects are described above and will not be repeated below.

[0099] Figure 10 This is a schematic diagram of a storage device performance testing apparatus provided in this application. See also... Figure 10 The device includes: The parameter acquisition module 1001 is used to acquire the test parameters input by the user. The test parameters include: storage device path, test run time, queue depth, number of threads, test mode, and block size. The parameter combination module 1002 is used to generate at least one combination of test parameters based on the test parameters, wherein each combination of test parameters corresponds to a test mode, a queue depth, a number of threads, and a block size for a storage device. The configuration generation module 1003 is used to generate test configuration data corresponding to each combination of test parameters. The test configuration data includes global parameters and test task parameters. The test execution module 1004 is used to call preset test tools to execute test actions based on the test configuration data corresponding to each test parameter combination, and continuously monitor the test process corresponding to each test parameter combination to determine the test results and test error information corresponding to each test parameter combination. The result parsing module 1005 is used to generate and display a test report based on the test configuration data, test results, and test error information corresponding to each combination of test parameters.

[0100] Optionally, the parameter acquisition module 1001 is further configured to: retrieve pre-stored reference test parameters when no user-inputted test parameters are obtained, including: a set of reference values ​​for queue depth, a set of reference values ​​for the number of threads, and a set of reference values ​​for block size.

[0101] Optionally, the parameter combination module 1002 is specifically used to: parse the test parameters to obtain the set of queue depth values, the set of thread count values, the set of test mode values, and the set of block size values ​​corresponding to the test parameters; and perform Cartesian product calculation on the set of queue depth values, the set of thread count values, the set of test mode values, and the set of block size values ​​to obtain at least one combination of test parameters.

[0102] Optionally, the configuration generation module 1003 is specifically used to: construct a corresponding standard test configuration template based on each combination of test parameters, and write global parameters into the standard test configuration template, including: input / output engine, direct input / output test mode, and test data size; determine whether each combination of test parameters contains multiple storage device paths; if so, add corresponding test path configuration items for each storage device path in the standard test configuration template; if not, add a test path configuration item in the standard test configuration template, and determine the test task parameters corresponding to each combination of test parameters based on each combination of test parameters, and embed the test task parameters into the standard test configuration template to obtain the test configuration data corresponding to each combination of test parameters.

[0103] Optionally, the result parsing module 1005 is specifically used to: parse the test results corresponding to each combination of test parameters to obtain the key performance indicators corresponding to each combination of test parameters, including: the number of input / output operations, bandwidth, and latency; generate performance charts and summary tables for each storage device based on the parsed test results and key performance parameters corresponding to each combination of test parameters; and generate a test report based on the performance charts and summary tables for each storage device, as well as the test configuration data, test results, and test error information corresponding to each combination of test parameters.

[0104] Optionally, the result parsing module 1005 is further configured to: parse the test results corresponding to the test parameter combination to determine whether the test mode corresponding to the test parameter combination is a mixed read / write mode; if so, extract the read performance indicators, write performance indicators, bandwidth, and number of input / output operations from the parsed test results corresponding to the test parameter combination, wherein the read performance indicators include: read operation latency time and number of read input / output operations, and the write performance indicators include: write operation latency time and number of write input / output operations; determine the latency time corresponding to the test parameter combination based on the read performance indicators and write performance indicators, and determine the key performance indicators corresponding to the test parameter combination based on the latency time, bandwidth, and number of input / output operations.

[0105] Optionally, the result parsing module 1005 is further configured to: calculate the first product between the read operation delay time and the number of read input / output operations; calculate the second product between the write operation delay time and the number of write input / output operations; calculate the first sum between the first product and the second product; calculate the second sum between the number of read input / output operations and the number of write input / output operations; and calculate the ratio between the first sum and the second sum to determine the delay time corresponding to the test parameter combination.

[0106] Optionally, the result parsing module 1005 is further configured to: determine the storage device, test mode, and block size corresponding to each test parameter combination based on the parsed test results corresponding to each test parameter combination; group the parsed test results and key performance parameters corresponding to each test parameter combination based on the storage device, test mode, and block size corresponding to each test parameter combination to obtain the parsed test results and key performance parameters corresponding to each storage device; and generate performance charts and summary tables corresponding to each storage device based on the parsed test results and key performance parameters corresponding to each storage device. The performance charts include: input / output operation count - queue depth change curve, bandwidth - queue depth change curve, and latency - queue depth change curve. The summary table is used to record the key performance parameters corresponding to each combination of thread count and queue depth.

[0107] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0108] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0109] Figure 11 This is a schematic diagram of the structure of a computer device provided in this application. See also: Figure 11 The computer device includes: a memory 1101 and a processor 1102. The memory 1101 stores a computer program that can run on the processor 1102. When the processor 1102 executes the computer program, it implements the steps in any of the above method embodiments.

[0110] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the various method embodiments described above.

[0111] Optionally, this application also provides a program product, such as a computer-readable storage medium, including a program that, when executed by a processor, performs any of the above-described storage device performance testing method embodiments.

[0112] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0115] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute partial steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0117] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for testing the performance of a storage device, characterized in that, The method includes: Obtain test parameters input by the user, wherein the test parameters include: storage device path, test run time, queue depth, number of threads, test mode, and block size; Based on the test parameters, at least one combination of test parameters is generated, wherein each combination of test parameters corresponds to a test task of at least one test device in a test scenario with a test mode, a queue depth, a number of threads, and a block size. Generate test configuration data corresponding to each combination of test parameters. The test configuration data includes global parameters and test task parameters. Based on the test configuration data corresponding to each test parameter combination, the preset test tools are called to execute test actions, and the test process corresponding to each test parameter combination is continuously monitored to determine the test results and test error information corresponding to each test parameter combination. Based on the test configuration data, test results, and test error information corresponding to each combination of test parameters, a test report is generated and displayed.

2. The storage device performance testing method according to claim 1, characterized in that, The method further includes: When no user-inputted test parameters are obtained, pre-stored reference test parameters are retrieved. These reference test parameters include: a set of reference values ​​for queue depth, a set of reference values ​​for the number of threads, and a set of reference values ​​for block size.

3. The storage device performance testing method according to claim 1, characterized in that, The step of generating a test parameter matrix based on the test parameters includes: The test parameters are parsed to obtain the corresponding sets of queue depth values, thread count values, test mode values, and block size values. The Cartesian product of the set of queue depth values, the set of thread count values, the set of test mode values, and the set of block size values ​​is calculated to obtain the at least one combination of test parameters.

4. The storage device performance testing method according to claim 1, characterized in that, The generation of test configuration data corresponding to each combination of test parameters includes: Based on the combination of various test parameters, a corresponding standard test configuration template is constructed, and global parameters are written into the standard test configuration template. The global parameters include: input / output engine, direct input / output test mode, and test data size. Based on each combination of test parameters, determine whether each combination of test parameters includes multiple storage device paths; If yes, then add a corresponding test path configuration item for each storage device path in the standard test configuration template; if no, then add a test path configuration item in the standard test configuration template, and determine the test task parameters corresponding to each test parameter combination according to each test parameter combination, and embed the test task parameters into the standard test configuration template to obtain the test configuration data corresponding to each test parameter combination.

5. The storage device performance testing method according to claim 1, characterized in that, The process of generating and displaying a test report based on the test configuration data, test results, and test error information corresponding to each combination of test parameters includes: The test results corresponding to each combination of test parameters are analyzed to obtain the key performance indicators corresponding to each combination of test parameters. The key performance indicators include: number of input and output operations, bandwidth, and latency. Based on the analytical test results and key performance parameters corresponding to each combination of test parameters, generate performance charts and summary tables for each storage device. A test report is generated based on the performance charts and summary tables for each storage device, as well as the test configuration data, test results, and test error information for each combination of test parameters.

6. The storage device performance testing method according to claim 5, characterized in that, The process of analyzing the test results corresponding to each combination of test parameters to obtain the key performance indicators corresponding to each combination of test parameters includes: Analyze the test results corresponding to the test parameter combination to determine whether the test mode corresponding to the test parameter combination is a mixed read / write mode; If so, then from the parsed test results corresponding to the test parameter combination, extract the read performance index, write performance index, bandwidth, and number of input / output operations respectively. The read performance index includes: read operation latency time and number of read input / output operations. The write performance index includes: write operation latency time and number of write input / output operations. Based on the read performance metrics and the write performance metrics, the latency time corresponding to the test parameter combination is determined, and based on the latency time, the bandwidth, and the number of input / output operations, the key performance metrics corresponding to the test parameter combination are determined.

7. The storage device performance testing method according to claim 6, characterized in that, The step of determining the latency corresponding to the test parameter combination based on the read performance metric and the write performance metric includes: Calculate the first product between the read operation delay time and the number of read input / output operations; Calculate the second product between the write operation delay time and the number of write input / output operations; Calculate the first sum between the first product and the second product; Calculate the second sum between the number of read input / output operations and the number of write input / output operations; Calculate the ratio between the first sum and the second sum to determine the delay time corresponding to the combination of test parameters.

8. The storage device performance testing method according to claim 5, characterized in that, The process involves generating performance charts and summary tables for each storage device based on the analyzed test results and key performance parameters corresponding to each combination of test parameters, including: Based on the parsed test results corresponding to each combination of test parameters, determine the storage device, test mode, and block size corresponding to each combination of test parameters; Based on the storage device, test mode, and block size corresponding to each combination of test parameters, the parsed test results and key performance parameters corresponding to each combination of test parameters are grouped to obtain the parsed test results and key performance parameters corresponding to each storage device. Based on the parsed test results and key performance parameters of each storage device, performance charts and summary tables are generated for each storage device. The performance charts include: input / output operation count-queue depth change curve, bandwidth-queue depth change curve, and latency-queue depth change curve. The summary table is used to record the key performance parameters corresponding to each combination of thread count and queue depth.

9. A storage device performance testing apparatus, characterized in that, The device includes: The parameter acquisition module is used to acquire test parameters input by the user, wherein the test parameters include: storage device path, test run time, queue depth, number of threads, test mode, and block size; The parameter combination module is used to generate at least one combination of test parameters based on the test parameters, wherein each combination of test parameters corresponds to a test mode, a queue depth, a number of threads, and a block size for a storage device. The configuration generation module is used to generate test configuration data corresponding to each combination of test parameters. The test configuration data includes global parameters and test task parameters. The test execution module is used to call preset test tools to execute test actions based on the test configuration data corresponding to each combination of test parameters, and continuously monitor the test process corresponding to each combination of test parameters to determine the test results and test error information corresponding to each combination of test parameters. The results parsing module is used to generate and display test reports based on the test configuration data, test results, and test error information corresponding to each combination of test parameters.

10. A computer device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program that can run on the processor, and when the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images