Method and device for automatically testing performance of distributed storage system

Abstract

The invention relates to a distributed storage system performance automatic testing method and device. In response to a preset test configuration interface called by a user, automatically identifying test parameter information in a test task set input by the user to generate a plurality of task definition files executable by a benchmark test tool of the distributed storage system; submitting the task definition file to a benchmark test tool, and monitoring the execution state of the corresponding test task based on the plurality of task identifiers; and after all the test tasks are executed, automatically carrying out aggregation analysis processing on the test result data, and generating a structured test report. By integrating the generation, submission, monitoring and analysis processes of the configuration file and automatically summarizing and analyzing the test results, the full-process automatic processing is realized; and meanwhile, batch submission of test tasks is supported, the test throughput is improved, the technical threshold and the labor cost of the test are remarkably reduced, and the test efficiency and accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of computers, and in particular to an automated testing method and apparatus for the performance of a distributed storage system. Background Technology

[0002] With the development of the Internet, distributed storage has become a core component of cloud infrastructure. The performance of distributed storage directly affects the experience and cost of upper-layer applications. Existing technologies generally use standardized benchmark testing tools to test and evaluate the performance of distributed storage systems.

[0003] However, existing performance testing methods rely heavily on manually writing configuration files, which makes it difficult to manage a large number of test cases and has high maintenance costs. Testers need to manually identify, extract, and summarize key indicators in the test results, which is labor-intensive and prone to errors, thus affecting the accuracy and reliability of the test results. Summary of the Invention

[0004] This application provides a method and apparatus for automated performance testing of distributed storage systems, which solves the technical problem that existing performance testing methods for distributed storage systems rely on testers manually writing and maintaining configuration files and manually identifying and summarizing key indicators in test results, resulting in high testing costs and reduced testing efficiency.

[0005] Firstly, this application provides an automated performance testing method for a distributed storage system, the method comprising: In response to a user's call to a pre-defined test configuration interface, obtain the set of test tasks input by the user based on the test configuration interface; The test task set is parsed to obtain multiple test parameter information, and multiple task definition files that can be executed by the benchmark testing tool of the distributed storage system are generated based on the test parameter information. The multiple task definition files are submitted to the benchmark testing tool, and the multiple task identifiers returned by the benchmark testing tool are received. The execution status of the corresponding test tasks is monitored based on the multiple task identifiers; wherein, the test task is the test task defined in the task definition file. After all test tasks have been completed, the raw result data corresponding to each task identifier is obtained, the raw result data is aggregated and analyzed, and a structured test report is generated.

[0006] In one possible implementation, the test task set is in configuration file format or command line parameter format; The test task set includes one or more test tasks, wherein each test task includes at least: operation type, object size, number of objects, concurrency, and runtime parameters.

[0007] In one possible implementation, generating multiple task definition files executable by the benchmark testing tool for the distributed storage system based on the test parameter information includes: For each test parameter, perform the following processing to generate multiple task definition files that can be executed by the benchmark tool for the distributed storage system: The test parameter information is parsed to obtain the number of endpoint addresses, total concurrency, and operation type; Based on the number of endpoint addresses and the total concurrency, determine the concurrency corresponding to each endpoint address; Based on the operation type and the concurrency corresponding to each endpoint address, generate the corresponding executable task definition file for the benchmark tool of the distributed storage system.

[0008] In one possible implementation, generating the corresponding executable task definition file for the distributed storage system benchmark tool based on the operation type and the concurrency corresponding to each endpoint address includes: When the operation type includes a read operation, determine the data preparation stage information of the current test task, and write the data preparation stage information and the concurrency number corresponding to each endpoint address into the task definition file executable by the benchmark testing tool of the distributed storage system. When the operation type is a mixed read-write operation, the number of read operation nodes and write operation nodes are determined according to a pre-set ratio rule, and the number of read operation nodes, the number of write operation nodes, and the concurrency corresponding to each endpoint address are written into the task definition file executable by the benchmark tool of the distributed storage system.

[0009] In one possible implementation, the aggregation analysis of the raw result data includes: Extract key performance indicators from the raw result data; wherein, the key performance indicators include at least object size and concurrency. The original result data is classified in multiple dimensions according to the object size, concurrency, and operation type to obtain the classification results after aggregation analysis.

[0010] In one possible implementation, the step of classifying the original result data in multiple dimensions according to the object size, concurrency, and operation type to obtain the classification result after aggregation analysis includes: The objects are classified based on their size units to obtain a first classification result; The first classification result is further classified based on the numerical value of the object size to obtain a second classification result; The second classification result is classified three times based on the value of the concurrency number to obtain the classification result after aggregation analysis. Calculate the comprehensive performance indexes for each read operation type and write operation type, and classify them according to read operation type and write operation type.

[0011] In one possible implementation, generating a structured test report includes: Based on the classification results of the original result data, a first report containing data details and a formatted summary report showing the performance index comparison under different combinations of object size and concurrency are generated.

[0012] Secondly, this application provides an automated performance testing device for a distributed storage system, the device comprising: The test task set acquisition module is used to acquire the test task set input by the user based on the test configuration interface in response to the user calling the pre-set test configuration interface; The task definition file generation module is used to parse the test task set, obtain multiple test parameter information, and generate multiple task definition files that can be executed by the benchmark testing tool of the distributed storage system based on the test parameter information. The task submission module is used to submit the multiple task definition files to the benchmark testing tool, receive multiple task identifiers returned by the benchmark testing tool, and monitor the execution status of the corresponding test tasks based on the multiple task identifiers; wherein, the test task is the test task defined in the task definition file; The test report generation module is used to obtain the raw result data corresponding to each task identifier after all test tasks have been executed, perform aggregation analysis on the raw result data, and generate a structured test report.

[0013] In one possible implementation, the test task set is in configuration file format or command line parameter format; The test task set includes one or more test tasks, wherein each test task includes at least: operation type, object size, number of objects, concurrency, and runtime parameters.

[0014] In one possible implementation, the task definition file generation module includes: The task definition file generation unit performs the following processing for each test parameter information to generate multiple task definition files executable by the benchmark tool for the distributed storage system: The test parameter parsing subunit is used to parse the test parameter information to obtain the number of endpoint addresses, the total number of concurrent connections, and the operation type; The concurrency determination subunit is used to determine the concurrency corresponding to each endpoint address based on the number of endpoint addresses and the total concurrency. The task definition file generation subunit is used to generate an executable task definition file for the benchmark tool of the corresponding distributed storage system based on the operation type and the concurrency corresponding to each endpoint address.

[0015] In one possible implementation, the task definition file generation subunit is specifically used for: When the operation type includes a read operation, determine the data preparation stage information of the current test task, and write the data preparation stage information and the concurrency number corresponding to each endpoint address into the task definition file executable by the benchmark testing tool of the distributed storage system. When the operation type is a mixed read-write operation, the number of read operation nodes and write operation nodes are determined according to a pre-set ratio rule, and the number of read operation nodes, the number of write operation nodes, and the concurrency corresponding to each endpoint address are written into the task definition file executable by the benchmark tool of the distributed storage system.

[0016] In one possible implementation, the test report generation module includes: A performance metric extraction unit is used to extract key performance metrics from the raw result data; wherein, the key performance metrics include at least object size and concurrency. The classification result generation unit is used to classify the original result data in multiple dimensions according to the object size, concurrency, and operation type to obtain the classification result after aggregation analysis.

[0017] In one possible implementation, the classification result generation unit is specifically used for: The objects are classified based on their size units to obtain a first classification result; The first classification result is further classified based on the numerical value of the object size to obtain a second classification result; The second classification result is classified three times based on the value of the concurrency number to obtain the classification result after aggregation analysis. Calculate the comprehensive performance indexes for each read operation type and write operation type, and classify them according to read operation type and write operation type.

[0018] In one possible implementation, the test report generation module is specifically used for: Based on the classification results of the original result data, a first report containing data details and a formatted summary report showing the performance index comparison under different combinations of object size and concurrency are generated.

[0019] Thirdly, this application provides an electronic device, including a processor and a memory, wherein the processor is configured to execute a performance testing program stored in the memory to implement the distributed storage system performance automated testing method described in any one of the first aspects.

[0020] Fourthly, this application provides a storage medium storing one or more programs, which can be executed by one or more processors to implement the distributed storage system performance automation testing method described in any one aspect.

[0021] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application responds to a user's call to a pre-set test configuration interface, obtains the test task set input by the user based on the test configuration interface, automatically identifies the test parameter information in the test task set, generates multiple task definition files executable by the benchmark testing tool of the distributed storage system, and submits the task definition files to the benchmark testing tool. It then receives multiple task identifiers returned by the benchmark testing tool and monitors the execution status of the corresponding test tasks based on these multiple task identifiers. After all test tasks are completed, the method automatically aggregates and analyzes the raw result data corresponding to each task identifier and generates a structured test report. By integrating the generation, submission, monitoring, and analysis processes of configuration files and automating the summary and analysis of test results, the method achieves fully automated processing without manual intervention, significantly saving testers' time. It also supports batch submission of test tasks, improving test throughput. Furthermore, it significantly reduces the technical threshold and labor costs of testing, improving testing efficiency and accuracy. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0025] Figure 1A flowchart of an automated performance testing method for a distributed storage system provided in this application embodiment; Figure 2 A flowchart illustrating an embodiment of another performance testing method provided in this application; Figure 3 A flowchart illustrating another embodiment of the performance testing method provided in this application; Figure 4 A structural block diagram of an automated performance testing device for a distributed storage system provided in this application embodiment; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0028] To address the technical problem that existing performance testing methods for distributed storage systems rely on testers manually writing and maintaining configuration files, and manually identifying and summarizing key metrics in test results, leading to high testing costs and reduced efficiency, this application provides a performance testing method, apparatus, electronic device, and storage medium. By integrating the generation, submission, monitoring, and analysis processes of configuration files, and automating the summarization and analysis of test results, it achieves fully automated processing without manual intervention, significantly saving testers' time. It also supports batch submission of test tasks, increasing test throughput. Furthermore, it significantly reduces the technical threshold and labor costs of testing, improving testing efficiency and accuracy.

[0029] Figure 1 A flowchart of an automated performance testing method for a distributed storage system provided in this application embodiment is shown below. Figure 1 As shown, the execution entity is a distributed storage test system, and the process includes the following steps: Step 101: In response to the user's call to the pre-defined test configuration interface, obtain the test task set input by the user based on the test configuration interface.

[0030] The test configuration interface provides a unified entry point for users of the distributed storage testing system to receive test commands input by users. Essentially, it's an input point for test commands, which can be a web form, a JSON-formatted configuration template, command-line commands, etc., and this application embodiment does not impose any limitations on this.

[0031] A test task set can be a collection of files containing one or more test requirements or test parameter information. Essentially, it is not a single test task, but rather a collection of test plans. For example, it can contain one or more test cases, each explicitly defining a complete test scenario.

[0032] In one embodiment, if it is detected that a user has performed an input operation in the corresponding test configuration interface, or triggered the start operation of the test configuration interface, then the test task set input by the user through the test configuration interface is obtained. The test task set can be in configuration file format or command-line parameter format; the test task set includes one or more test tasks, wherein each test task includes at least: operation type, object size, number of objects, concurrency, and runtime parameters.

[0033] For example, a user can first write a JSON-formatted configuration file, within which multiple test tasks (test cases) can be defined. The first test task might be to test the write performance of a file under 50 concurrent users for 5 minutes. The second test task could be to test the performance of a 1MB file in a distributed storage system under a mixed read / write scenario with 100 concurrent users for 10 minutes. The user then selects the file path corresponding to the configuration file through the interface (i.e., calls the pre-defined test configuration interface), and the system reads and parses the file. Alternatively, the user can directly input relevant command-line parameters in the corresponding test configuration interface. Upon detecting user input or upload operations, the system automatically parses these command-line parameters, ultimately obtaining a structured set of test tasks containing one or more test cases.

[0034] Step 102: Parse the test task set to obtain multiple test parameter information, and generate multiple task definition files that can be executed by the benchmark testing tool for the distributed storage system based on the test parameter information.

[0035] Based on the above steps, a test task set can contain one or more test parameter information, with each test parameter corresponding to a test task. For example, a test task set in JSON format is shown below: { "Test Task Set Name": "Distributed Storage Performance Test - 20251125" "Test Task List":[ { Task Name: 100KB File Write Performance Test Operation type: "Write" "Object size": "100KB" "Number of objects": 10000 "Concurrency": 50 "Running time": "10 minutes" }, { Task Name: 1GB File Read Performance Test Operation type: Read Object size: 1GB "Number of objects": 100 "Concurrency": 20 "Running time": 15 minutes }, { Task Name: Mixed Read / Write (7:3) Test Operation type: Mixed read / write "Reading-to-writing ratio": "70% reading: 30% writing" "Object size": "500MB" Number of objects: 500 Concurrency: 80 Runtime: 20 minutes } ] }, This test task set contains multiple test parameter information, and each test parameter information corresponds to a test task.

[0036] Benchmarking tools can refer to commonly used distributed storage performance testing tools, such as COSBench (Cloud Object Storage Benchmark) and FIO (Flexible I / O Tester).

[0037] The task definition file serves as a bridge between the distributed storage performance testing system and the underlying benchmarking tools (such as COSBench). It is typically an XML configuration file whose syntax and structure must fully conform to the requirements of the underlying testing tools, precisely instructing them on what type of load to initiate to ensure load balancing.

[0038] Load balancing refers to the system's ability to intelligently distribute the total number of concurrent users across multiple storage access endpoints in a test environment. This ensures a balanced test load, preventing one endpoint from becoming overloaded while others remain idle, thus guaranteeing the accuracy and representativeness of the test results. The specific methods for determining the concurrency levels for each endpoint will be described in detail in the relevant embodiments below.

[0039] In one embodiment, after obtaining the test task set, the testing system of this application does not execute it directly. Instead, it automatically and intelligently translates the test task set into a task definition file that can be executed by the COSBench benchmark testing tool, and generates a corresponding task definition file for each test parameter. The aforementioned task definition file can be an XML configuration file.

[0040] For example, the system's parameter parsing module reads the test task set to obtain test parameter information. Some parameters in the test parameter information are then formatted, for example, converting 10 minutes into 600 seconds that the benchmark tool can understand, and default values ​​are added, such as defaulting to HTTP protocol and 3 test endpoints. Finally, the format-converted file is validated according to preset validation rules. These validation rules can verify that the number of objects in the test parameter information does not exceed 1 million; this is merely an example, and this embodiment does not impose any limitations. Furthermore, after successful validation, the XML generation module, according to the benchmark tool's specifications, fills the processed test parameters into the corresponding XML template to generate an XML configuration file (i.e., a task definition file). Step 103: Submit multiple task definition files to the benchmark tool, receive multiple task identifiers returned by the benchmark tool, and monitor the execution status of the corresponding test tasks based on the multiple task identifiers; wherein, the test task is the test task defined in the task definition file.

[0041] A task identifier can be understood as a unique ID returned by the testing tool after each task definition file is submitted to it. This ID is like a tracking number for a package; the system subsequently uses it to query the execution status and results of this specific test task.

[0042] In one embodiment, the system uses a task submission module to send all XML files in batches to the submission interface of the benchmark tool, receives task identifiers returned by the benchmark tool, and associates the task identifiers with the corresponding test tasks. Then, a separate monitoring thread is started to periodically query the status of all tasks until all tasks are completed.

[0043] For example, the system synchronously submits the three generated task definition files to the corresponding benchmark testing tool in batches, and receives three task identifiers returned by the tool. Every 30 or 60 seconds, the system queries the testing tool for the corresponding task status based on the task identifier, until all tasks are in the "completed" or "terminated" state. In this way, users do not need to manually poll the commands at the computer, greatly freeing up manpower and realizing one-click unattended execution of the test task queue.

[0044] Step 104: After all test tasks have been completed, obtain the raw result data corresponding to each task identifier, perform aggregation analysis on the raw result data, and generate a structured test report.

[0045] The raw results data can be detailed performance test data generated by the underlying testing tools after each task is completed. This data is usually stored in formats such as CSV files (Comma-Separated Values), containing very fine-grained information, such as throughput per second and latency for each operation. Although detailed, it is poorly readable and not suitable for direct reporting and analysis.

[0046] Aggregate analysis refers to the process of cleaning, classifying, statistically analyzing, and calculating the raw data results mentioned above. Its purpose is to extract key performance indicators from massive amounts of details and combine and sort them according to business logic.

[0047] A structured test report can be the final output generated after aggregate analysis. For example, it may include detailed data (such as in CSV format) that can be further processed by the program, and a human-readable formatted report (such as in Excel format) that includes charts and summary tables.

[0048] In one embodiment, once the monitoring thread confirms that all tasks have been completed, the system automatically locates the original result data corresponding to each test task based on the task identifier, extracts the core performance indicators from it, and intelligently classifies and sorts all the original result data according to predefined rules. Furthermore, for mixed read and write test tasks, the system can automatically separate the read and write indicators and calculate the comprehensive performance of read and write operations separately. Finally, a structured test report is generated.

[0049] For example, the raw result data may include a CSV file generated by the COSBench testing tool after each test task is completed, containing a detailed record of each operation (e.g., 2025-11-25, 10:05:30, read operation, 500MB, time taken 200ms, successful). Key performance indicators extracted from the raw result data may include bandwidth, average response time, number of operations per second, and operation success rate. Then, intelligent classification operations are performed according to the object size and concurrency in the raw result data. Finally, the analyzed and processed data is filled into the corresponding template to generate a structured test report.

[0050] The method provided in this application, in response to a user's call to a pre-defined test configuration interface, obtains the test task set input by the user based on the test configuration interface, automatically identifies the test parameter information in the test task set, generates multiple task definition files executable by the benchmark testing tool of the distributed storage system, and submits the task definition files to the benchmark testing tool. It then receives multiple task identifiers returned by the benchmark testing tool and monitors the execution status of the corresponding test tasks based on these task identifiers. After all test tasks have been executed, it automatically aggregates and analyzes the raw result data corresponding to each task identifier and generates a structured test report. By integrating the generation, submission, monitoring, and analysis processes of configuration files and automating the summary and analysis of test results, it achieves fully automated processing without manual intervention, significantly saving testers' time. It also supports batch submission of test tasks, improving test throughput. Furthermore, it significantly reduces the technical threshold and labor costs of testing, improving testing efficiency and accuracy.

[0051] Figure 2 A flowchart illustrating an embodiment of another performance testing method provided in this application is shown below. Figure 2 As shown, in Figure 1 Based on the illustrated process, this section mainly describes how to generate a task definition file based on test parameter information, including the following steps: Step 201: In response to the user's call to the pre-defined test configuration interface, obtain the test task set input by the user based on the test configuration interface.

[0052] Step 202: Analyze the test task set to obtain multiple test parameter information.

[0053] For steps 201-202 above, please refer to the above. Figure 1 Detailed description of the relevant embodiments.

[0054] Step 203: Perform the following processing for each test parameter information to generate multiple task definition files executable by the benchmark testing tool for the distributed storage system: Step 231: Parse the test parameter information to obtain the number of endpoint addresses, total concurrency, and operation type.

[0055] The number of endpoint addresses refers to the number of storage service access points configured in a distributed storage test environment. For example, a storage cluster may provide three different URLs (such as http: / / ep1.storage.com, http: / / ep2.storage.com, and http: / / ep3.storage.com) for load balancing, in which case the number of endpoint addresses is 3.

[0056] Total concurrency refers to the total number of simulated client worker threads in a test case. It represents the stress level of simultaneously making requests to the storage system. For example, a test case might be set to a total concurrency of 150.

[0057] Operation type refers to the basic mode of the test task, which essentially reflects the user's core actions and can include writing, reading, and mixed reading and writing. It determines what basic I / O operations the testing tool will perform.

[0058] In one embodiment, the number of endpoint addresses, total concurrency, and operation type corresponding to the current test task are extracted from the test parameter information to provide data support for the subsequent generation of task definition files.

[0059] Step 232: Based on the number of endpoint addresses and the total number of concurrent connections, determine the number of concurrent connections corresponding to each endpoint address.

[0060] The concurrency count for each endpoint address can refer to the total concurrency count, which is a global number, being reasonably distributed to each specific endpoint address, and each address being assigned a dedicated number of worker threads.

[0061] In one embodiment, the total number of concurrent connections is divided by the number of endpoint addresses to obtain a basic number of concurrent connections that should be allocated to each endpoint. Then, the remainders after dividing the total number of concurrent connections by the number of endpoint addresses are calculated. These remainders indicate that some concurrent connections cannot be distributed evenly. The system will allocate these remaining concurrent connections to the first few endpoints in sequence, increasing the number of concurrent connections for each endpoint by one.

[0062] For example, assuming a total concurrency of 150 and 3 endpoint addresses, first allocate 50 concurrent connections to each endpoint address. Since 150%3=0, there are no remaining concurrent connections, so set 50 concurrent connections for each endpoint. If the total concurrency is 152, allocate 50 concurrent connections to one endpoint and 51 concurrent connections to the other two endpoints. In this way, the test load is ensured to be distributed as evenly as possible across all storage nodes, thereby obtaining more representative and accurate overall system performance data.

[0063] Step 233: Based on the operation type and the number of concurrent connections corresponding to each endpoint address, generate the corresponding executable task definition file for the benchmark testing tool of the distributed storage system.

[0064] In one embodiment, when the operation type includes a read operation, the data preparation stage information of the current test task is determined, and the data preparation stage information and the concurrency corresponding to each endpoint address are written into the task definition file executable by the benchmark tool of the distributed storage system; when the operation type is a mixed read and write operation, the number of read operation nodes and write operation nodes are determined according to a pre-set ratio rule, and the number of read operation nodes, the number of write operation nodes, and the concurrency corresponding to each endpoint address are written into the task definition file executable by the benchmark tool of the distributed storage system.

[0065] The data preparation phase is a crucial preliminary step in read performance testing. Its purpose is to pre-write the data that the test will read into the storage system. Without this phase, the read test will fail because there is no data to read. Manually, testers must remember and manually configure this phase, while this embodiment automates the generation of data preparation phase information.

[0066] For example, if a read operation is detected in the current operation, a data preparation phase is automatically inserted into the task definition file. This phase is typically a separate preparatory phase with write operations as the primary operation. Its concurrency settings may differ from the main phase, but the system will use reasonable default values ​​or configure it according to specific rules. Subsequently, the actual read operation main phase is created, and the allocated concurrency is configured into it. The final generated XML task definition file will contain two consecutive working phases. The first phase initializes and writes data, and the second phase tests the read operation.

[0067] Furthermore, if the current operation is identified as a mixed read / write operation, the system will parse the read / write ratio attached to the user configuration, and then automatically create two parallel operation nodes within the same work phase of the task definition file: one node is marked as "read" with a ratio of 70; the other node is marked as "write" with a ratio of 30. The system will apply the total concurrency (allocated to each endpoint) to this mixed work phase, and the testing tool will automatically divide the concurrent threads into read threads and write threads according to the ratio. For example, for an endpoint allocated 50 concurrency, in the 70 / 30 mixed read / write mode, the testing tool will roughly simulate 35 threads performing read operations and 15 threads performing write operations (the specific scheduling is implemented by the testing tool), thereby accurately simulating the real business scenario expected by the user.

[0068] Step 204: Submit multiple task definition files to the benchmark tool, receive multiple task identifiers returned by the benchmark tool, and monitor the execution status of the corresponding test tasks based on the multiple task identifiers; wherein, the test task is the test task defined in the task definition file.

[0069] Step 205: After all test tasks have been completed, obtain the raw result data corresponding to each task identifier, perform aggregation analysis on the raw result data, and generate a structured test report.

[0070] For steps 204-205 above, please refer to the above. Figure 1 Detailed description of the relevant embodiments.

[0071] pass Figure 2 The detailed description of the illustrated embodiment shows that the load can be intelligently and evenly distributed to all test endpoints based on the number of endpoint addresses and the total concurrency. This ensures that the test pressure is applied evenly across the entire distributed storage cluster, resulting in performance data (such as total throughput and overall latency) that more accurately reflects the overall service capacity of the cluster and avoids misjudgments of performance bottlenecks due to uneven load distribution. Furthermore, the automated load balancing eliminates human configuration bias, making the comparative test results more fair and reliable. Simultaneously, the automated generation of task definition files is implemented. For test tasks that require frequent execution and have diverse parameters, this high degree of automation can save significant manpower and time costs.

[0072] Figure 3 A flowchart illustrating another embodiment of the performance testing method provided in this application is shown below. Figure 1 Based on the illustrated process, this section mainly describes how to perform aggregation analysis on the acquired raw result data, and how to generate a structured test report, such as... Figure 3 As shown, it includes the following steps: Step 301: In response to the user's call to the pre-defined test configuration interface, obtain the test task set input by the user based on the test configuration interface.

[0073] Step 302: Parse the test task set to obtain multiple test parameter information, and generate multiple task definition files that can be executed by the benchmark testing tool for the distributed storage system based on the test parameter information.

[0074] Step 303: Submit multiple task definition files to the benchmark testing tool, receive multiple task identifiers returned by the benchmark testing tool, and monitor the execution status of the corresponding test tasks based on the multiple task identifiers; wherein, the test task is the test task defined in the task definition file.

[0075] Step 304: After all test tasks have been completed, obtain the raw result data corresponding to each task identifier.

[0076] For steps 301-304 above, please refer to the above. Figure 1 Detailed description of the relevant embodiments.

[0077] Step 305: Extract key performance indicators from the raw result data; among them, key performance indicators include at least object size and concurrency.

[0078] Key performance indicators (KPIs) are core data that directly reflect the performance of distributed storage. They can include object size, concurrency, bandwidth (data transfer speed), average response time (time taken per operation), operation success rate, and OPS (operations per second). They are the core basis for evaluating storage performance.

[0079] In one embodiment, all raw result data is traversed, and the raw result data is automatically cleaned to filter out abnormal values. Key performance indicators are then selected and extracted from the cleaned raw result data. These key performance indicators may include object size, concurrency, operation type, and bandwidth, etc., but this embodiment does not impose any limitations on these aspects.

[0080] Step 306: Classify the original result data in multiple dimensions according to object size, concurrency, and operation type to obtain the classification results after aggregation analysis.

[0081] Multi-dimensional classification can refer to filtering and classifying data layer by layer according to three core dimensions: object size, concurrency, and operation type. For example, first classify by file unit, then by file size, and finally by concurrency, so that scattered data can be organized in an orderly manner according to rules.

[0082] In one embodiment, the objects are classified based on their size units to obtain a first classification result; the first classification result is further classified based on the numerical value of the object size to obtain a second classification result; the second classification result is further classified based on the numerical value of the number of concurrent operations to obtain a classification result after aggregation analysis; the comprehensive performance index corresponding to each read operation type and write operation type is calculated respectively, and the objects are classified according to the read operation type and write operation type.

[0083] Comprehensive performance metrics can refer to the results of statistical calculations on raw data of the same operation type (read / write) under the same combination of object size and concurrency, such as the average bandwidth of read operations and the average response time of write operations, which can intuitively reflect the overall performance in this scenario.

[0084] For example, data is split according to the size unit of the object (e.g., KB / MB / GB). For instance, all test data is divided into KB-level file groups, MB-level file groups, and GB-level file groups, resulting in the first classification result. Then, the data within each unit group is further classified, sorted and grouped by the numerical value of the object size from smallest to largest. For example, the MB-level file group is further split into 100MB, 500MB, and 1GB groups, resulting in the second classification result. Then, each numerical group is classified a third time, sorted and grouped by the numerical value of concurrency from smallest to largest. For example, the 500MB group is further split into 20-concurrency groups, 50-concurrency groups, and 80-concurrency groups, finally resulting in an ordered third classification result. If it is a mixed read / write operation, the system will also separately split the read and write operation data during the classification process, calculate the comprehensive performance indicators of the two types of operations (e.g., average bandwidth for read operations, average response time for write operations), and then classify them separately by operation type to avoid confusion between read and write data.

[0085] Step 307: Based on the classification results of the original result data, generate a first report containing data details, and a formatted summary report showing the performance index comparison under different combinations of object size and concurrency.

[0086] In one embodiment, based on the final classification results of the original data, two reports with different functions are automatically generated. The first report is a detailed data report, which fully retains all key performance indicators after extraction, with each record corresponding to a specific test operation, facilitating users' subsequent verification of the original data. The other report is a formatted summary report, which organizes the classified performance data into comparison tables or visualization charts, clearly showing the performance differences under different combinations of object sizes and concurrency.

[0087] pass Figure 3 The detailed description of the illustrated embodiment, based on a multi-level, multi-dimensional classification method, can automatically and intelligently sort and group massive amounts of raw test results according to object size units, object size values, and concurrency logic. It also automatically separates, calculates, and categorizes performance indicators for read and write operations, overcoming the challenges of manual classification, which requires manually distinguishing and calculating various read and write indicators separately from the same result file. This process is complex and prone to errors, leading to misjudgments of read and write performance. It can separately assess the true capabilities of the storage system in handling read and write traffic, which is crucial for optimizing storage system configurations (such as read / write cache ratios). It achieves completely unattended operation from user input of test intent to delivery of the final analysis report, making it a truly one-click performance testing solution. Furthermore, based on a dual-report output mechanism, non-technical users can quickly understand the performance advantages and disadvantages in different scenarios, fulfilling the dual needs of accurate traceability and rapid interpretation, thus enhancing the practical value of the test results.

[0088] Figure 4A structural block diagram of an automated performance testing device for a distributed storage system provided in this application is shown below. Figure 4 As shown, the device includes: The test task set acquisition module 41 is used to acquire the test task set input by the user based on the test configuration interface in response to the user calling the pre-set test configuration interface; The task definition file generation module 42 is used to parse the test task set, obtain multiple test parameter information, and generate multiple task definition files that can be executed by the benchmark testing tool of the distributed storage system based on the test parameter information. The task submission module 43 is used to submit the multiple task definition files to the benchmark testing tool, receive multiple task identifiers returned by the benchmark testing tool, and monitor the execution status of the corresponding test tasks based on the multiple task identifiers; wherein, the test task is the test task defined in the task definition file; The test report generation module 44 is used to obtain the original result data corresponding to each task identifier after all test tasks have been executed, perform aggregation analysis on the original result data, and generate a structured test report.

[0089] In one possible implementation, the test task set is in configuration file format or command line parameter format; The test task set includes one or more test tasks, wherein each test task includes at least: operation type, object size, number of objects, concurrency, and runtime parameters.

[0090] In one possible implementation, the task definition file generation module 42 includes: The task definition file generation unit performs the following processing for each test parameter information to generate multiple task definition files executable by the benchmark tool for the distributed storage system: The test parameter parsing subunit is used to parse the test parameter information to obtain the number of endpoint addresses, the total number of concurrent connections, and the operation type; The concurrency determination subunit is used to determine the concurrency corresponding to each endpoint address based on the number of endpoint addresses and the total concurrency. The task definition file generation subunit is used to generate an executable task definition file for the benchmark tool of the corresponding distributed storage system based on the operation type and the concurrency corresponding to each endpoint address.

[0091] In one possible implementation, the task definition file generation subunit is specifically used for: When the operation type is a read operation, determine the data preparation stage information of the current test task, and write the data preparation stage information and the concurrency number corresponding to each endpoint address into the task definition file executable by the benchmark testing tool of the distributed storage system. When the operation type is a mixed read-write operation, the number of read operation nodes and write operation nodes are determined according to a pre-set ratio rule, and the number of read operation nodes, the number of write operation nodes, and the concurrency corresponding to each endpoint address are written into the task definition file executable by the benchmark tool of the distributed storage system.

[0092] In one possible implementation, the test report generation module 44 includes: A performance metric extraction unit is used to extract key performance metrics from the raw result data; wherein, the key performance metrics include at least object size and concurrency. The classification result generation unit is used to classify the original result data in multiple dimensions according to the object size, concurrency, and operation type to obtain the classification result after aggregation analysis.

[0093] In one possible implementation, the classification result generation unit is specifically used for: The objects are classified based on their size units to obtain a first classification result; The first classification result is further classified based on the numerical value of the object size to obtain a second classification result; The second classification result is classified three times based on the value of the concurrency number to obtain the classification result after aggregation analysis. Calculate the comprehensive performance indexes for each read operation type and write operation type, and classify them according to read operation type and write operation type.

[0094] In one possible implementation, the test report generation module 44 is specifically used for: Based on the classification results of the original result data, a first report containing data details and a formatted summary report showing the performance index comparison under different combinations of object size and concurrency are generated.

[0095] like Figure 5 As shown in the figure, this application provides an electronic device, including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. Memory 113 is used to store computer programs; In one embodiment of this application, when the processor 111 executes a program stored in the memory 113, it implements the performance testing method provided in any of the foregoing method embodiments, including: In response to a user's call to a pre-defined test configuration interface, obtain the set of test tasks input by the user based on the test configuration interface; The test task set is parsed to obtain multiple test parameter information, and multiple task definition files that can be executed by the benchmark testing tool of the distributed storage system are generated based on the test parameter information. The multiple task definition files are submitted to the benchmark testing tool, and multiple task identifiers returned by the benchmark testing tool are received. The execution status of the corresponding test tasks is monitored based on the multiple task identifiers. The test tasks are the test tasks defined in the task definition files. After all test tasks have been completed, the raw result data corresponding to each task identifier is obtained, the raw result data is aggregated and analyzed, and a structured test report is generated.

[0096] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the performance testing method provided in any of the foregoing method embodiments.

[0097] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0099] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also mean including the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0100] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An automated performance testing method for a distributed storage system, characterized in that, The method includes: In response to a user's call to a pre-defined test configuration interface, obtain the set of test tasks input by the user based on the test configuration interface; The test task set is parsed to obtain multiple test parameter information, and multiple task definition files that can be executed by the benchmark testing tool of the distributed storage system are generated based on the test parameter information. The multiple task definition files are submitted to the benchmark testing tool, and multiple task identifiers returned by the benchmark testing tool are received. The execution status of the corresponding test tasks is monitored based on the multiple task identifiers. The test tasks are the test tasks defined in the task definition files. After all test tasks have been completed, the raw result data corresponding to each task identifier is obtained, the raw result data is aggregated and analyzed, and a structured test report is generated.

2. The method according to claim 1, characterized in that, The test task set is in configuration file format or command line parameter format; The test task set includes one or more test tasks, wherein each test task includes at least: operation type, object size, number of objects, concurrency, and runtime parameters.

3. The method according to claim 1, characterized in that, The step of generating multiple task definition files executable by the benchmark testing tool for the distributed storage system based on the test parameter information includes: For each test parameter, perform the following processing to generate multiple task definition files that can be executed by the benchmark tool for the distributed storage system: The test parameter information is parsed to obtain the number of endpoint addresses, total concurrency, and operation type; Based on the number of endpoint addresses and the total concurrency, determine the concurrency corresponding to each endpoint address; Based on the operation type and the concurrency corresponding to each endpoint address, generate the corresponding executable task definition file for the benchmark tool of the distributed storage system.

4. The method according to claim 3, characterized in that, The step of generating an executable task definition file for the benchmark testing tool of the distributed storage system based on the operation type and the concurrency corresponding to each endpoint address includes: When the operation type includes a read operation, determine the data preparation stage information of the current test task, and write the data preparation stage information and the concurrency number corresponding to each endpoint address into the task definition file executable by the benchmark testing tool of the distributed storage system. When the operation type is a mixed read-write operation, the number of read operation nodes and write operation nodes are determined according to a pre-set ratio rule, and the number of read operation nodes, the number of write operation nodes, and the concurrency corresponding to each endpoint address are written into the task definition file executable by the benchmark tool of the distributed storage system.

5. The method according to claim 1, characterized in that, The aggregation analysis of the original result data includes: Extract key performance indicators from the raw result data; wherein, the key performance indicators include at least object size and concurrency. The original result data is classified in multiple dimensions according to the object size, concurrency, and operation type to obtain the classification results after aggregation analysis.

6. The method according to claim 5, characterized in that, The process of classifying the original result data in multiple dimensions according to the object size, concurrency, and operation type to obtain the classification results after aggregation analysis includes: The objects are classified based on their size units to obtain a first classification result; The first classification result is further classified based on the numerical value of the object size to obtain a second classification result; The second classification result is classified three times based on the value of the concurrency number to obtain the classification result after aggregation analysis. Calculate the comprehensive performance indexes for each read operation type and write operation type, and classify them according to read operation type and write operation type.

7. The method according to claim 1, characterized in that, The generation of the structured test report includes: Based on the classification results of the original result data, a first report containing data details and a formatted summary report showing the performance index comparison under different combinations of object size and concurrency are generated.

8. An automated performance testing device for a distributed storage system, characterized in that, The device includes: The test task set acquisition module is used to acquire the test task set input by the user based on the test configuration interface in response to the user calling the pre-set test configuration interface; The task definition file generation module is used to parse the test task set, obtain multiple test parameter information, and generate multiple task definition files that can be executed by the benchmark testing tool of the distributed storage system based on the test parameter information. The task submission module is used to submit the multiple task definition files to the benchmark testing tool, receive multiple task identifiers returned by the benchmark testing tool, and monitor the execution status of the corresponding test tasks based on the multiple task identifiers; wherein, the test task is the test task defined in the task definition file; The test report generation module is used to obtain the raw result data corresponding to each task identifier after all test tasks have been executed, perform aggregation analysis on the raw result data, and generate a structured test report.

9. An electronic device, characterized in that, include: A processor and a memory, the processor being configured to execute a performance testing program stored in the memory to implement a method for automated performance testing of a distributed storage system according to any one of claims 1-7.

10. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the distributed storage system performance automation testing method according to any one of claims 1-7.

Images