Performance test method, apparatus, device, and medium

Abstract

This disclosure provides a performance testing method, apparatus, device, and medium, relating to the field of cluster testing. The method includes: in response to acquired cluster test parameters, generating a test script based on the cluster test parameters, the test script including resource allocation instructions; allocating test resources for executing the test script on at least one computing node based on the resource allocation instructions, so that the at least one computing node executes the test script in parallel based on the test resources; and acquiring performance test data obtained after the at least one computing node executes the test script. This not only achieves automatic script generation, automatic resource allocation, and automatic result collection for multi-node performance testing in a cluster environment, solving the problems of low efficiency in manual testing and susceptibility to interference from other running processes on the same node, but also effectively improves the accuracy and efficiency of testing.

Description

Technical Field

[0001] This disclosure relates to the field of testing technology, and in particular to a performance testing method, apparatus, equipment and medium. Background Technology

[0002] Performance evaluation of high-performance computing (HPC) clusters is a crucial step in cluster operation and optimization. Accurate performance evaluation provides important data for resource scheduling, system upgrades, and fault diagnosis. Typically, evaluating HPC cluster performance involves multiple metrics, such as computing power, input / output (I / O) throughput, and memory bandwidth. However, current HPC cluster performance testing technologies suffer from low accuracy and efficiency. Summary of the Invention

[0003] This disclosure provides a performance testing method, apparatus, device, and medium to at least solve the above-mentioned technical problems existing in the prior art.

[0004] In a first aspect, embodiments of this disclosure provide a performance testing method, the method comprising: In response to the acquired cluster test parameters, a test script is generated based on the cluster test parameters, and the test script includes resource allocation instructions; Based on the resource allocation instruction, test resources for executing the test script are allocated on at least one computing node, so that the at least one computing node executes the test script in parallel based on the test resources; Obtain the performance test data obtained after the at least one computing node executes the test script.

[0005] In one embodiment, the resource allocation instruction includes a resource permission allocation instruction and / or a required resource allocation instruction; the step of allocating test resources for executing the test script on at least one computing node based on the resource allocation instruction includes: For each computing node, based on the resource permission allocation instruction, the resource usage permissions of the test task corresponding to the test script and other tasks other than the test task on the computing node are determined; And / or, For each computing node, based on the required resource allocation instructions, a computing resource unit for executing the test script is determined in the computing node.

[0006] In one embodiment, the resource permission allocation instruction includes a first parameter, which is used to instruct the test script to exclusively use the computing node; The step of determining the resource usage permissions of the computing node for the test task corresponding to the test script and other tasks besides the test task based on the resource permission allocation instruction includes: Based on the first parameter, the computing node is locked to prevent tasks other than the test script from sharing the computing resources of the computing node.

[0007] In one embodiment, the required resource allocation instruction further includes a second parameter, which indicates the number of CPU cores required for the test script to run; The step of determining the computing resource unit in the computing node for executing the test script based on the required resource allocation instruction includes: Based on the second parameter, target CPU cores matching the number of CPU cores are selected from the available CPU cores in the computing node; The test script is bound to the target CPU core, so that the test script runs on the target CPU core.

[0008] In one embodiment, the cluster test parameters include test environment parameters; generating a test script based on the cluster test parameters includes: During the process of generating test scripts based on the cluster test parameters, the runtime environment of the test scripts is configured based on the test environment parameters. The test environment parameters include at least one of the following: compiler type, compiler parameters, and array size.

[0009] In one embodiment, after obtaining the performance test data obtained by the at least one computing node executing the test script, the method further includes: Obtain the node information for each computing node; The system associates and stores the node information of each computing node with the performance test data of that computing node.

[0010] In one embodiment, the method further includes: The performance test data of different computing nodes are displayed in a comparative manner according to the first display format. And / or, According to the second display format, the performance data of at least two test items in the performance test data of the same computing node are compared and displayed.

[0011] In one embodiment, both the first display format and the second display format may include at least one of the following: bar chart, column chart, and line chart.

[0012] In one embodiment, the method further includes: For each computing node, determine the comparison result between the performance test data of the computing node and the benchmark result; Based on the comparison results, abnormal nodes in at least one computing node are identified.

[0013] Secondly, embodiments of this disclosure provide a performance testing method apparatus, the apparatus comprising: A generation module is used to generate a test script based on the acquired cluster test parameters in response to the cluster test parameters. The test script includes resource allocation instructions. An allocation module is configured to allocate test resources for executing the test script on at least one computing node based on the resource allocation instruction, so that the at least one computing node executes the test script in parallel based on the test resources; The acquisition module is used to acquire performance test data obtained by the at least one computing node after executing the test script.

[0014] In one embodiment, the resource allocation instruction includes a resource permission allocation instruction and / or a required resource allocation instruction; the apparatus includes: The determination module is used to determine, for each computing node, the resource usage permissions of the test task corresponding to the test script and other tasks other than the test task on the computing node, based on the resource permission allocation instruction; And / or, The determining module is further configured to, for each computing node, determine a computing resource unit in the computing node for executing the test script based on the required resource allocation instruction.

[0015] In one embodiment, the resource permission allocation instruction includes a first parameter, the first parameter being used to instruct the test script to exclusively use the computing node; the apparatus includes: A locking module is used to lock the computing node based on the first parameter to prevent tasks other than the test script from sharing the computing resources of the computing node.

[0016] In one embodiment, the required resource allocation instruction further includes a second parameter, the second parameter indicating the number of CPU cores required for the test script to run; the apparatus includes: The filtering module is used to filter target CPU cores that match the number of CPU cores from the available CPU cores in the computing node based on the second parameter. A binding module is used to bind the test script to the target CPU core, so that the test script runs on the target CPU core.

[0017] In one embodiment, the cluster test parameters include test environment parameters; the apparatus includes: The configuration module is used to configure the runtime environment of the test script based on the test environment parameters during the process of generating the test script based on the cluster test parameters. The test environment parameters include at least one of the following: compiler type, compiler parameters, and array size.

[0018] In one embodiment, the apparatus includes: The acquisition module is also used to acquire node information for each computing node; An associated storage module is used to associate and store the node information of each computing node and the performance test data of the computing node.

[0019] In one embodiment, the apparatus further includes: The display module is used to compare and display the performance test data of different computing nodes in a first display format. And / or, The display module is also used to compare and display the performance data of at least two test items in the performance test data of the same computing node in a second display format.

[0020] In one embodiment, both the first display format and the second display format may include at least one of the following: bar chart, column chart, and line chart.

[0021] In one embodiment, the apparatus includes: The determination module is used to determine, for each computing node, the comparison result between the performance test data of the computing node and the benchmark result; An identification module is used to identify abnormal nodes among the at least one computing node based on the comparison results.

[0022] Thirdly, embodiments of this disclosure provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the performance testing method of the first aspect.

[0023] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform a performance testing method according to the first aspect.

[0024] The beneficial technical effects of the performance testing method provided in this disclosure are at least as follows: In response to acquired cluster test parameters, a test script is generated based on these parameters. Based on resource allocation instructions within the test script, test resources are allocated on at least one computing node to execute the test script. This allows at least one computing node to execute the test script in parallel based on these test resources, thereby enabling the acquisition of performance test data obtained after each computing node executes the test script. This not only achieves automatic script generation, automatic resource allocation, and automatic result collection for multi-node performance testing in a cluster environment, but also solves the problems of low efficiency in manual testing and susceptibility to interference from other running processes on the same node, effectively improving the accuracy and efficiency of the test.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0026] Figure 1 This is a schematic flowchart of a performance testing method provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the structure of a performance testing device provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0027] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0028] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0029] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0031] As described in the background section, HPC cluster performance evaluation is crucial, and evaluating HPC cluster performance involves multiple dimensions of metrics, such as computing power, I / O throughput, and memory bandwidth.

[0032] However, in traditional operation and maintenance models, cluster performance testing typically relies on manual execution by operations and maintenance personnel. Taking the testing of metrics such as memory bandwidth as an example, operations and maintenance personnel need to log in to each compute node in the cluster sequentially, manually compile the test code on the node, and execute the test script via command line. When the cluster is large, this manual operation method is cumbersome and prone to configuration errors or node omissions due to human error, resulting in low testing efficiency and difficulty in guaranteeing accuracy. In addition, performance test data is stored in a scattered manner across various nodes, requiring manual collection, organization, and analysis, making it difficult to quickly generate a comprehensive cluster performance report and to intuitively identify nodes with abnormal performance.

[0033] Furthermore, it's difficult to ensure complete uniformity in key parameters such as compiler version, compiler optimization parameters, and array size when executing tests across different compute nodes. These inconsistencies directly impact performance test data, leading to lower test accuracy and a lack of cross-sectional comparability between data from different nodes, rendering the data meaningless.

[0034] Meanwhile, traditional performance testing methods cannot precisely control the binding relationship between the test program and the physical core of the Central Processing Unit (CPU) or the Non-Uniform Memory Access (NUMA) node, and cannot ensure that the test process exclusively uses the specified computing resources or the entire NUMA node. In this case, the testing process is easily interfered with by other running processes on the same node, affecting the accuracy of the test.

[0035] Therefore, how to automate HPC cluster performance testing while improving its accuracy and efficiency is a pressing technical problem that needs to be solved.

[0036] Based on this, the present disclosure provides a performance testing method, apparatus, device, and medium to at least solve the technical problem of low accuracy and efficiency in cluster performance testing in the prior art. The performance testing method provided by the present disclosure will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] Figure 1 This is a flowchart illustrating a performance testing method provided in an embodiment of this disclosure.

[0038] like Figure 1 As shown, the performance testing method provided in this disclosure embodiment may include the following steps: S110, in response to the acquired cluster test parameters, generates a test script based on the cluster test parameters.

[0039] The cluster test parameters can be the relevant test information configured by the user when initiating the test script. The cluster test parameters include at least one of the following: a list of target nodes for specifying the nodes or partitions to be tested; test environment parameters for configuring the compiler, compilation optimization options, and array size; and resource request parameters for defining the required number of computing cores and binding strategies. No specific limitations are made here.

[0040] In this way, users can define cluster test parameters through a configuration interface (e.g., a web frontend or command-line tool) or a predefined configuration file (such as YAML or JavaScript Object Notation (JSON) format). In response to the obtained cluster test parameters, a Slurm job scheduling script (e.g., a script file with a .sh extension) can be automatically generated. This test script must at least include resource allocation instructions (such as the #SBATCH instruction), which is used to request the cluster scheduler to allocate computing resources to the test script corresponding to the test script; specific limitations are not specified here.

[0041] In addition, the test script may also include declaration statements for interpreting script execution (such as #! / bin / bash), compilation instructions for automatically embedding user-defined compiler and optimization options for compiling the STREAM source code, and execution instructions for running the compiled STREAM binary file, without specific limitations here.

[0042] S120, based on resource allocation instructions, allocates test resources for executing test scripts on at least one computing node, so that at least one computing node executes test scripts in parallel based on the test resources.

[0043] S130, Obtain performance test data obtained after at least one computing node executes the test script.

[0044] The performance test data may include at least one performance parameter among memory bandwidth, computing power, and input / output (I / O) throughput, without specific limitations here.

[0045] Specifically, in response to the acquired cluster test parameters, a test script is generated based on the cluster test parameters, and test resources for executing the test script are allocated on at least one computing node based on the resource allocation instructions in the test script, so that at least one computing node executes the test script in parallel based on the test resources, thereby obtaining performance test data obtained by the at least one computing node after executing the test script respectively.

[0046] In one example, the generated test script can be submitted via Slurm's command-line interface (such as `sbatch stream_job.sh`). The Slurm scheduler, acting as the central controller of the cluster, then parses the resource allocation instructions in the test script and allocates test resources for executing the script across one or more compute nodes based on these instructions. Afterward, each allocated compute node can independently and in parallel execute the same test process.

[0047] Based on the performance testing method provided in this disclosure, in response to acquired cluster test parameters, a test script is automatically generated based on the cluster test parameters. Then, based on the resource allocation parameters in the test script, test resources are allocated on at least one computing node to execute the test script. This allows at least one computing node to execute the test script in parallel based on the test resources, thereby obtaining performance test data obtained after each of the at least one computing node executes the test script. This not only achieves automatic script generation, automatic resource allocation, and automatic result collection for multi-node performance testing in a cluster environment, but also solves the problems of low efficiency in manual testing and susceptibility to interference from other running processes on the same node, effectively improving the accuracy and efficiency of testing.

[0048] In order to accurately describe the performance testing method provided in the embodiments of this disclosure, in one embodiment, the above-mentioned resource allocation instruction may include a resource permission allocation instruction and / or a required resource allocation instruction.

[0049] The resource permission allocation instruction can be used to define the usage permissions of the test task corresponding to the test script on the computing node resources, such as whether it is allowed to share resources with other tasks. The aforementioned required resource allocation instruction can be used to request the specific amount of resources required for the test task corresponding to the test script to run, such as the required number of CPU cores, memory size, etc., which are not specifically limited here.

[0050] Based on this, the above steps of allocating test resources for executing test scripts on at least one computing node based on resource allocation instructions may specifically include the following steps: For each compute node, based on resource permission allocation instructions, determine the resource usage permissions of the test task corresponding to the test script and other tasks on the compute node. And / or, For each compute node, based on the required resource allocation instructions, the compute resource units used to execute the test script are determined within the compute node.

[0051] The computing resource unit may include CPU cores, memory blocks or other types of hardware computing units, without specific limitations here.

[0052] Specifically, since the resource allocation instruction may include resource permission allocation instructions and / or required resource allocation instructions, if the resource allocation instruction includes resource permission allocation instructions, then for each computing node, based on the resource permission allocation instructions, the resource usage permissions of the test task corresponding to the test script and other tasks other than the test task for that computing node can be determined.

[0053] If the resource allocation instruction includes a required resource allocation instruction, for each computing node, based on the required resource allocation instruction, the computing resource unit used to execute the test script is determined in the computing node.

[0054] In this embodiment, the access control of computing resources on computing nodes can be achieved based on the resource permission allocation instructions within the resource allocation commands. This effectively avoids interference from other running processes on the same node during the testing process, thus ensuring the accuracy of the test results. Simultaneously, the accurate allocation of computing resource units within the computing node can be achieved based on the required resource allocation instructions within the resource allocation commands. This avoids performance fluctuations caused by improper resource allocation (such as insufficient CPU cores or core drift), thereby effectively improving the accuracy of the test.

[0055] Based on this, in order to accurately describe the performance testing method provided by the embodiments of this disclosure, in one embodiment, the above-mentioned resource permission allocation instruction may include a first parameter, which may be used to indicate that the test script has exclusive access to the computing node, without being specifically limited here.

[0056] Based on this, the steps described above for determining the resource usage permissions of the test task corresponding to the test script and other tasks on the computing node based on resource permission allocation instructions can specifically include: Based on the first parameter, the compute node is locked to prevent tasks other than the test script from sharing the compute node's computing resources.

[0057] Specifically, when the resource permission allocation instruction includes a first parameter indicating that the test script has exclusive access to the computing node, the entire computing node allocated to the test script can be locked according to the first parameter in the resource permission allocation instruction to prevent any other tasks (such as background processes from the same user or other users, scheduled tasks or other jobs) from running on that node simultaneously.

[0058] In one example, the resource permission allocation directive could be the #SBATCH --exclusive parameter, which instructs the test script to exclusively use the entire compute node. This allows the resource permission allocation directive to be parsed, ensuring that the test script exclusively uses all the compute resources of the compute node (including CPU cores, memory bandwidth, cache, etc.) and prohibiting other tasks from sharing any compute resources of the node.

[0059] In this embodiment, the first parameter in the resource permission allocation instruction can lock the entire allocated computing node, preventing any other tasks from running simultaneously on that node. This allows the test script to exclusively utilize all the computing resources of the node (including CPU cores, memory bandwidth, cache, etc.) during execution, thus preventing interference from other tasks and effectively improving test accuracy.

[0060] To accurately describe the performance testing method provided in this disclosure, in another embodiment, the required resource allocation instruction may further include a second parameter, which indicates the number of CPU cores required for the test script to run; this is not specifically limited here. Therefore, the step of determining the computing resource unit in the computing node for executing the test script based on the required resource allocation instruction may specifically include: Based on the second parameter, target CPU cores matching the number of CPU cores are selected from the available CPU cores in the computing node. Bind the test script to the target CPU core so that the test script runs on the target CPU core.

[0061] Specifically, when the required resource allocation instruction includes a second parameter indicating the number of CPU cores required for the test script to run, a target CPU core matching the number of CPU cores can be selected from the available CPU cores of the computing node based on the second parameter in the required resource allocation instruction, and the test script can be bound to the target CPU core, so that the test script can be run on the bound target CPU core without core drift.

[0062] In one example, the second parameter could be the `#SBATCH --cpus-per-task=128` directive in the Slurm script, used in conjunction with the `--cpu-bind=cores` binding policy. This allows for the selection of 128 target CPU cores from the available CPU cores on the compute node, and then the binding policy can be used to lock the test process onto these cores.

[0063] In this embodiment, when the second parameter in the required resource allocation instruction is used to indicate the number of CPU cores required for the test script to run, a specified number of target CPU cores are selected from the available CPU cores of the computing node based on the second parameter, and the test script is bound to the target CPU core, so that the test script can be run on the target CPU core. This effectively avoids the migration of the test script process between cores, thereby effectively improving the accuracy of the test and ensuring the repeatability of the test.

[0064] Furthermore, in order to accurately describe the performance testing method provided in the embodiments of this disclosure, in one embodiment, the cluster test parameters may include test environment parameters. Thus, the step of generating a test script based on the cluster test parameters may include: During the process of generating test scripts based on cluster test parameters, the runtime environment of the test scripts is configured based on test environment parameters.

[0065] In some embodiments, the above test environment parameters may include at least one of the following: compiler type (GNU CompilerCollection), Intel C / C++ Compiler (ICC), compiler parameters, and array size, without specific limitations here. The compiler type refers to the type of compiler used to compile the test program. Compiler parameters refer to additional optimization or configuration options added during the compilation process; the array size refers to the dimension of the core data structure used in the test script.

[0066] Specifically, since the acquired cluster test parameters can include test environment parameters, and these test environment parameters can guarantee at least one of the following: compiler type, compiler parameters, and array size, the runtime environment of the test script can be configured based on these test environment parameters, thereby ensuring that the runtime environment of each computing node is consistent.

[0067] In one example, the user specifies the use of the GNU Compiler Collection (GCC) 9.3.0 compiler with specific optimization parameters (such as -O3 -march=native) and an array size of 200,000,000. Based on this, the system can automatically configure the corresponding compiler loading instructions, array size definition, and compilation instructions in the generated test script in response to these test environment parameters. This ensures that when the test script is distributed to various compute nodes for execution, it is compiled and run using the same compiler version, the same compiler parameters, and the same array size.

[0068] In another example, users can also choose the ICC compiler and specify its specific compilation parameters (such as -xHost -qopenmp). The compilation directives can be replaced accordingly. This allows for flexible support of performance testing under different compilers and compilation options by changing the test environment parameters.

[0069] In this embodiment, the runtime environment can be configured during the test script generation stage using the test environment parameters in the cluster test parameters. This ensures that key parameters such as the compiler version, compiler optimization parameters, and array size used by each computing node are consistent, effectively avoiding interference with performance test data due to parameter inconsistencies, thereby improving the accuracy and reproducibility of the test.

[0070] In order to provide a comprehensive and detailed description of the performance testing method provided in the embodiments of this disclosure, after obtaining the performance test data obtained after at least one computing node executes the test script, the performance testing method provided in the embodiments of this disclosure may further include the following steps: Obtain node information for each computing node; The system associates node information and performance test data of each compute node with the data.

[0071] The node information can be related information used to identify each computing node, such as CPU model, memory frequency, node name, etc., without specific limitations.

[0072] Specifically, after each computing node completes the execution of the test script, the node information of that computing node is obtained and associated with the corresponding performance test data. In this way, each piece of performance test data can be traced back to its source node, facilitating subsequent comparative analysis of test results from different nodes.

[0073] In one example, after each compute node completes the test, its standard output is redirected by Slurm to an output file in a specified format (such as a file containing a job identifier (ID) and node name). The parser then retrieves the node information (such as the node name) corresponding to each test result (i.e., the performance test data mentioned above) by parsing the output filename or calling Slurm's application programming interface. Simultaneously, the parser extracts key performance data from each node's output file, including replication bandwidth, scaling bandwidth, cumulative bandwidth, and triplet bandwidth (units uniformly in MB / s). Subsequently, the extracted performance test data is associated with the obtained node information (such as node name, CPU model, memory frequency, etc.) and stored in a structured manner in a database or a comma-separated values ​​(CSV) file.

[0074] In this embodiment, after each computing node completes the execution of the test script, the node information of each computing node is obtained and stored in association with its performance test data. This achieves an accurate mapping between each computing node and its test results, facilitating subsequent accurate comparative analysis.

[0075] Furthermore, in order to comprehensively and thoroughly describe the performance testing method provided by the embodiments of this disclosure, in one embodiment, after obtaining the performance test data obtained after at least one computing node executes the test script, the performance testing method provided by the embodiments of this disclosure may further include the following steps: The performance test data of different computing nodes are displayed in a comparative manner according to the first display format. And / or, According to the second display format, the performance data of at least two test items in the performance test data of the same computing node are compared and displayed.

[0076] The first and second display formats can each include at least one of the following: bar chart, column chart, and line chart, without specific limitations. Taking memory bandwidth as a performance indicator as an example, at least two test items in the performance test data can include at least two of the following: copy bandwidth, scaling bandwidth, cumulative bandwidth, and triplet bandwidth, without specific limitations.

[0077] Specifically, after obtaining the performance test data, the performance test data of different computing nodes can be compared and displayed in a first display format, and / or, the performance data of at least two test items within the same computing node can be compared and displayed in a second display format.

[0078] In one example, the memory bandwidth performance of different compute nodes can be compared and displayed in a primary display format (such as a bar chart). For instance, the horizontal axis of the bar chart represents the node name (such as node01, node02, etc.), and the vertical axis represents the memory bandwidth value. The height of the bar corresponding to each node represents the triplet bandwidth performance value of that node, thus visually presenting the differences in memory performance among the nodes of the entire cluster. This primary display format can also be a bar chart or a line chart.

[0079] In another example, performance data for multiple test items within the same compute node can be compared and displayed in a second display format (such as a bar chart). For instance, generating a bar chart showing the performance of node 01 in four test results—replication bandwidth, scaling bandwidth, cumulative bandwidth, and triplet bandwidth—helps analyze the node's performance characteristics under different memory access modes. Similarly, the second display format could also be a bar chart or a line chart.

[0080] In this embodiment, by comparing and displaying the performance test data through a first display format and / or a second display format, the performance differences between different computing nodes and the performance differences of each test item within the same computing node can be presented intuitively, thereby improving the readability of the test results and the efficiency of analysis.

[0081] In order to provide a comprehensive and detailed description of the performance testing method provided by the embodiments of this disclosure, in one embodiment, after obtaining the performance test data obtained after at least one computing node executes the test script, the performance testing method provided by the embodiments of this disclosure may further include the following steps: For each compute node, determine the comparison results between the compute node's performance test data and the benchmark results; Identify at least one abnormal node in the computing nodes based on the comparison results.

[0082] The aforementioned benchmark results may be pre-set standard performance thresholds, test data from a reference node selected from at least one computing node, or normal performance baselines from historical test data; no specific limitations are imposed here.

[0083] Specifically, for each computing node, its performance test data is compared with the benchmark results, and quantitative indicators such as deviation rate or relative error are calculated to obtain the comparison results. Then, according to the preset anomaly judgment rules (such as a performance indicator deviation exceeding a set threshold), abnormal nodes whose performance deviates significantly from the benchmark results are identified from all computing nodes.

[0084] In one example, the triplet bandwidth value of each compute node is compared to a benchmark result. The benchmark result can be determined based on the average of test results for all tested nodes or the normal performance range of that compute node in historical test data. Then, abnormal nodes can be identified based on the comparison results. For example, a threshold can be set, and if a node's triplet bandwidth value is below this threshold, it is marked as a performance anomaly and reported.

[0085] In this embodiment, by comparing the performance test data of each computing node with the benchmark results and automatically identifying abnormal nodes based on the comparison results, automated evaluation and diagnosis of cluster performance testing can be achieved. This allows for the rapid and accurate identification of faulty computing nodes, preventing the accuracy of the overall test from being affected by the anomalies of individual nodes.

[0086] In order to accurately describe the performance testing method provided by the embodiments of this disclosure, it should be noted that the performance testing method provided by the embodiments of this disclosure has the following advantages compared with the manual testing method in the prior art: (1) Automation: This embodiment of the present disclosure can automatically generate job scripts conforming to the Slurm scheduling specification in response to user-configured cluster test parameters, and submit them with one click through the Slurm command-line interface. During the test, the system automatically completes the distribution of test scripts on each computing node, remote compilation of STREAM source code, distributed execution of test programs, and automatic collection and parsing of test results. The entire process requires no manual intervention, greatly saving the manpower and time costs of cluster operation and maintenance personnel.

[0087] (2) Standardization and Repeatability: This embodiment of the disclosure uses a unified test script to solidify the test environment and parameters, including compiler type (such as GCC or ICC), compiler optimization options, STREAM array size, and other key configurations. Since the same script template and parameter configuration are used for each test, the conditions for different batches of tests are completely consistent, thus ensuring high comparability and repeatability of the test results. This is of great significance for long-term tracking and regression testing of cluster performance.

[0088] (3) Precise Resource Control: This embodiment fully utilizes the resource management capabilities of the Slurm cluster management system. By using the first parameter in the resource allocation instruction (such as --exclusive), the test task achieves exclusive access to the computing node, effectively avoiding interference from other running processes on the same node on the memory bandwidth test results. At the same time, by using the second parameter (such as --cpus-per-task) and the task binding strategy (such as --cpu-bind=cores), the test process is precisely locked on the specified CPU core, preventing performance fluctuations caused by core drift, thereby obtaining the most accurate and stable memory bandwidth test data.

[0089] (4) Cluster-level perspective: This embodiment of the present disclosure can instantly complete the baseline test of memory bandwidth performance of all computing nodes (or nodes within a specified partition) in the entire cluster. By associating and storing the node information and performance test data of each computing node, the system can automatically generate a comparative analysis report of the memory bandwidth performance of each node in the cluster (such as bar charts, column charts, and other visualization forms), and supports performance comparison of different test items (Copy, Scale, Add, Triad) within the same node. On this basis, the system can also automatically identify nodes with abnormal performance (such as memory degradation nodes) based on preset benchmark results, helping operation and maintenance personnel to quickly locate performance bottlenecks or hardware failure nodes in the cluster.

[0090] (5) Flexibility and Scalability: Although the above embodiments use the STREAM memory bandwidth test as an example for illustration, those skilled in the art will understand that the performance testing methods and systems provided in this disclosure are not limited to this. Based on the same inventive concept, simply replacing the compilation objects and execution instructions in the test script with other benchmark testing programs (such as the HPL linear system software package, UnixBench general benchmark test suite, etc.) can easily extend to other performance evaluation scenarios such as computing power testing and I / O throughput testing, thereby forming a complete automated performance evaluation system for HPC clusters.

[0091] Furthermore, based on the same inventive concept, this disclosure provides a performance testing device, which can be specifically combined with the appendix. Figure 2 A performance testing apparatus provided in the embodiments of this disclosure will be described in detail.

[0092] Figure 2 This is a schematic diagram of the structure of a performance testing device provided in an embodiment of this disclosure.

[0093] like Figure 2 As shown, the performance testing device 200 may include: The generation module 210 is used to generate a test script based on the acquired cluster test parameters in response to the cluster test parameters, the test script including resource allocation instructions; The allocation module 220 is configured to allocate test resources for executing the test script on at least one computing node based on the resource allocation instruction, so that the at least one computing node executes the test script in parallel based on the test resources; The acquisition module 230 is used to acquire performance test data obtained by the at least one computing node after executing the test script.

[0094] In one embodiment, the resource allocation instruction includes a resource permission allocation instruction and / or a required resource allocation instruction; the performance testing apparatus provided in this disclosure embodiment may include: The determination module is used to determine, for each computing node, the resource usage permissions of the test task corresponding to the test script and other tasks other than the test task on the computing node, based on the resource permission allocation instruction; And / or, The determining module is further configured to, for each computing node, determine a computing resource unit in the computing node for executing the test script based on the required resource allocation instruction.

[0095] In one embodiment, the resource permission allocation instruction includes a first parameter, which is used to instruct the test script to exclusively use the computing node; the performance testing apparatus provided in this disclosure embodiment may include: A locking module is used to lock the computing node based on the first parameter to prevent tasks other than the test script from sharing the computing resources of the computing node.

[0096] In one embodiment, the required resource allocation instruction further includes a second parameter, the second parameter being used to indicate the number of CPU cores required for the test script to run; the performance testing apparatus provided in this disclosure embodiment may include: The filtering module is used to filter target CPU cores that match the number of CPU cores from the available CPU cores in the computing node based on the second parameter. The binding module is used to bind the test script to the target CPU core, so that the test script runs on the target CPU core.

[0097] In one embodiment, the cluster test parameters include test environment parameters; the performance testing apparatus provided in this disclosure embodiment may include: The configuration module is used to configure the runtime environment of the test script based on the test environment parameters during the process of generating the test script based on the cluster test parameters. The test environment parameters include at least one of the following: compiler type, compiler parameters, and array size.

[0098] In one embodiment, the performance testing apparatus provided in this disclosure may include: The acquisition module is also used to acquire node information for each computing node; An associated storage module is used to associate and store the node information of each computing node and the performance test data of the computing node.

[0099] In one embodiment, the performance testing apparatus provided in this disclosure may include: The display module is used to compare and display the performance test data of different computing nodes in a first display format. And / or, The display module is also used to compare and display the performance data of at least two test items in the performance test data of the same computing node in a second display format.

[0100] In one embodiment, both the first display format and the second display format may include at least one of the following: bar chart, column chart, and line chart.

[0101] In one embodiment, the performance testing apparatus provided in this disclosure may include: The determination module is used to determine, for each computing node, the comparison result between the performance test data of the computing node and the benchmark result; An identification module is used to identify abnormal nodes among the at least one computing node based on the comparison results.

[0102] Based on the performance testing method provided in this disclosure, in response to acquired cluster test parameters, a test script is automatically generated based on the cluster test parameters. Then, based on the resource allocation parameters in the test script, test resources are allocated on at least one computing node to execute the test script. This allows at least one computing node to execute the test script in parallel based on the test resources, thereby obtaining performance test data obtained after each of the at least one computing node executes the test script. This not only achieves automatic script generation, automatic resource allocation, and automatic result collection for multi-node performance testing in a cluster environment, but also solves the problems of low efficiency in manual testing and susceptibility to interference from other running processes on the same node, effectively improving the accuracy and efficiency of testing.

[0103] It is understood that, when implementing the corresponding performance testing method apparatus provided in the above embodiments, the above processing can be distributed to different program modules as needed to complete all or part of the processing described above. Furthermore, the apparatus and the corresponding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0104] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a performance testing method.

[0105] This application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored and when executed by a processor, the processor will execute the performance testing method provided in this application.

[0106] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0107] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0108] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0109] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0110] Figure 3This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure; as shown below. Figure 3 As shown, the electronic device 30 includes: a processor 301, and a memory 302 communicatively connected to the processor 301; the memory 302 stores instructions executable by the processor 301. The instructions are executed by the processor 301 to enable the processor 301 to perform the following: In response to the acquired cluster test parameters, a test script is generated based on the cluster test parameters, and the test script includes resource allocation instructions; Based on the resource allocation instruction, test resources for executing the test script are allocated on at least one computing node, so that the at least one computing node executes the test script in parallel based on the test resources; Obtain the performance test data obtained after the at least one computing node executes the test script.

[0111] The electronic devices and corresponding performance testing methods provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0112] In practical applications, the electronic device 30 may further include at least one network interface 303. The various components of the electronic device 30 are coupled together via a bus system 304. It is understood that the bus system 304 is used to implement communication between these components. In addition to a data bus, the bus system 304 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 3 All buses are labeled as bus system 304. The number of processors 301 and the number of memories 302 can be at least one. The network interface 303 is used for wired or wireless communication between the electronic device 30 and other devices.

[0113] The memory 302 in this embodiment is used to store various types of data to support the operation of the electronic device 30.

[0114] The methods disclosed in the above embodiments of this disclosure can be applied to processor 301, or implemented by processor 301. Processor 301 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 301 or by instructions in the form of software. The processor 301 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 301 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 302. Processor 301 reads the information in memory 302 and, in conjunction with its hardware, completes the steps of the aforementioned performance testing method.

[0115] In some embodiments, the electronic device 30 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.

[0116] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0117] In the above description, the term "some embodiments" refers to a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0118] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used in this disclosure is for the purpose of describing embodiments of this disclosure only and is not intended to be limiting of this disclosure.

[0119] It should be understood that in the various embodiments of this disclosure, the sequence number of each implementation process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0120] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0121] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A performance testing method, characterized in that, The method includes: In response to the acquired cluster test parameters, a test script is generated based on the cluster test parameters, and the test script includes resource allocation instructions; Based on the resource allocation instruction, test resources for executing the test script are allocated on at least one computing node, so that the at least one computing node executes the test script in parallel based on the test resources; Obtain the performance test data obtained after the at least one computing node executes the test script.

2. The method according to claim 1, characterized in that, The resource allocation instructions include resource permission allocation instructions and / or required resource allocation instructions; The step of allocating test resources for executing the test script on at least one computing node based on the resource allocation instruction includes: For each computing node, based on the resource permission allocation instruction, the resource usage permissions of the test task corresponding to the test script and other tasks other than the test task on the computing node are determined; And / or, For each computing node, based on the required resource allocation instructions, a computing resource unit for executing the test script is determined within the computing node.

3. The method according to claim 2, characterized in that, The resource permission allocation instruction includes a first parameter, which is used to instruct the test script to exclusively occupy the computing node; The step of determining the resource usage permissions of the computing node for the test task corresponding to the test script and other tasks besides the test task based on the resource permission allocation instruction includes: Based on the first parameter, the computing node is locked to prevent tasks other than the test script from sharing the computing resources of the computing node.

4. The method according to claim 2, characterized in that, The required resource allocation instruction also includes a second parameter, which indicates the number of CPU cores required for the test script to run. The step of determining the computing resource unit in the computing node for executing the test script based on the required resource allocation instruction includes: Based on the second parameter, target CPU cores matching the number of CPU cores are selected from the available CPU cores in the computing node; The test script is bound to the target CPU core, so that the test script runs on the target CPU core.

5. The method according to any one of claims 1 to 4, characterized in that, The cluster test parameters include test environment parameters; the generation of test scripts based on the cluster test parameters includes: During the process of generating test scripts based on the cluster test parameters, the runtime environment of the test scripts is configured based on the test environment parameters. The test environment parameters include at least one of the following: compiler type, compiler parameters, and array size.

6. The method according to any one of claims 1 to 4, characterized in that, After obtaining the performance test data obtained after the at least one computing node executes the test script, the method further includes: Obtain the node information for each computing node; The system associates and stores the node information of each computing node with the performance test data of that computing node.

7. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The performance test data of different computing nodes are displayed in a comparative manner according to the first display format. And / or, According to the second display format, the performance data of at least two test items in the performance test data of the same computing node are compared and displayed.

8. The method according to claim 7, characterized in that, Both the first display format and the second display format may include at least one of the following: bar chart, column chart, and line chart.

9. The method according to any one of claims 1 to 4, characterized in that, The method further includes: For each computing node, determine the comparison result between the performance test data of the computing node and the benchmark result; Based on the comparison results, abnormal nodes in the at least one computing node are identified.

10. A performance testing device, characterized in that, The device includes: A generation module is used to generate a test script based on the acquired cluster test parameters in response to the cluster test parameters. The test script includes resource allocation instructions. An allocation module is configured to allocate test resources for executing the test script on at least one computing node based on the resource allocation instruction, so that the at least one computing node executes the test script in parallel based on the test resources; The acquisition module is used to acquire performance test data obtained by the at least one computing node after executing the test script.

11. The apparatus according to claim 10, characterized in that, The resource allocation instruction includes a resource permission allocation instruction and / or a required resource allocation instruction; the device includes: The determination module is used to determine, for each computing node, the resource usage permissions of the test task corresponding to the test script and other tasks other than the test task on the computing node, based on the resource permission allocation instruction; And / or, The determining module is further configured to, for each computing node, determine a computing resource unit in the computing node for executing the test script based on the required resource allocation instruction.

12. The apparatus according to claim 11, characterized in that, The resource permission allocation instruction includes a first parameter, which instructs the test script to exclusively use the computing node; the device includes: A locking module is used to lock the computing node based on the first parameter to prevent tasks other than the test script from sharing the computing resources of the computing node.

13. The apparatus according to claim 11, characterized in that, The required resource allocation instruction further includes a second parameter, which indicates the number of CPU cores required for the test script to run; the device includes: The filtering module is used to filter target CPU cores that match the number of CPU cores from the available CPU cores in the computing node based on the second parameter. The binding module is used to bind the test script to the target CPU core, so that the test script runs on the target CPU core.

14. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the performance testing method according to any one of claims 1 to 9.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the performance testing method according to any one of claims 1 to 9.

Images