Performance test method and device and related equipment

Abstract

The embodiment of the invention provides a performance test method and device and related equipment, and the method comprises the steps: determining a to-be-tested server which comprises a plurality of load units, and each load unit is used for executing a performance test task; executing a performance test program on the to-be-tested server, wherein the performance test program comprises multiple rounds of load distribution processes; and in the process of executing the performance test program, based on a predefined load distribution sequence, distributing loads for each load unit of the to-be-tested server, the load distribution sequence being used for balancing load distribution results after the performance test program finishes multiple rounds of load distribution, and obtaining a test evaluation result of the to-be-tested server. According to the technical scheme provided by the embodiment of the invention, the efficiency and the performance of the performance test program can be improved, and the reliability of a test evaluation result is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and specifically to a performance testing method, apparatus, and related equipment. Background Art

[0002] High-Performance Linpack Benchmark (HPL) is a performance test program used to evaluate the floating-point computing capabilities of high-performance computing clusters. The floating-point computing capabilities of a server (such as a high-performance computing cluster) affect its efficiency and speed in processing complex computing tasks. Therefore, the efficiency and performance of the HPL test are very important for reliably evaluating the performance of the server. In this context, how to provide a technical solution to improve the test efficiency and performance of HPL and improve the reliability of the test evaluation results of the server has become a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0003] In view of this, the embodiments of the present invention provide a performance testing method, apparatus and related equipment to improve the testing efficiency and performance of HPL and enhance the reliability of the test evaluation results of the server.

[0004] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions.

[0005] In a first aspect, an embodiment of the present invention provides a performance testing method, including:

[0006] Determine a server to be tested, wherein the server to be tested includes a plurality of load units, each load unit being used to perform a performance test task;

[0007] Execute a performance test program on the server to be tested, wherein the performance test program includes multiple rounds of load distribution process;

[0008] And, during the execution of the performance test program, loads are distributed to the load units of the server to be tested based on a predefined load distribution sequence, wherein the load distribution sequence is used to balance the load distribution results after the performance test program completes multiple rounds of load distribution;

[0009] A test evaluation result of the server to be tested is obtained.

[0010] In a second aspect, an embodiment of the present invention provides a performance testing device, including:

[0011] A server to be tested determining module, used to determine a server to be tested, wherein the server to be tested includes a plurality of load units, each of which is used to perform a performance test task;

[0012] A program execution module, configured to execute a performance test program on the server to be tested, wherein the performance test program includes a multi-round load distribution process; and, in the process of executing the performance test program, to distribute loads to various load units of the server to be tested based on a predefined load distribution sequence, wherein the load distribution sequence is used to balance the load distribution results after the performance test program completes the multi-round load distribution;

[0013] The execution result obtaining module is used to obtain the test evaluation result of the server to be tested.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a program, and the processor calls the program stored in the memory to execute the performance testing method as described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present invention provides a storage medium, wherein the storage medium stores a program, and when the program is executed, the performance testing method as described in the first aspect is implemented.

[0016] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program, which implements the performance testing method described in the first aspect when executed by a processor.

[0017] A performance testing method provided by an embodiment of the present invention determines a server to be tested, wherein the server to be tested includes multiple load units, each load unit is used to perform a performance testing task; and executes a performance testing program on the server to be tested, wherein the performance testing program includes a multi-round load distribution process; and in the process of executing the performance testing program, based on a predefined load distribution order, loads are distributed to each load unit of the server to be tested, and the load distribution order is used to balance the load distribution results after the performance testing program completes multiple rounds of load distribution; and a test evaluation result of the server to be tested is obtained.

[0018] It can be seen that the technical solution provided by the embodiment of the present invention, when executing the performance test program to perform performance testing on the server to be tested, by changing the load distribution order used by the performance test program, the performance test program can distribute the load for each load unit of the server to be tested based on the predefined load distribution order when distributing the load; because the load distribution order can ensure that the load distribution result of the performance test program is balanced after the multiple rounds of load distribution process are completed; avoid the situation that the load distribution result cannot be balanced due to the characteristics of the performance test program after the performance test program performs multiple rounds of load distribution for each load unit based on the original load distribution order. After the performance test program ends multiple rounds of load distribution, whether the load distribution result can be balanced affects the efficiency and performance of the subsequent performance test program to test the server to be tested, and then affects the reliability of the test evaluation results. Therefore, in the embodiment of the present invention, by designing a predefined load distribution order, it is ensured that the load distribution result can be balanced after the multiple rounds of load distribution are completed, thereby improving the efficiency and performance of the performance test program and improving the reliability of the test evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0020] Figure 1 is a flow chart of a performance testing method provided by an embodiment of the present invention;

[0021] Figure 2 is a schematic diagram of calculated load distribution results of a first load unit group;

[0022] Figure 3 is a schematic diagram of a distribution process of a fixed computing load distribution order provided by an embodiment of the present invention;

[0023] Figure 4 It is a schematic diagram of the distribution process of calculating the load distribution order in the performance testing method provided in the embodiment of the present invention;

[0024] Figure 5 It is a schematic diagram of the distribution result of the reverse distribution process in the bidirectional computing load distribution sequence provided by an embodiment of the present invention;

[0025] Figure 6 It is a schematic diagram of load distribution results when calculating the load distribution sequence in the performance testing method provided in an embodiment of the present invention;

[0026] Figure 7 It is a schematic diagram of a communication process under fixed-order communication load distribution provided by an embodiment of the present invention;

[0027] Figure 8 It is a schematic diagram of a process of implementing communication load distribution based on the communication load distribution strategy provided by an embodiment of the present invention;

[0028] Fig. 9 is another flow chart of the performance testing method provided by an embodiment of the present invention;

[0029] Fig.10 is a result schematic diagram of the optimal calculation load distribution result provided by an embodiment of the present invention;

[0030] Fig.11 It is a structural schematic diagram of a performance testing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] As server computing power increases, it is necessary to evaluate server performance and design and apply the corresponding servers. HPL (High-Performance Linpack Benchmark) is usually used to evaluate the performance of a server. For example, HPL is used to evaluate the performance of a server based on a NUMA architecture consisting of several NUMA (Non-Uniform Memory Access) units interconnected by sockets (connecting nodes).

[0033] HPL is a benchmark program used to test the floating-point performance of high-performance computing cluster systems. It evaluates the floating-point computing capabilities of servers by using Gaussian elimination to solve a set of univariate N-order dense linear algebraic equations. In the High Performance Computing (HPC) scenario, whether the load distribution results finally distributed by HPL to each load unit in the server (such as the processor that performs performance test tasks on the server) are balanced affects the test performance and efficiency of HPL, and further affects the reliability of the test evaluation results of the server.

[0034] Therefore, when testing the performance of a server (such as a server with a NUMA architecture), it is very necessary to optimize and adjust the HPL.

[0035] Based on this, the embodiment of the present invention provides a performance testing method to improve the testing efficiency and performance of HPL and improve the reliability of the test evaluation results of the server.

[0036] Please refer to Figure 1 , Figure 1 It is a flow chart of a performance testing method provided by an embodiment of the present invention.

[0037] like Figure 1 As shown, the method comprises the following steps:

[0038] Step S100: determining a server to be tested, wherein the server to be tested includes a plurality of load units, each of which is used to execute a performance test task.

[0039] The server to be tested may be a server for high performance computing, such as the server with the NUMA architecture described above.

[0040] There are multiple processors in a NUMA architecture server. Each processor has its own independent local memory and supports fast access to its own local memory. The processors are connected through a "bus". The processor can access the "memory" of other processors through the "bus", but it is slower than the processor accessing its own memory.

[0041] Since NUMA architecture servers take into account both scalability and data sharing convenience, ensuring their adaptability in handling "complex transactions" issues, in the multi-core era, using NUMA architecture servers as basic units to build computing systems has become a mainstream choice that takes into account both performance and benefits.

[0042] Each load unit of the server to be tested described above may be each processor or processing core on the server to be tested, which is used to execute a performance test task, so as to judge the performance of the load unit according to the execution result, and further obtain the performance of the server to be tested.

[0043] Step S101, executing a performance testing program on the server to be tested, wherein the performance testing program includes a multi-round load distribution process; and in the process of executing the performance testing program, distributing loads to various load units of the server to be tested based on a predefined load distribution order.

[0044] The load distribution sequence is used to balance the load distribution results after the performance testing program completes multiple rounds of load distribution.

[0045] The performance test program may be the HPL described in the above embodiment. The performance test program is executed on the server to be tested, that is, solving a univariate N-th order dense linear algebraic equation system, and the performance test task is a task corresponding to each step required to solve the N-th order dense linear algebraic equation system.

[0046] During the execution of the performance test program, the performance test program will distribute loads to each load unit of the server to be tested. Since the order of load distribution in the performance test program is fixed, after multiple rounds of load distribution, the load distribution results of each load unit cannot be kept balanced as expected by the performance test program, thus affecting the efficiency and performance of the performance test program.

[0047] In an embodiment of the present invention, the load distribution sequence of the performance testing program is improved so that under the control of a predefined load distribution sequence, the load allocated to each load unit in multiple rounds of load distribution of the performance testing program can be dynamically changed so that the final load distribution result can be balanced.

[0048] Step S102: obtaining a test evaluation result of the server to be tested.

[0049] It can be seen that the technical solution provided by the embodiment of the present invention, when executing the performance test program to perform performance testing on the server to be tested, by changing the load distribution order used by the performance test program, the performance test program can distribute the load for each load unit of the server to be tested based on the predefined load distribution order when distributing the load; because the load distribution order can ensure that the load distribution result of the performance test program is balanced after the multiple rounds of load distribution process are completed; avoid the situation that the load distribution result cannot be balanced due to the characteristics of the performance test program after the performance test program performs multiple rounds of load distribution for each load unit based on the original load distribution order. After the performance test program ends multiple rounds of load distribution, whether the load distribution result can be balanced affects the efficiency and performance of the subsequent performance test program to test the server to be tested, and then affects the reliability of the test evaluation results. Therefore, in the embodiment of the present invention, by designing a predefined load distribution order, it is ensured that the load distribution result can be balanced after the multiple rounds of load distribution are completed, thereby improving the efficiency and performance of the performance test program and improving the reliability of the test evaluation results.

[0050] In order to obtain a load distribution sequence that can keep the load distribution result balanced, before executing the performance test program, the original load distribution sequence of the performance test program is improved, that is, a new load distribution sequence is predefined.

[0051] Please continue to refer to Figure 1 Before step S101, the method may further include:

[0052] Step S1001, defining a load distribution order based on the type of load.

[0053] In HPL, the loads allocated to each load unit of the server to be tested include multiple categories, and each category of load has a corresponding set of load distribution sequences. Therefore, in order to apply the basic usage process and mechanism of the program to be tested, when defining the load distribution sequence, targeted improvements are made to the load distribution sequence according to the type of load.

[0054] Step S1001 may include:

[0055] Based on the type of load, determine the corresponding load unit array, which is composed of various load units; define a load distribution strategy for distributing loads to various load units in the determined load unit array during each round of load distribution in the performance testing program, and obtain the load distribution order.

[0056] The number of the load units in the load unit array is the number of processors of the server to be tested.

[0057] Due to different load types, different array definitions are used when distributing loads, and the corresponding load distribution order is also different. Therefore, in the embodiment of the present invention, based on different load types, the load unit array is determined, and then the improved load distribution order is defined in a targeted manner, so that after the performance test program performs multiple rounds of load distribution, the load distribution result can be ensured to be more balanced, thereby improving the performance and efficiency of the performance test program.

[0058] In one embodiment, the load includes a computational load of a computational type. In the HPL, the computational load is used to enable the load unit to process the computational task, so as to execute the performance test task assigned to each load unit.

[0059] When the load is a calculation load of a calculation type, the load distribution order includes: a calculation load distribution order; and determining the corresponding load unit array based on the type of the load may include:

[0060] Based on the calculation type of the calculated load, determining that the corresponding load unit array is a first load unit array;

[0061] The load distribution strategy for distributing loads to each load unit of the determined load unit array in each round of load distribution process of the performance test program is defined to obtain the load distribution sequence, including:

[0062] The distribution strategy of the calculated load is defined as follows: according to the group order of each group of first load unit groups in the first load unit array, a distribution order of alternating forward distribution and reverse distribution is defined for the plurality of groups of first load unit groups, so that each load unit in adjacent groups of load unit groups performs forward distribution and reverse distribution in turn;

[0063] Obtaining the calculation load distribution order;

[0064] The forward distribution is defined as distribution according to the positive arrangement direction of each load unit in each unit group; the reverse distribution is defined as distribution according to the reverse arrangement direction of each load unit in each unit group.

[0065] When the load is a computational load, the load unit array is determined to be a first load unit array. The first load unit array is a two-dimensional process grid (process array) used in HPL, which is used to perform computations in parallel in a distributed memory system (eg, a server with a NUMA architecture).

[0066] The first load cell array is a two-dimensional array formed by P and Q. P represents the number of rows of the first load cell array, and Q represents the number of columns of the first load cell array. The product of P and Q, "P×Q", is equal to the total number of MPI (Message Passing Interface) processes used to run HPL, which is equal to the number of load cells (processors) of the server to be tested.

[0067] When HPL distributes computing load, it usually adopts static computing load balancing, that is, when the performance test program starts to be executed, the computing load allocated to each load unit is calculated.

[0068] Since the solution scale N in HPL is known, that is, the scale N of the univariate N-order dense linear algebraic equations to be solved has been set and can be determined, and the load data volume "P×Q" of the server to be tested is also known, it can be determined that the calculation load allocated to each load unit during the execution of the performance test program is:

[0069] In the P row direction, the calculated load of each load unit in each row is: N / P;

[0070] In the Q column direction, the calculated load of each load unit in each column is: N / Q.

[0071] The above static computing load balancing is performed in two directions because in HPL, when solving a univariate N-order dense linear algebraic equation system, Gaussian elimination method is used, which requires LU decomposition. Therefore, the computing load of each load unit in the row direction is balanced, and each load unit in the column direction is uniformly balanced.

[0072] Then, the calculated average values ​​N / P and N / Q of the calculated loads are allocated to the load cells of the first load cell array during the execution of the performance test program.

[0073] However, in the process of executing HPL to solve the univariate N-order dense linear algebraic equations, HPL performs block-by-block decomposition calculations in a fixed load distribution order, that is, from the left to the right of each row of the first load unit array, and from the top to the bottom of each column of the first load unit array. Since the amount of performance test tasks that need to be performed by the later load units is greater during the execution of HPL, the accumulated computational load is greater, which leads to a problem that the computational load allocated to each load unit is initially fixed, so that after multiple rounds of load distribution, the load distribution results of a group of first load units are unbalanced.

[0074] The first load unit group is a combination of load units in a row or a column of the first load unit array.

[0075] The main component of the computational load in HPL is the update operation of solving the matrix, mainly the matrix multiplication operation in the update operation.

[0076] The HPL algorithm mechanism can ensure that during different rounds of calculation load distribution, the load units do not need to synchronize the calculation progress of each load unit. Therefore, after multiple rounds of calculation load distribution, in the first load unit group of a group, the cumulative results of the calculation load are as balanced as possible, which is an effective way to improve the operating efficiency of HPL.

[0077] That is to say, when performing calculation load distribution, after the performance test program completes multiple rounds of calculation load distribution, the balanced load distribution result means that the cumulative results of the calculation loads of each group of first load units (load distribution results) are balanced, that is, they tend to be equal in theory. This ensures that the HPL has good efficiency and performance.

[0078] During the solution process, HPL distributes the calculation load in a fixed order, that is, in each row of the first load unit array: from left to right, in each column: from top to bottom, the calculation load is distributed in this order, and the decomposed parts will no longer participate in the calculation. However, since HPL usually adopts a static calculation load balancing method, when solving the first load unit array in HPL, the load units in the same row or the same column will have an unbalanced calculation load, that is, the calculation amount of the first load unit group on the left is always smaller than that of the first load unit group on the right, and the calculation amount of the first load unit group on the top is always smaller than that of the first load unit group on the bottom.

[0079] To understand the cumulative change of the calculated load on a group of first load units after the above HPL completes multiple rounds of load (calculated load) distribution, please refer to Figure 2 , Figure 2 It is a schematic diagram of the calculated load distribution results of a first load unit group.

[0080] Figure 2 The example shown is that the first load unit array includes four columns of first load unit groups, that is, one row-wise first load unit group includes load units on four different columns.

[0081] Next, Figure 2 The calculated load distribution results are discussed in detail.

[0082] assumed Figure 2 In the first load unit array shown, the first load unit group in the currently determined row is the decomposition row. So before decomposition, the data scale that needs to be calculated by all P load units in the current row is m, and the size of the matrix that needs to participate in the matrix multiplication is m*m. After the decomposition starts, the load units that are decomposed at this time (decomposition load units or decomposition processes) rotate in turn in the current row. Then among all the load units in this row, the load unit located on the left side of the decomposition load unit has a solution matrix size of (m-NB)*m, and the load unit located on the right side of the decomposition load unit has not been decomposed at this time, so the solution scale of the load unit on the right is still m*m.

[0083] The decomposition process includes two simultaneous processes: decomposition and update. Therefore, from the initial determined decomposition load unit to the current row ( Figure 2 During the process of one round of decomposition, all load cells of the first load cell group (shown) have completed:

[0084] The total number of processed data of the load unit in the first column is: P*(m-NB)*m;

[0085] The total number of processed data of the load unit in the second column is: (P-1)*(m-NB)*m+m*m;

[0086] The total number of processed data of the load unit in the third column is: (P-2)*(m-NB)*m+2*m*m;

[0087] …

[0088] The total number of processed data of the load unit in the Pth column is: (m-NB)*m+(P-1)*m*m.

[0089] In summary, during the decomposition process of the current row, the processing data scale of all load units in the current row will show a linear increase, that is, in the process of HPL distributing the computing load, the computing load allocated to each load unit in each group of the first load unit group increases successively according to the order of distribution.

[0090] Where NB (N Block size) is the block size into which N is logically divided into each matrix block. Each load unit is responsible for processing the matrix block assigned to it in the first load unit array. These matrix blocks are the small blocks into which the matrix N formed by the solution scale is logically divided, and the size of each small block is usually defined by the NB parameter.

[0091] Of course, the total number of data decomposed in the column direction is similar and will not be repeated here.

[0092] Therefore, for a row of load cells (from left to right) or a column of load cells (from top to bottom), the calculated load distribution results are as follows: Figure 2 As shown, the height of the column represents the size of the distributed computing load.

[0093] Based on the computational load distribution rule of the above performance test program: the cumulative result of the computational load increases linearly in the row or column direction of the load unit array. When the linearly increasing computational load is distributed using a fixed computational load distribution order, the final computational load distribution result will be more unbalanced.

[0094] The process of distributing the computational load in a fixed order using HPL can be found in Figure 3 , Figure 3 It is a schematic diagram of the distribution process of a fixed computing load distribution order provided by an embodiment of the present invention.

[0095] HPL usually adopts a unidirectional fixed order (from left to right / from top to bottom) for computing load distribution. In each round of computing load distribution, the computing load is distributed to each load unit in each row or each round in turn according to the unidirectional fixed order.

[0096] As shown in 3 , two rows of first load unit groups in the first load unit array, each row of the first load unit group including P load units, are used as an example for illustration.

[0097] like Figure 3 As shown, in the first load unit group in the first row, the calculation load is distributed to the load units in each column in turn according to the unidirectional calculation load distribution order (data binding order) from left to right, where the calculation load distribution result of the load unit in each column on the first row is as follows: Figure 2 shown.

[0098] In for Figure 3 When allocating the calculated load to each load unit in the first load unit group of the second row shown in FIG. 1 , the calculated load is allocated to the load units in each column in turn according to the unidirectional calculated load allocation order from left to right, wherein the calculated load allocation result of each load unit is still Figure 2 shown.

[0099] By analogy, according to Figure 3 The unidirectional calculation load distribution sequence shown is completed after all rows of the first load unit array are distributed ( Figure 3 After that, multiple rounds of calculation load distribution are completed (taking the first load unit array as 2 rows as an example for demonstration), and at this time, the calculation load accumulation of the load units in the same column is more unbalanced.

[0100] According to the calculation load distribution rule of the above performance test program, the present invention defines a new calculation load distribution strategy to form a new calculation load distribution order.

[0101] The new computing load distribution strategy aims to ensure that after the performance test program completes multiple rounds of computing load distribution, the load distribution results of the computing load of each first load unit group remain balanced. The main process includes:

[0102] When the HPL solution process is in the forward rotation process (i.e., the allocation order of the forward allocation), in one row or one column, the accumulated processing data of the first load cell group on the left is always less than the accumulated processing data of the first load cell group on the right, and the accumulated processing data of the first load cell group on the top is always less than the accumulated processing data of the first load cell group on the bottom. Therefore, the accumulated calculation load of the first load cell group on the left is always less than the accumulated calculation load of the first load cell group on the right, and the accumulated calculation load of the first load cell group on the top is always less than the accumulated calculation load of the first load cell group on the bottom.

[0103] When the HPL solution process is in the reverse rotation process (i.e., the reverse allocation order), in one row or one column, the accumulated processing data of the first load cell group on the left is always greater than the accumulated processing data of the first load cell group on the right, and the accumulated processing data of the first load cell group on the top is always greater than the accumulated processing data of the first load cell group on the bottom. Therefore, the accumulated calculation load of the first load cell group on the left is always greater than the accumulated calculation load of the first load cell group on the right, and the accumulated calculation load of the first load cell group on the top is always greater than the accumulated calculation load of the first load cell group on the bottom.

[0104] The above process can refer to Figure 4 , Figure 4 It is a schematic diagram of the distribution process of calculating the load distribution sequence in the performance testing method provided in the embodiment of the present invention.

[0105] As shown in Figure 4, two rows of first load unit groups in the first load unit array, each row of the first load unit group including P load units, are used as an example for illustration.

[0106] like Figure 4 As shown, in the first load cell group of the first row, the calculation load is first distributed in the order of calculation load distribution (data binding order) from left to right, that is, the forward distribution order of calculation load distribution, and the calculation load is distributed to the load cells of each column in the first load cell group of this row in turn, wherein the calculation load distribution result of the load cells of each column in the first load cell group of the first row is as follows: Figure 2 shown.

[0107] In the first load cell group of the second row, the calculated load is distributed in the order of distribution from right to left, that is, the order of distribution of the calculated load in reverse, and the calculated load is distributed to the load cells of each column in the first load cell group of this row in turn, wherein the calculated load distribution result of the load cells of each column in the first load cell group of the second row is Figure 5 As shown, Figure 5 It is a schematic diagram of the distribution results of the reverse distribution process in the bidirectional computing load distribution sequence provided by an embodiment of the present invention.

[0108] like Figure 5 As shown in Figure 2, in the reverse allocation process, the allocation order changes, while the allocation rule of the HPL computing load does not change. Figure 5 The calculated load distribution result of the first load unit group in the reverse distribution process is shown in FIG. Figure 2 The calculation load distribution results shown are opposite. Therefore, after completing multiple rounds of calculation load distribution, the calculation load distribution results between the first load unit groups in the same column or row can be balanced as a whole.

[0109] Since the load distribution order of the first row and the second row is opposite, but the distribution amount of the calculated load remains unchanged, the accumulation of each load unit in different rows of the same column, that is, the load distribution result is balanced as a whole, thereby maintaining the load distribution result of the performance test program balanced, making the performance test program have better performance and efficiency, and improving the reliability of the test evaluation results of the server to be tested.

[0110] When the load is a calculation load, the step S101 of distributing the load to each load unit of the server to be tested based on a predefined load distribution order during the execution of the performance test program includes:

[0111] During the execution of the performance test program, the computing load is allocated to each load unit based on the computing load allocation strategy defined in the computing load allocation sequence.

[0112] According to different load types, load distribution is performed using the corresponding predefined load distribution sequence, which can flexibly adapt to the application of performance test programs.

[0113] Since the defined computing load distribution strategy adopts a two-way rotation (forward order distribution and reverse order distribution) alternating mode, the load distribution result of the computing load can be kept balanced between each first load unit group, thereby ensuring the execution effect of the performance test program, improving the reliability of the test evaluation result, and making the test evaluation result closer to the actual situation of the server to be tested.

[0114] In one embodiment, during the execution of the performance test program, allocating the computing load to each load unit based on the computing load allocation strategy defined in the computing load allocation sequence includes:

[0115] During the execution of the performance test program, starting from the first load unit group at the initial position of the first load unit array, the planned calculated load of the performance test program is sequentially allocated to each load unit in the first load unit group according to the allocation order of forward allocation;

[0116] and, in the first load unit group of the next group according to the group order, sequentially allocating the planned calculated load of the performance test program to each load unit in the first load unit group of the group according to the allocation order of reverse allocation;

[0117] Alternately executing the above allocation process until all the first load unit groups of the first load unit array have been allocated the calculation load, completing multiple rounds of calculation load allocation of the performance test program;

[0118] Wherein, in the distribution direction indicated by the distribution strategy of the computing load, the value of the planned computing load amount increases successively.

[0119] The planned computing load is as follows: Figure 2 The size of the calculation load distribution shown is that the size of the calculation load distributed to each load unit of the first load unit group in each row or column increases linearly as the distribution direction moves forward.

[0120] The distribution direction indicated by the distribution strategy of the computing load is the forward direction and the reverse direction, that is, the distribution strategy of the computing load defined in the embodiment of the present invention complies with the distribution rule of the computing load in the performance test program, and the purpose of balancing the load distribution results of the computing load is achieved by changing the load distribution direction in the computing load distribution process of each group of load units.

[0121] Since the HPL calculates the load distribution in units of NB*NB blocks, the load distribution is calculated starting from the load unit determined in the current round and the distribution is performed in a round-robin manner. Figure 3 As shown in the figure, taking the row direction as an example, the first block of computing load is allocated to load unit 0 in row direction column 0, the second block of computing load is allocated to load unit 1 in row direction column 1, and so on, the Pth block of data is allocated to load unit P-1 in row direction column P-1. The subsequent computing load allocation continues to repeat the above process until the computing load allocation is completed. The whole process can be abstracted as "one-way round-robin".

[0122] The process of distributing the computing load using the method provided in the embodiment of the present invention can continue to refer to Figure 4 Taking the row direction as an example, the distribution of the adjusted computing load is as follows:

[0123] First, the first block of calculated load is allocated to load unit 0 in row 0 column, the second block of calculated load is allocated to load unit 1 in row 1 column, and so on, the Pth block of data is allocated to load unit P-1 in row P-1 column. This is the rotation process in the distribution direction of forward distribution.

[0124] Furthermore, the P+1th block calculates the load and distributes it to the load unit P-1 in the first load unit group of the second row, the load unit P-1 in the row direction P-1 column, the P+2th block calculates the load and distributes it to the load unit P-2 in the row direction P-2 column, and so on, the 2Pth block calculates the load and distributes it to the load unit 0 in the row direction 0 column. This is the rotation process when the distribution direction is reversed.

[0125] Furthermore, the subsequent computing load distribution continues to repeat the above process until the computing load distribution is completed. This process can be abstracted as "two-way round-robin".

[0126] When allocating to columns, it is the same as allocating to rows.

[0127] By alternating forward allocation and reverse allocation as described above, the cumulative results of the computational loads allocated to the load units in each column or row can be balanced, thereby achieving overall (each row or each round) balance of the load distribution results of the computational loads.

[0128] In order to accurately determine the allocation direction currently used in the first load unit group, in one embodiment, the method may further include:

[0129] In each first load unit group of the first load unit array, determining a current first arrangement number of a load unit to which a calculation load is currently to be allocated;

[0130] Using a multiple of the total number of load units in each first load unit group, a remainder operation is performed on the current first arrangement number, and if the remainder is greater than or equal to the total number of load units in each first load unit group, the currently used allocation direction is maintained, and the calculation load is allocated to each load unit in the first load unit group;

[0131] The current first arrangement number is modulo a multiple of the total number of load units in each group of first load unit groups. If the remainder is less than the total number of load units in each group of first load unit groups, the calculation load is allocated to each load unit in the next group of first load unit groups adjacent to the group of first load unit groups, using a direction opposite to the currently used allocation direction.

[0132] The arrangement number is used to indicate the arrangement position of the load unit currently being decomposed; since the distribution strategy for calculating the load defined in the embodiment of the present invention uses a bidirectional distribution method, it is necessary to determine whether the load unit currently being decomposed is the last load unit of the first load unit group of the current group, so as to determine the distribution direction used when allocating the next load unit.

[0133] Since the allocation direction between two adjacent rows is bidirectionally alternating, the multiple may be 2.

[0134] by Figure 4 Taking the allocation process shown as an example, it is assumed that P=5, the number of rows is still two, and the usage multiple is 2.

[0135] When calculating the load distribution in the first row, if the current arrangement number of the load unit to be distributed is determined to be 6, Figure 4 As shown, the arrangement starts from 0, so the load unit at position 6 can be expressed as: 5+1, then using the multiple of the total number of load units in each first load unit group: 5*2, the remainder operation is performed on the current arrangement number, and the remainder can be obtained as 4, which is less than the number of load units in each row 5 (P is the number of load units on each row), which means that the load units in the first load unit group of the current row have completed the distribution of the calculated load, and the distribution direction opposite to that of the current row is adopted to distribute to the first load unit group of the next row.

[0136] In the load distribution process of calculating the load, the first load unit group described in the above embodiment is a combination of load units in each row or each column in the first load unit array.

[0137] According to the distribution rule of the computing load in the performance test program, in order to keep the load distribution result of the computing load balanced as a whole, a row or a column of the first load unit array is used to ensure the balance of the load distribution result.

[0138] After distributing the calculated load using the above calculation load distribution sequence, the load distribution result of each group of first load units can be referred to Figure 6 As shown, Figure 6 It is a schematic diagram of the load distribution result under the load distribution order calculated in the performance testing method provided in the embodiment of the present invention.

[0139] like Figure 6 As shown, the computational loads of the same column and different rows are generally Figure 2 and Figure 5 The calculated load distribution results shown are complemented with each other so that the accumulation of the calculated loads distributed to the load units in different columns of the first load unit array (the load distribution result of the calculated load) can be balanced.

[0140] In another embodiment, the load includes a communication load; and the load distribution sequence includes: a communication load distribution sequence;

[0141] The step of determining a corresponding load unit array based on the type of load comprises:

[0142] Based on the communication type of the communication load, determining that the corresponding load unit array is a second load unit array;

[0143] The load distribution strategy for distributing loads to each load unit of the determined load unit array in each round of load distribution process of the performance test program is defined to obtain the load distribution sequence, including:

[0144] The communication load distribution strategy is defined as follows: for each group of second load units in the second load unit array, in the second load unit group at the initial position, the communication load received by the target load unit determined in the communication load distribution process of the current round is transmitted one by one to the second load unit group at the non-initial position, so as to determine the target load unit in the second load unit group at the non-initial position;

[0145] And, for each second load unit group in the second load unit array, according to the intra-group cycle order of each second load unit group, starting from the target load unit determined in the current round of communication load distribution process, the communication load received by the target load unit is transmitted to each non-target load unit in the second load unit group in sequence;

[0146] The communication load distribution order is obtained.

[0147] In HPL, the distribution of communication load is a key factor to ensure the communication efficiency and performance in the server. The broadcasting process is a specific communication mode, and its efficiency and effect are directly affected by the distribution strategy of communication load.

[0148] The broadcast process usually occurs in the matrix (unary N-order dense linear algebraic equation system) initialization and preprocessing stage. Broadcast is a specific communication mode that allows a load unit (usually the main process: the target load unit described in the embodiment of the present invention) to send the communication load to all other load units, that is, during the broadcast process, the communication load formed by the decomposition result of each load unit will be communicated and propagated between each row and each column of the second load unit array.

[0149] In HPL, there are a variety of broadcast optimization algorithms for cluster MPP (Massive Parallel Processing) architecture. During the testing of NUMA architecture servers, these broadcast optimization algorithms usually involve multiple cross-socket communication processes, and the order of the communication processes is relatively fixed.

[0150] Therefore, when the broadcast optimization algorithm suitable for MPP architecture is extended to the server of NUMA architecture, due to the complex interconnection structure of the server of NUMA architecture, different types of communication modes and paths will appear, which will also lead to different communication loads. For example, a conventional multi-socket NUMA architecture server will have cross-socket communication and internal communication. Due to the characteristics of the broadcast algorithm used by HPL, some fixed load units always receive cross-socket communication messages, which causes the communication overhead of the fixed load unit to be greater than the communication overhead of other load units in the socket, resulting in unbalanced communication load.

[0151] Based on the above content, it can be known that the communication load allocated in HPL is mainly used in two parts: one part is the row exchange process in the second load unit array, and the other part is the broadcast process (column exchange process in the second load unit array).

[0152] The row exchange process is the communication exchange between load units in different rows in the same column of the second load unit array, which is mainly short messages and has a large number of times. The broadcast process is the communication between load units in different columns of the second load unit array, which has a relatively long message length and a relatively small number of times.

[0153] In a NUMA-based server, memory is divided into multiple modules, each of which is usually closely associated with one or more processors (CPUs), which can quickly access their local memory modules. In a NUMA architecture, each processor or group of processors is usually organized into "sockets," and each socket has its own local memory. When processors access local memory, access is fast because they are within the same socket. However, when a processor needs to access non-local memory (i.e., memory located in other nodes), access is much slower because communication across sockets is required.

[0154] The process of row switching is mapped to the hardware structure. Its correspondence refers to the way in which processes (load units) are mapped to physical memory in a NUMA architecture server, so that the memory access of the load units is concentrated as much as possible on the connection nodes to which they belong. This can reduce memory access across connection nodes, thereby reducing communication delay and load.

[0155] Therefore, each load unit of the second load unit group described above corresponds to each processor connected to a socket, and in the second load array, the second load unit group is a combination of load units in each column (in each socket). Accordingly, the load units in each column of the same row are distributed in different sockets.

[0156] Since the distribution of communication load is mainly reflected in the row exchange process and the broadcast process, and the row exchange process belongs to the communication process between each processor in a socket, that is, the communication between load units in different rows in the same column. Therefore, in the hardware device, the communication overhead between each processor in a socket is relatively small. The broadcast process belongs to the communication process between different sockets, that is, the communication between the second load unit groups in different columns. Therefore, in the hardware device, the communication overhead between different sockets is relatively large.

[0157] In the process of distributing the communication load in software execution, that is, when executing the performance test program (HPL), the distributed communication load is equal. Therefore, when mapped to the hardware device, due to the structural connection and layout characteristics of the hardware device, the processor set at the first position in each socket always receives the communication information sent by the socket located outside, resulting in unbalanced communication loads of each processor in a socket. In some cases, there will be two cross-socket communications in a socket.

[0158] To understand the above communication process based on communication load, please refer to Figure 7 , Figure 7It is a schematic diagram of the communication process under fixed-order communication load distribution provided by an embodiment of the present invention.

[0159] like Figure 7 As shown, each socket corresponds to each column in the second load unit array, and in the second load unit group in each column, for example, the first column of load units in the first second load unit group (the load unit in the first row and first column in the second load unit array), its communication load during the broadcast process is higher than the communication load of the load units in other columns on this socket.

[0160] And in Figure 7 In a complete broadcast process shown in FIG. 1 , when the root load unit is not the load unit in the first column of the second load unit group of a group, an additional cross-socket broadcast communication process will occur, such as Figure 7 As shown in the bold dashed line.

[0161] Therefore, in the distribution of the communication load based on the fixed sequence, the communication loads of the load units in each group of the second load units are not balanced.

[0162] In order to solve the above problem, the embodiment of the present invention improves the broadcast algorithm for distributing the communication load during the broadcast process. Based on the above analysis, it can be known that the imbalance of the communication load occurs within a socket (within a group of second load units), which is mainly because in the hardware device, since the setting position of the processor is fixed, when the communication load is distributed in a fixed order, the processor set at the first position will always receive the communication information of the external socket, so the communication overhead of the processor at the first position is greater than the communication overhead of other processors on this socket.

[0163] Based on this, in the embodiment of the present invention, by improving the software algorithm, the distribution order of the communication load is changed in the software process to avoid always fixing the load unit that communicates with the external socket as the load unit at the first position. That is, the distribution strategy of the communication load defined in the embodiment of the present invention mentioned above.

[0164] In order to flexibly change the load unit receiving the communication of the external socket, the communication load distribution strategy defined in the embodiment of the present invention includes two parts:

[0165] First, by using the basic process of the HPL algorithm for communication load distribution, the function of the target load unit determined in the current round is changed, that is, the target load unit in other columns of the second load unit group is determined by using the currently determined target load unit, so that the communication load between each second load unit group is received by the target load unit in the first group of the second load unit group;

[0166] In the second aspect, the order of distributing the communication loads between the load units in each group of second load units is changed from Figure 7 The unidirectional sequential transmission shown is changed to a cyclic transmission between load cells within a second set of load cells.

[0167] Since HPL distributes the communication load by rotating the load units within a group of second load units, based on the communication load distribution strategy defined above, each load unit within a group of second load units can be responsible for transmitting the communication load to an external second load unit group, so the communication load of each load unit can be balanced at the hardware level.

[0168] After defining the communication load distribution strategy and obtaining the communication load distribution mode, in the process of executing the performance test program, based on the predefined load distribution order, the load is distributed to each load unit of the server to be tested, including:

[0169] During the execution of the performance test program, the communication load is allocated to each load unit based on the communication load allocation strategy defined in the communication load allocation sequence.

[0170] The communication load distribution strategy in the communication load distribution sequence can be based on the target load unit selected in each round of the HPL algorithm, and the role of the target load unit can be changed, so that in the load distribution process of each round of the communication load, it is not only responsible for transmitting the communication load within the second load unit group, but also responsible for transmitting the communication load to each external second load unit group. In this way, the load unit receiving the communication load of the external second load unit group is avoided from being fixed, and the purpose of balancing the communication load between each load unit in the second load unit group is achieved.

[0171] At the same time, based on the communication load distribution method, each second load unit group executes a cyclic transmission communication load method, thereby avoiding the following Figure 7 As shown, there is a situation where a second load unit group transmits the communication load to other external second load unit groups twice, which can also achieve the purpose of reducing the communication load.

[0172] In one embodiment, during the execution of the performance test program, allocating the communication load to each load unit based on the communication load allocation strategy defined in the communication load allocation sequence includes:

[0173] During execution of the performance testing procedure, determining a target load cell in a second load cell group at an initial position of the second load cell array;

[0174] Based on the target load unit determined in the second load unit group at the initial position during the current round of communication load distribution, the target load unit in the second load unit group at the non-initial position is determined; and the target load unit determined in the second load unit group at the initial position transmits the received communication load to the target load unit in the second load unit group at the non-initial position one by one;

[0175] In each group of second load units, the communication load received by the target load unit is transmitted to each non-target load unit in sequence according to the cycle order within the group;

[0176] Until each load unit in the second load unit group of each group has been used as a target load unit to transmit the communication load, multiple rounds of communication load distribution of the performance test program are completed.

[0177] The second load unit group at the initial position may be a second load unit group corresponding to a socket that is first experienced when distributing communication load on the server to be tested.

[0178] In the communication load distribution sequence provided by the embodiment of the present invention, the defined communication load distribution strategy includes: improvement between the second load unit groups and internal improvement within each second load unit group.

[0179] Therefore, among the second load unit groups, the target load unit in other second load unit groups can be determined according to the target load unit determined by the second load unit group at the initial position during the load distribution process of the current round of communication load, thereby realizing the distribution of communication loads between different second load unit groups.

[0180] After the target load unit in each group of second load units receives the communication load, the communication load received by the target load unit can be cyclically transmitted among the non-target load units in each group of second load units, thereby realizing the distribution of the communication load in each group of second load units.

[0181] Since the target load unit will change during each round of communication load distribution, the load units in each group of second load units have one chance to be the target load unit. Therefore, based on the communication load distribution strategy defined in the embodiment of the present invention, each load unit in each group of second load units can assume the responsibility of transmitting the communication load to other external second load unit groups, that is, to achieve communication load balancing of each load unit in each group of second load units.

[0182] At the same time, due to the cyclic transmission mechanism within the second load unit group, the process of transmitting the communication load across the second load unit group twice within one second load unit group can be avoided, thereby reducing the communication load.

[0183] In one embodiment, the target load unit determined from the second load unit group at the initial position transmits the received communication load to the target load units in the second load unit group at the non-initial position one by one, including:

[0184] Based on the target load unit determined in the second load unit group at the initial position during the current round of communication load distribution, a corresponding current second arrangement number is determined;

[0185] Determine, according to the current second arrangement number and the number of load units included in each second load unit group, a current second arrangement number for indicating a target load unit in the second load unit group at a non-initial position;

[0186] The target load units in the second load unit group at the initial position transmit the received communication loads to the target load units in the non-initial load unit group one by one.

[0187] The current second arrangement number indicates the arrangement position of the target load unit in the second load unit group at the initial position. Since the total number of load units and the number of sockets of the server to be tested are known, the number of load units on each connection node can be calculated, that is, the number of load units in each group of second load units can be calculated.

[0188] Assuming that the number of load units of the server to be tested is x and the number of connection nodes is k, the number l of load units on each group of second load units is: x / k.

[0189] Therefore, in the communication load distribution process of the current round, after the second load unit group at the initial position determines the current second arrangement number of the target load unit, for example, the current second arrangement number is i, 0≦i≦l-1.

[0190] Since each second load unit group includes l load units, the current second arrangement number of the target unit in the second load unit groups at other non-initial positions can be obtained based on the current second arrangement number i and the number l of the load units.

[0191] For example, taking k=2 as an example, according to the current second arrangement number i and the number of load units l, the second arrangement number corresponding to the target load unit in the second load unit group at the second non-initial position can be determined: i+l. Of course, when k=3, the current second arrangement number corresponding to the target load unit in the second load unit group at other non-initial positions can be expressed as: i+l, i+2l. Similarly, according to the number of k and the number of l, as well as the current second arrangement number i, the target load unit in the second load unit group at other non-initial positions can be determined.

[0192] Furthermore, the received communication load can be transmitted to the target load unit in the second load unit group at the initial position by using the target load unit determined in the second load unit group at the initial position, thereby realizing the distribution of the communication load between each group of second load unit groups.

[0193] After completing the distribution of the communication loads between each group of second load unit groups, the communication loads are distributed within each group of second load unit groups. In one embodiment, within each group of second load unit groups, the communication loads received by the target load unit are sequentially transmitted to each non-target load unit according to the cycle order within the group, including:

[0194] In each group of second load units, determining whether the current second arrangement number of the target load unit determined in the communication load distribution process of the current round is the last arrangement number;

[0195] If yes, the communication load received by the target load unit is allocated to the non-target load unit corresponding to the initial arrangement number;

[0196] If not, the communication load received by the target load unit is distributed to the non-target load unit corresponding to the next adjacent second arrangement number.

[0197] When determining the target load unit in the second load unit group at the initial position, the current second arrangement number i at this time and the current second arrangement numbers i+1, i+21, ..., i+(k-1)1 of the target load units in the second load unit group at the non-initial position can be determined. Therefore, in each second load unit group, the current second arrangement number and the last arrangement number of the last load unit in this group can be determined.

[0198] If the current second arrangement number is the load unit at the last position in the second load unit group, then when the next communication load is transmitted, the communication load will be transmitted to the non-target load unit ranked first in the second load unit group.

[0199] If it is not the last load unit, the communication load can be continuously transmitted downward, that is, transmitted to the next adjacent non-target load unit.

[0200] By performing a cross-socket transmission between external second load unit groups and a cyclic transmission between the load units of the internal second load unit group, each load unit can be used as the subject of a cross-socket transmission, balancing the communication load between the load units within a group of second load unit groups, and reducing the communication load between different second load unit groups.

[0201] The communication load distribution process provided by the above embodiment of the present invention can refer to Figure 8 , Figure 8 The present invention is a schematic diagram of a process for implementing communication load distribution based on the communication load distribution strategy provided in an embodiment of the present invention.

[0202] Assume that the total number of load units is x and the server to be tested has k ( Figure 8 As shown, there are k=2) sockets. Then there are l=x / k load units on each socket, that is, each column of the second load unit group has l rows of load units. To facilitate the description of the communication load distribution of the load units, each column of the load units in the second load unit group is represented in the form of a row.

[0203] Figure 8 In the figure, i=1, k=2 is used as an example. Figure 8 shown.

[0204] The communication load distribution process includes:

[0205] First, according to the second load unit group at the initial position, the target load unit is determined in the current round of communication load distribution process, and the current second arrangement number 1 of the target load unit is determined, and then the current second arrangement number 1+1 corresponding to the target load unit in the second load unit group of the second group is determined.

[0206] Furthermore, among the 1 load units distributed in each group of the second load units, the communication load is rotated in an internal cycle until all the load units in the second load unit group obtain the communication load.

[0207] Since the target load unit is transferred between the second load unit groups of a group by the column process, the target load unit receiving the cross-socket communication load is evenly borne by each column of the load unit in the second load unit group, and is no longer borne by the load unit in the first column of the second load unit group. At the same time, by adopting the communication load distribution sequence provided by the embodiment of the present invention, it is possible to use the transmission of the communication load within the second load unit group once, instead of the complex transmission of the communication across the socket twice, so as to achieve the purpose of reducing the transmission communication load between different second load unit groups.

[0208] It can be seen that, through the above communication load distribution sequence, the communication load of each load unit in the second load unit group can be uniformly optimized.

[0209] In order to make the performance test program have a better test effect, before executing the performance test program, you can configure the initial parameters of the performance test program, select a performance test program with the best test evaluation results based on different configurations, and perform subsequent load distribution.

[0210] Please refer to Fig. 9 , Fig. 9 It is another flowchart of the performance testing method provided by an embodiment of the present invention.

[0211] like Fig. 9 As shown, the method may include the following steps:

[0212] Step S200: determining a server to be tested, wherein the server to be tested includes a plurality of load units, each of which is used to execute a performance test task.

[0213] Step S201, obtaining an initial performance test program, wherein the initial performance test program defines a performance test task, and the solution scale of the performance test task has an adjustment coefficient.

[0214] The adjustment coefficient is determined based on the memory space and the number of computing cores of the server to be tested.

[0215] Step S202, determining different solution scales based on different adjustment coefficients, and executing initial performance test programs with different solution scales on the server to be tested to obtain various initial test evaluation results.

[0216] Step S203, selecting an initial test evaluation result of an optimal computing load distribution result, and using an initial performance test program corresponding to the selected initial test evaluation result as a performance test program executed on the server to be tested.

[0217] Step S204, executing a performance testing program on the server to be tested, wherein the performance testing program includes a multi-round load distribution process; and in the process of executing the performance testing program, distributing loads to various load units of the server to be tested based on a predefined load distribution order.

[0218] Step S205: obtaining a test evaluation result of the server to be tested.

[0219] The solution scale is the dimension of the matrix formed by the univariate N-order dense linear algebraic equations solved in HPL, and the solution scale can be represented by N. The adjustment of the solution scale directly affects the distribution of the computing load, and may also indirectly affect the distribution of the communication load. Solving the univariate N-order dense linear algebraic equations is the performance test task.

[0220] Therefore, in order to make the performance testing method have better efficiency and performance when using the load distribution sequence provided by the embodiment of the present invention, the performance testing program is preprocessed, and the solution scale is adjusted on the basis of the initial performance testing program (i.e., the standard performance testing program), so as to obtain different test evaluation results. Since the adjustment of the solution scale directly affects the distribution of the computing load, when selecting the performance testing program, the initial performance testing program with the optimal computing load distribution result can be selected as the performance testing program executed on the server to be tested.

[0221] In HPL, the solution scale N can be set according to the HPL setting guidance file. For example, the memory capacity, computing load and communication load of the server to be tested, as well as the expected performance target can be comprehensively considered. Through reasonable settings, the computing resources of the server to be tested can be fully utilized while avoiding resource waste and bottlenecks. In actual operation, it may be necessary to perform multiple tests and adjustments according to the specific situation of the server to be tested to find the best N value.

[0222] Exemplarily, the embodiment of the present invention adjusts N based on the basic algorithm framework of HPL, assuming that the load unit array is a P*P square matrix, and adjusts the total solution scale N by adjusting the calculation step: block size NB.

[0223] First, select a suitable NB.

[0224] Further, the balance optimization of the computing load is performed, that is, appropriate parameter selection is performed to obtain the optimal solution scale N. The formula for solving the scale N used in the HPL algorithm in the embodiment of the present invention is:

[0225] N = n*P*NB+NB*(P / 2);

[0226] In the above formula, n is the adjustment coefficient, which is an integer. The selection of this value depends on the size of the memory space and the number of computing cores of the server to be tested (for example, a server with a NUMA architecture).

[0227] After N is set using the above formula, different solution scales N can be obtained by selecting different values ​​of n.

[0228] And in the solution scale calculated under each value of n, the corresponding initial performance test program is executed on the service to be tested, and the test evaluation result of the initial performance test program under each value of n is obtained.

[0229] According to the analysis of the above content, based on the parallel load distribution mode of HPL, in a parallel row (or column) of the load unit array, the load (computational load) scale allocated to the first half of the load units is always greater than the load scale of the second half of the load units.

[0230] Therefore, among the test evaluation results obtained by the initial performance test program under multiple n values, the initial test evaluation result with the optimal calculation load distribution result is selected. The optimal calculation load distribution result means that in the calculation load of the load unit of a certain row, the calculation load of the load unit of the right half is reduced. And the calculation load of the process of a certain column, the load of the lower half of the process is reduced. The calculation load of the front and back parts of a certain row (or column) of the load unit array is balanced.

[0231] After the preliminary balance optimization of computing balance is performed through the above steps S201-203, the optimization result of computing load obtained can be referred to Fig.10 , Fig.10 It is a result schematic diagram of the optimal calculation load distribution result provided by an embodiment of the present invention.

[0232] like Fig.10 As shown, the calculated load of a load cell in a row (from left to right) or a load cell in a column (from top to bottom) of the load cell array, and Figure 2 Compared with the overall uneven computing load, it is significantly balanced.

[0233] Therefore, by adopting the above steps S201-203, an initial performance test program that can perform preliminary optimization processing on the distribution of computing load is selected as the performance test program executed on the server to be tested, which can facilitate subsequent load balancing processing based on the load distribution order.

[0234] An embodiment of the present invention further provides a performance testing device for implementing the performance testing method provided by any of the aforementioned embodiments.

[0235] Please refer to Fig.11 , Fig.11It is a structural schematic diagram of a performance testing device provided by an embodiment of the present invention.

[0236] like Fig.11 As shown, the device may include:

[0237] The server to be tested determining module 30 is used to determine the server to be tested, wherein the server to be tested includes a plurality of load units, each of which is used to perform a performance test task;

[0238] A program execution module 31 is used to execute a performance test program on the server to be tested, wherein the performance test program includes a multi-round load distribution process; and in the process of executing the performance test program, the load is distributed to each load unit of the server to be tested based on a predefined load distribution sequence, wherein the load distribution sequence is used to balance the load distribution results after the performance test program ends the multi-round load distribution;

[0239] The execution result obtaining module 32 is used to obtain the test evaluation result of the server to be tested.

[0240] Optionally, the device further comprises: a load distribution order definition module, configured to define a load distribution order based on the type of load;

[0241] The load distribution sequence definition module is used to define the load distribution sequence based on the load type, including:

[0242] Based on the type of the load, determining a corresponding load unit array, wherein the load unit array is composed of various load units;

[0243] A load distribution strategy for distributing loads to each load unit of the determined load unit array is defined in each round of load distribution process of the performance test program to obtain the load distribution sequence.

[0244] Optionally, the load includes a calculation load of a calculation type; the load distribution sequence includes: a calculation load distribution sequence;

[0245] The load distribution sequence definition module is used to determine the corresponding load unit array based on the type of load, including:

[0246] Based on the calculation type of the calculated load, determining that the corresponding load unit array is a first load unit array;

[0247] The load distribution sequence definition module is used to define a load distribution strategy for distributing loads to each load unit of the determined load unit array in each round of load distribution process of the performance test program to obtain the load distribution sequence, including:

[0248] The distribution strategy of the calculated load is defined as follows: according to the group order of each group of first load unit groups in the first load unit array, a distribution order of alternating forward distribution and reverse distribution is defined for the plurality of groups of first load unit groups, so that each load unit in adjacent groups of load unit groups performs forward distribution and reverse distribution in turn;

[0249] Obtaining the calculation load distribution order;

[0250] The forward distribution is defined as distribution according to the positive arrangement direction of each load unit in each unit group; the reverse distribution is defined as distribution according to the reverse arrangement direction of each load unit in each unit group.

[0251] Optionally, the load includes a communication load of a communication type; the load distribution sequence includes: a communication load distribution sequence;

[0252] The load distribution sequence definition module is used to determine the corresponding load unit array based on the type of load, including:

[0253] Based on the communication type of the communication load, determining that the corresponding load unit array is a second load unit array;

[0254] The load distribution sequence definition module is used to define a load distribution strategy for distributing loads to each load unit of the determined load unit array in each round of load distribution process of the performance test program to obtain the load distribution sequence, including:

[0255] The communication load distribution strategy is defined as follows: for each group of second load units in the second load unit array, in the second load unit group at the initial position, the communication load received by the target load unit determined in the communication load distribution process of the current round is transmitted one by one to the second load unit group at the non-initial position, so as to determine the target load unit in the second load unit group at the non-initial position;

[0256] And, for each second load unit group in the second load unit array, according to the intra-group cycle order of each second load unit group, starting from the target load unit determined in the current round of communication load distribution process, the communication load received by the target load unit is transmitted to each non-target load unit in the second load unit group in sequence;

[0257] The communication load distribution order is obtained.

[0258] Optionally, the device further comprises:

[0259] An initial performance test program acquisition module, used to acquire an initial performance test program, wherein the initial performance test program defines a performance test task, and the solution scale of the performance test task has an adjustment coefficient, and the adjustment coefficient is determined based on the memory space and the number of computing cores of the server to be tested;

[0260] An initial test module, used to determine different solution scales based on different adjustment coefficients, and execute initial performance test programs with different solution scales on the server to be tested to obtain various initial test evaluation results;

[0261] The performance sequencing program selection module is used to select the initial test evaluation result of the optimal computing load distribution result, and use the initial performance test program corresponding to the selected initial test evaluation result as the performance test program executed on the server to be tested.

[0262] It can be seen that the technical solution provided by the embodiment of the present invention, when executing the performance test program to perform performance testing on the server to be tested, by changing the load distribution order used by the performance test program, the performance test program can distribute the load for each load unit of the server to be tested based on the predefined load distribution order when distributing the load; because the load distribution order can ensure that the load distribution result of the performance test program is balanced after the multiple rounds of load distribution process are completed; avoid the situation that the load distribution result cannot be balanced due to the characteristics of the performance test program after the performance test program performs multiple rounds of load distribution for each load unit based on the original load distribution order. After the performance test program ends multiple rounds of load distribution, whether the load distribution result can be balanced affects the efficiency and performance of the subsequent performance test program to test the server to be tested, and then affects the reliability of the test evaluation results. Therefore, in the embodiment of the present invention, by designing a predefined load distribution order, it is ensured that the load distribution result can be balanced after the multiple rounds of load distribution are completed, thereby improving the efficiency and performance of the performance test program and improving the reliability of the test evaluation results.

[0263] An embodiment of the present invention provides an electronic device, such as a terminal device, a server device, or other computer device, including a memory and a processor, wherein the memory stores a program, and the processor calls the program stored in the memory to execute a performance testing method as described in any of the aforementioned embodiments.

[0264] An embodiment of the present invention provides a storage medium, wherein the storage medium stores a program, and when the program is executed, the performance testing method as described in any of the above embodiments is implemented.

[0265] An embodiment of the present invention provides a computer program product, including a computer program, and when the computer program is executed by a processor, the performance testing method as described in any one of the above embodiments is implemented.

[0266] The above describes multiple implementation schemes provided by the embodiments of the present invention. The various optional methods introduced in the implementation schemes can be combined and cross-referenced with each other without conflict, thereby extending a variety of possible implementation schemes, which can all be considered as implementation schemes disclosed and open in the embodiments of the present invention.

[0267] Although the embodiments of the present invention are disclosed above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A performance testing method, characterized in that: include: Determine a server to be tested, wherein the server to be tested includes a plurality of load units, each load unit being used to perform a performance test task; Execute a performance test program on the server to be tested, wherein the performance test program includes multiple rounds of load distribution process; And, during the execution of the performance test program, loads are distributed to the load units of the server to be tested based on a predefined load distribution sequence, wherein the load distribution sequence is used to balance the load distribution results after the performance test program completes multiple rounds of load distribution; A test evaluation result of the server to be tested is obtained.

2. The performance testing method according to claim 1, characterized in that: Before the step of executing the performance test program on the server to be tested, the method further includes: defining a load distribution order based on the type of load; the method of defining a load distribution order based on the type of load includes: Based on the type of the load, determining a corresponding load unit array, wherein the load unit array is composed of various load units; A load distribution strategy for distributing loads to each load unit of the determined load unit array is defined in each round of load distribution process of the performance test program to obtain the load distribution sequence.

3. The performance testing method according to claim 2, characterized in that: The load includes a computational load of a computational type; The load distribution sequence includes: calculating the load distribution sequence; The step of determining a corresponding load unit array based on the type of load comprises: Based on the calculation type of the calculated load, determining that the corresponding load unit array is a first load unit array; The load distribution strategy for distributing loads to each load unit of the determined load unit array in each round of load distribution process of the performance test program is defined to obtain the load distribution sequence, including: The distribution strategy of the calculated load is defined as follows: according to the group order of each group of first load unit groups in the first load unit array, a distribution order of alternating forward distribution and reverse distribution is defined for the plurality of groups of first load unit groups, so that each load unit in adjacent groups of load unit groups performs forward distribution and reverse distribution in turn; Obtaining the calculation load distribution order; The forward distribution is defined as distribution according to the positive arrangement direction of each load unit in each unit group; the reverse distribution is defined as distribution according to the reverse arrangement direction of each load unit in each unit group.

4. The performance testing method according to claim 3, characterized in that: In the process of executing the performance test program, based on a predefined load distribution order, loads are distributed to each load unit of the server to be tested, including: During the execution of the performance test program, the computing load is allocated to each load unit based on the computing load allocation strategy defined in the computing load allocation sequence.

5. The performance testing method according to claim 4, characterized in that: In the process of executing the performance test program, based on the calculation load distribution strategy defined in the calculation load distribution sequence, the calculation load is distributed to each load unit, including: During the execution of the performance test program, starting from the first load unit group at the initial position of the first load unit array, the planned calculated load of the performance test program is sequentially allocated to each load unit in the first load unit group according to the allocation order of forward allocation; and, in the first load unit group of the next group according to the group order, sequentially allocating the planned calculated load of the performance test program to each load unit in the first load unit group of the group according to the allocation order of reverse allocation; Alternately executing the above allocation process until all the first load unit groups of the first load unit array have been allocated the calculation load, completing multiple rounds of calculation load allocation of the performance test program; Wherein, in the distribution direction indicated by the distribution strategy of the computing load, the value of the planned computing load amount increases successively.

6. The performance testing method according to claim 5, characterized in that: Also includes: In each first load unit group of the first load unit array, determining a current first arrangement number of a load unit to which a calculation load is currently to be allocated; Using a multiple of the total number of load units in each first load unit group, a remainder operation is performed on the current first arrangement number, and if the remainder is greater than or equal to the total number of load units in each first load unit group, the currently used allocation direction is maintained, and the calculation load is allocated to each load unit in the first load unit group; The current first arrangement number is modulo a multiple of the total number of load units in each group of first load unit groups. If the remainder is less than the total number of load units in each group of first load unit groups, the calculation load is allocated to each load unit in the next group of first load unit groups adjacent to the group of first load unit groups, using a direction opposite to the currently used allocation direction.

7. The performance testing method according to any one of claims 3 to 6, characterized in that: A first load unit group is a combination of load units in each row or each column in the first load unit array.

8. The performance testing method according to claim 2, characterized in that: The load includes a communication load of a communication type; the load distribution sequence includes: a communication load distribution sequence; The step of determining a corresponding load unit array based on the type of load comprises: Based on the communication type of the communication load, determining that the corresponding load unit array is a second load unit array; The load distribution strategy for distributing loads to each load unit of the determined load unit array in each round of load distribution process of the performance test program is defined to obtain the load distribution sequence, including: The communication load distribution strategy is defined as follows: for each group of second load units in the second load unit array, in the second load unit group at the initial position, the communication load received by the target load unit determined in the communication load distribution process of the current round is transmitted one by one to the second load unit group at the non-initial position, so as to determine the target load unit in the second load unit group at the non-initial position; And, for each second load unit group in the second load unit array, according to the intra-group cycle order of each second load unit group, starting from the target load unit determined in the current round of communication load distribution process, the communication load received by the target load unit is transmitted to each non-target load unit in the second load unit group in sequence; The communication load distribution order is obtained.

9. The performance testing method according to claim 8, characterized in that: In the process of executing the performance test program, based on a predefined load distribution order, loads are distributed to each load unit of the server to be tested, including: During the execution of the performance test program, the communication load is allocated to each load unit based on the communication load allocation strategy defined in the communication load allocation sequence.

10. The performance testing method according to claim 9, characterized in that: In the process of executing the performance test program, based on the communication load distribution strategy defined in the communication load distribution sequence, the communication load is distributed to each load unit, including: During execution of the performance testing procedure, determining a target load cell in a second load cell group at an initial position of the second load cell array; Based on the target load unit determined in the second load unit group at the initial position during the current round of communication load distribution, the target load unit in the second load unit group at the non-initial position is determined; and the target load unit determined in the second load unit group at the initial position transmits the received communication load to the target load unit in the second load unit group at the non-initial position one by one; In each group of second load units, the communication load received by the target load unit is transmitted to each non-target load unit in sequence according to the cycle order within the group; Until each load unit in the second load unit group of each group has been used as a target load unit to transmit the communication load, multiple rounds of communication load distribution of the performance test program are completed.

11. The performance testing method according to claim 10, characterized in that: The target load unit determined from the second load unit group at the initial position transmits the received communication loads one by one to the target load units in the second load unit group at the non-initial position, including: Based on the target load unit determined in the second load unit group at the initial position during the current round of communication load distribution, a corresponding current second arrangement number is determined; Determine, according to the current second arrangement number and the number of load units included in each second load unit group, a current second arrangement number for indicating a target load unit in the second load unit group at a non-initial position; The target load units in the second load unit group at the initial position transmit the received communication loads to the target load units in the non-initial load unit group one by one.

12. The performance testing method according to claim 11, characterized in that: The method of transmitting the communication load received by the target load unit to each non-target load unit in sequence in each group of second load units according to the cycle sequence within the group includes: In each group of second load units, determining whether the current second arrangement number of the target load unit determined in the communication load distribution process of the current round is the last arrangement number; If yes, the communication load received by the target load unit is allocated to the non-target load unit corresponding to the initial arrangement number; If not, the communication load received by the target load unit is distributed to the non-target load unit corresponding to the next adjacent second arrangement number.

13. The performance testing method according to any one of claims 8 to 12, characterized in that: A second load cell group is a combination of load cells in each column of the second load cell array.

14. The performance testing method according to claim 1, characterized in that: Before the step of executing the performance test program on the server to be tested, the method further includes: Obtaining an initial performance test program, wherein a performance test task is defined in the initial performance test program, and a solution scale of the performance test task has an adjustment coefficient, and the adjustment coefficient is determined based on the memory space and the number of computing cores of the server to be tested; Determine different solution scales based on different adjustment coefficients, and execute initial performance test programs with different solution scales on the server to be tested to obtain initial test evaluation results; An initial test evaluation result of the optimal computing load distribution result is selected, and an initial performance test program corresponding to the selected initial test evaluation result is used as a performance test program executed on the server to be tested.

15. A performance testing device, characterized in that: include: A server to be tested determining module, used to determine a server to be tested, wherein the server to be tested includes a plurality of load units, each of which is used to perform a performance test task; A program execution module, used for executing a performance test program on the server to be tested, wherein the performance test program includes a multi-round load distribution process; And, during the execution of the performance test program, loads are distributed to the load units of the server to be tested based on a predefined load distribution sequence, wherein the load distribution sequence is used to balance the load distribution results after the performance test program completes multiple rounds of load distribution; The execution result obtaining module is used to obtain the test evaluation result of the server to be tested.

16. The performance testing device according to claim 15, characterized in that: Also includes: A load distribution order definition module is used to define the load distribution order based on the load type; The load distribution sequence definition module is used to define the load distribution sequence based on the load type, including: Based on the type of the load, determining a corresponding load unit array, wherein the load unit array is composed of various load units; A load distribution strategy for distributing loads to each load unit of the determined load unit array is defined in each round of load distribution process of the performance test program to obtain the load distribution sequence.

17. The performance testing device according to claim 16, characterized in that: The load includes a calculation load of a calculation type; the load distribution sequence includes: a calculation load distribution sequence; The load distribution sequence definition module is used to determine the corresponding load unit array based on the type of load, including: Based on the calculation type of the calculated load, determining that the corresponding load unit array is a first load unit array; The load distribution sequence definition module is used to define a load distribution strategy for distributing loads to each load unit of the determined load unit array in each round of load distribution process of the performance test program to obtain the load distribution sequence, including: The distribution strategy of the calculated load is defined as follows: according to the group order of each group of first load unit groups in the first load unit array, a distribution order of alternating forward distribution and reverse distribution is defined for the plurality of groups of first load unit groups, so that each load unit in adjacent groups of load unit groups performs forward distribution and reverse distribution in turn; Obtaining the calculation load distribution order; The forward distribution is defined as distribution according to the positive arrangement direction of each load unit in each unit group; the reverse distribution is defined as distribution according to the reverse arrangement direction of each load unit in each unit group.

18. The performance testing device according to claim 16, characterized in that: The load includes a communication load of a communication type; the load distribution sequence includes: a communication load distribution sequence; The load distribution sequence definition module is used to determine the corresponding load unit array based on the type of load, including: Based on the communication type of the communication load, determining that the corresponding load unit array is a second load unit array; The load distribution sequence definition module is used to define a load distribution strategy for distributing loads to each load unit of the determined load unit array in each round of load distribution process of the performance test program to obtain the load distribution sequence, including: The communication load distribution strategy is defined as follows: for each group of second load units in the second load unit array, in the second load unit group at the initial position, the communication load received by the target load unit determined in the communication load distribution process of the current round is transmitted one by one to the second load unit group at the non-initial position, so as to determine the target load unit in the second load unit group at the non-initial position; And, for each second load unit group in the second load unit array, according to the intra-group cycle order of each second load unit group, starting from the target load unit determined in the current round of communication load distribution process, the communication load received by the target load unit is transmitted to each non-target load unit in the second load unit group in sequence; The communication load distribution order is obtained.

19. The performance testing device according to any one of claims 15 to 18, characterized in that: Also includes: An initial performance test program acquisition module, used to acquire an initial performance test program, wherein the initial performance test program defines a performance test task, and the solution scale of the performance test task has an adjustment coefficient, and the adjustment coefficient is determined based on the memory space and the number of computing cores of the server to be tested; An initial test module, used to determine different solution scales based on different adjustment coefficients, and execute initial performance test programs with different solution scales on the server to be tested to obtain various initial test evaluation results; The performance sequencing program selection module is used to select the initial test evaluation result of the optimal computing load distribution result, and use the initial performance test program corresponding to the selected initial test evaluation result as the performance test program executed on the server to be tested.

20. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a program, and the processor calls the program stored in the memory to execute the performance testing method according to any one of claims 1 to 14.

21. A storage medium, characterized in that: The storage medium stores a program, and when the program is executed, the performance testing method according to any one of claims 1 to 14 is implemented.

22. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the performance testing method according to any one of claims 1 to 14 is implemented.

Images