Performance test method, device, equipment and medium

Abstract

The invention relates to the field of performance testing, in particular to a performance testing method and device, equipment and a medium, and the method comprises the steps that in the single-script multi-process testing process, the process number of a first target testing script is gradually increased for testing, and first performance data of the first target testing script is obtained; in the multi-script and multi-process testing process, the process number of each second target testing script of the at least two second target testing scripts is gradually increased for testing, and second performance data of the at least two second target testing scripts is obtained, the at least two second target test scripts are at least two scripts which are not associated in the plurality of test scripts; determining the performance of the baseboard management controller according to the first performance data and the second performance data; the performance of the multiple test scripts during concurrent execution can be tested, so that the performance of the substrate management controller can be reflected more accurately.

Description

Technical Field

[0001] The present application relates to the field of performance testing, and in particular to a performance testing method, device, equipment and medium. Background Art

[0002] With the rapid development of data centers and cloud computing, server stability and performance have become key factors. As an important component of the server, the performance of the BMC (Baseboard Management Controller) directly affects the management and monitoring capabilities of the server. Therefore, basic performance evaluation of the BMC is an important step to ensure the stable operation of the server.

[0003] Currently, the performance evaluation of BMC mostly adopts automated testing based on specific test software. Although it can evaluate the performance of BMC to a certain extent, the automated testing based on specific test software often focuses on the verification of a single function. The impact of concurrent operations on BMC performance is particularly important in actual operation and maintenance scenarios.

[0004] Therefore, how to provide a method capable of evaluating the performance of a BMC under concurrent operations is a technical problem to be solved urgently by those skilled in the art. Summary of the invention

[0005] The purpose of this application is to provide a performance testing method, apparatus, device and medium, which can evaluate the BMC performance under concurrent operations.

[0006] In a first aspect, a performance testing method is provided, comprising:

[0007] In a single-script multi-process test process, gradually increase the number of processes of a first target test script for testing, and obtain first performance data of the first target test script, wherein the first target test script is any script among a plurality of test scripts, and the first performance data is determined based on a start timestamp and an end timestamp;

[0008] In a multi-script multi-process test process, gradually increase the number of processes of each second target test script of at least two second target test scripts for testing, and obtain second performance data of the at least two second target test scripts, wherein the at least two second target test scripts are at least two scripts that are not associated with each other in the multiple test scripts;

[0009] The performance of the baseboard management controller is determined according to the first performance data and the second performance data.

[0010] In a preferred example, the present application may be further configured as follows: gradually increasing the number of processes of the first target test script to perform the test, and obtaining first performance data of the first target test script, including:

[0011] Testing the first target test script multiple times with a first target number of processes to obtain multiple execution times, wherein the first target number of processes is any number from 1 to a first maximum number of processes;

[0012] Determine first performance data corresponding to the target process quantity according to the plurality of execution times, the first performance data comprising: an average value, a standard deviation, a maximum value, and a minimum value of the execution time;

[0013] Obtaining first performance data of a first target test script according to first performance data corresponding to each first target process quantity;

[0014] Accordingly, gradually increasing the number of processes of each of the at least two second target test scripts for testing to obtain second performance data of the at least two second target test scripts includes:

[0015] determining at least two second target test scripts;

[0016] The at least two second target test scripts are tested multiple times with a second target process number to obtain multiple execution times of each second target test script, where the second target process number is any number from 1 to a second maximum process number.

[0017] In a preferred example, the present application may be further configured as follows: testing the first target test script multiple times with the first target process number to obtain multiple execution times, including:

[0018] Testing the first target test script with a first target number of processes, and recording a start timestamp and an end timestamp of each test;

[0019] The execution time is determined according to the start timestamp and the end timestamp, so as to obtain multiple execution times of testing the first target test script multiple times with the first target process number.

[0020] In a preferred example, the present application may be further configured as follows: the determining of at least two second target test scripts includes:

[0021] Get dependencies between multiple test scripts;

[0022] Combining the multiple test scripts according to the dependency relationship to determine multiple combinations, each combination including at least two test scripts that have no dependency relationship;

[0023] A target combination is determined from the plurality of combinations, the target combination including at least two second target test scripts.

[0024] In a preferred example, the present application can be further configured as follows:

[0025] During the single-script multi-process test, if the first performance data of the first target test script exceeds the preset data threshold, stop increasing the number of processes;

[0026] and / or,

[0027] During the multi-script multi-process test, if the second performance data of any second target test script of the at least two second target test scripts exceeds a preset data threshold, the number of processes is stopped from being increased.

[0028] In a preferred example, the present application can be further configured as follows:

[0029] Get the test mode selected by the user;

[0030] When the test mode is a multi-process test, a single-script multi-process test and a multi-script multi-process test are performed;

[0031] When the test mode is a single-process test, each test script is tested in sequence.

[0032] In a preferred example, the present application can be further configured as follows:

[0033] Acquire performance data corresponding to each of multiple rounds of target tests, wherein the target tests include: single-script multi-process testing and / or multi-script multi-process testing;

[0034] Calculate the target standard deviation based on the performance data corresponding to each of the multiple rounds of target tests;

[0035] Evaluate the stability of the baseboard management controller based on the target standard deviation.

[0036] In a second aspect, a performance testing device is provided, comprising:

[0037] A first test module is used to gradually increase the number of processes of a first target test script for testing during a single-script multi-process test, and obtain first performance data of the first target test script, wherein the first target test script is any script among a plurality of test scripts, and the first performance data is determined based on a start timestamp and an end timestamp;

[0038] A second test module is used to gradually increase the number of processes of each of at least two second target test scripts during a multi-script multi-process test to perform a test, and obtain second performance data of the at least two second target test scripts, wherein the at least two second target test scripts are at least two scripts that are not associated with each other in the multiple test scripts;

[0039] The performance data analysis module is used to determine the performance of the baseboard management controller according to the first performance data and the second performance data.

[0040] According to a third aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the performance testing method according to any one of the first aspects when running the computer program.

[0041] In a fourth aspect, a computer-readable storage medium is provided, wherein at least one program code is stored in the computer-readable storage medium, and the program code is loaded and executed by a processor to implement the performance testing method as described in any one of the first aspects.

[0042] In a fifth aspect, a computer program product is provided, comprising a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the performance testing method as described in any one of the first aspects is implemented.

[0043] In summary, the performance testing method provided by this application includes the following beneficial technical effects:

[0044] When evaluating the performance of a baseboard management controller for multiple processes, a single-script multi-process test and a multi-script multi-process test are set; during the single-script multi-process test, the number of processes of the first target test script is gradually increased to obtain first performance data that accurately reflects the execution efficiency of the test script under different numbers of processes; during the multi-script multi-process test, the number of processes of at least two unrelated second target test scripts is gradually increased for testing at the same time. Multiple independent test scripts may run at the same time and compete with each other for system resources. Through the obtained second performance data, the performance of the baseboard management controller when processing multiple independent tasks at the same time can be more comprehensively determined. Not only the performance under a single test script is considered, but also the performance when multiple test scripts are executed concurrently, so that the performance of the baseboard management controller can be more accurately reflected.

[0045] In addition, the present application also provides a performance testing device, equipment and medium, all of which have the above-mentioned beneficial technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 It is a flow chart of a performance testing method provided in an embodiment of the present application;

[0048] Figure 2 It is a flowchart of another performance testing method provided in an embodiment of the present application;

[0049] Figure 3 is a structural schematic diagram of a performance testing device provided in an embodiment of the present application;

[0050] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] The terms "including" and "having" in the specification of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0053] In order to better understand and illustrate the solutions of the embodiments of the present application, some technical terms involved in the embodiments of the present application are briefly explained below.

[0054] BMC is used for remote monitoring, management and maintenance of servers.

[0055] Timestamp: Data used to mark the time when an event occurs.

[0056] Process Number: Process Number refers to the number of test tasks or test scripts running simultaneously in the automated test system. By adjusting the number of processes, you can simulate different load conditions and evaluate the performance of the BMC under different loads.

[0057] Concurrent execution: Concurrent execution refers to the state in which multiple test tasks are run simultaneously in the same time period. In BMC performance evaluation, concurrent execution can simulate the load conditions in the real environment, so as to more accurately evaluate the concurrent processing capability of BMC.

[0058] The system of the test method provided in the embodiment of the present application includes a test script module, a timestamp recording module, a performance data analysis module and a report generation module. Among them, the test script module is responsible for writing test scripts for various functions of the BMC, such as DHCP (Dynamic Host Configuration Protocol) and static IP switching tests, BMC upgrade and downgrade tests, SEL list sending tests, and BMC restart tests. The timestamp recording module records the current time before and after the test script is executed and generates a timestamp. The performance data analysis module calculates the execution time of the test according to the timestamp and generates performance data. The report generation module organizes the performance data into reports for user viewing and analysis.

[0059] Test script module: This module is responsible for writing performance test scripts for various BMC functions, including but not limited to DHCP and static IP address switching tests, BMC version upgrade and downgrade tests, SEL list sending tests, and BMC restart tests. These scripts are modularly designed and have good scalability, and can easily cope with new test requirements that may arise in the future. By executing these scripts, various scenarios of BMC in actual operation can be simulated to fully verify its performance.

[0060] Code implementation example: The following is an example of a test script for BMC DHCP and static IP switching functions:

[0061] #! / bin / bash

[0062] # DHCP and static IP switching test script

[0063] # Record the test start timestamp

[0064] start_time=$(date +%s%3N)

[0065] # Perform a DHCP configuration test

[0066] echo "Testing DHCP configuration..."

[0067] # Add specific DHCP configuration commands and verification steps (such as using curl or wget commands to access the BMC IP address)

[0068] dhcp_config_command="..." # Specific DHCP configuration command

[0069] eval $dhcp_config_command

[0070] # Record the DHCP test end timestamp

[0071] dhcp_end_time=$(date +%s%3N)

[0072] # Switch to static IP configuration

[0073] echo "Switching to static IP configuration..."

[0074] # Add specific static IP configuration commands (such as modifying network configuration files and restarting network services)

[0075] static_ip_config_command="..." # Specific static IP configuration command

[0076] eval $static_ip_config_command

[0077] # Record the end timestamp of static IP configuration

[0078] static_ip_end_time=$(date +%s%3N)

[0079] # Output test results (including timestamp)

[0080] echo "DHCP test completed at: $dhcp_end_time"

[0081] echo "Static IP configuration completed at: $static_ip_end_time"

[0082] # Record the total end timestamp of the test

[0083] end_time=$(date +%s%3N)

[0084] # Calculate and output the total test time (for subsequent performance analysis)

[0085] total_test_time=$((end_time - start_time))

[0086] echo "Total test time: $total_test_time nanoseconds"

[0087] # Output timestamps and test results to a specified file or log so that the timestamp recording module can capture and analyze them

[0088] echo "Start Time: $start_time, DHCP End Time: $dhcp_end_time, StaticIP End Time: $static_ip_end_time, End Time: $end_time, Total Test Time: $total_test_time" >> test_results.log

[0089] For the timestamp recording module: This module records the current time before and after the test script is executed and generates a high-precision timestamp. These timestamps are the basis for subsequent performance data analysis. The timestamp recording module supports multiple time formats and time zone settings to ensure the accuracy and consistency of time data.

[0090] Code implementation example: The timestamp recording module can be a simple Shell function that records timestamps before and after the test script is executed. This function can be integrated into the test script or directly called as an independent module.

[0091] # Timestamp recording function

[0092] record_timestamp() {

[0093] local timestamp=$(date +%s%3N)

[0094] echo $timestamp

[0095] }

[0096] # Call this function at the beginning and end of the test script

[0097] start_time=$(record_timestamp)

[0098] # Run the test...

[0099] end_time=$(record_timestamp)

[0100] For the performance data analysis module: This module calculates the execution time of the test script based on the timestamp and generates performance data. The performance data analysis module supports multiple data analysis methods, such as the average, standard deviation, maximum, and minimum execution time of each test script module, so that users can fully understand the performance of the BMC. In addition, the module also supports comparing performance data with preset thresholds and automatically determines whether the test results are qualified, thereby achieving quantitative evaluation of BMC performance.

[0101] Code implementation example: The performance data analysis module can be implemented using a Shell script or Python script to calculate the test execution time based on the recorded timestamps and generate performance data. The following is a simple Shell script example that calculates the test execution time and outputs it to a file:

[0102] #! / bin / bash

[0103] # Enter the start and end times (these timestamps can be captured from the output of the test script)

[0104] start_time=$1

[0105] end_time=$2

[0106] # Calculate test execution time

[0107] execution_time=$((end_time - start_time))

[0108] # Output performance data to file (or standard output)

[0109] echo "Test execution time: $execution_time nanoseconds" >>performance_data.txt

[0110] For the report generation module: This module organizes the performance data into reports, including the execution time of the test script, performance data charts, test result judgments, etc. The report generation module supports multiple report formats and output methods, such as PDF, Excel, HTML, etc., to meet the diverse needs of users.

[0111] Code implementation example: The report generation module can use Python scripts combined with the Jinja2 template engine to generate formatted reports. The following is a simple Python script example that reads performance data and generates an HTML report:

[0112] from jinja2 import Template

[0113] # Read performance data

[0114] with open('performance_data.txt', 'r') as file:

[0115] performance_data = file.readlines()

[0116] # Define HTML report template

[0117] template = Template('''

[0118] <!DOCTYPE html>

[0119]

[0120]

[0121] <title> BMC basic performance evaluation report< / title>

[0122]

[0123]

[0124] <h1> BMC basic performance evaluation report< / h1>

[0125]

[0126]

[0127]

[0128]

[0129]

[0130]

[0131]

[0132]

[0133]

[0134]

[0135]

[0136] Test items Execution time (nanoseconds) {% for line in performance_data %} {{ line.split(": ")[0]}} {{ line.split(": ")[1].strip()}} {% endfor %}

[0137]

[0138]

[0139] ''')

[0140] # Generate HTML report

[0141] with open('performance_report.html', 'w') as file:

[0142] file.write(template.render(performance_data=performance_data))

[0143] Specifically, the present application provides a performance testing method, such as Figure 1 As shown, the method provided in the embodiment of the present application can be executed by an electronic device, and the electronic device is a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The embodiment of the present application is not limited here. The method includes:

[0144] S101, in a single-script multi-process test process, gradually increase the number of processes of a first target test script for testing, and obtain first performance data of the first target test script;

[0145] The first target test script is any one of the plurality of test scripts, and the first performance data is determined based on a start timestamp and an end timestamp;

[0146] The single-script multi-process test refers to a test method that starts multiple processes at the same time to execute a test script, and is used to evaluate the performance of the BMC of a single script in a multi-process environment.

[0147] The number of processes is used to indicate the number of processes of the first target test script that are run simultaneously during the test. After the first target test script is tested multiple times with a certain number of processes to obtain performance data, the number of processes is increased and the test is performed again until the stop condition is reached, and all the obtained performance data is used as the first performance data. The number of times each process number is tested can be customized by the user, and the embodiments of the present application are no longer limited. Moreover, the stop condition can be the maximum number of processes set by the user, or it can be that when it is detected that any data in the performance data corresponding to the current number of processes exceeds the corresponding threshold, it indicates that the BMC performance has been significantly reduced, and it is determined that the number of processes will not be increased.

[0148] A possible implementation method of the embodiment of the present application is to gradually increase the number of processes of the first target test script for testing to obtain the first performance data of the first target test script, including: testing the first target test script multiple times with the first target number of processes to obtain multiple execution times, the first target number of processes is any number from 1 to the first maximum number of processes; determining the first performance data corresponding to the target number of processes based on the multiple execution times, the first performance data including: the average value, standard deviation, maximum value, and minimum value of the execution time; obtaining the first performance data of the first target test script based on the first performance data corresponding to each first target number of processes. Specifically, testing the first target test script multiple times with the first target number of processes to obtain multiple execution times includes: testing the first target test script with the first target number of processes, and recording the start timestamp and end timestamp of each test; determining the execution time based on the start timestamp and the end timestamp, so as to obtain multiple execution times of testing the first target test script multiple times with the first target number of processes.

[0149] The first performance data includes performance data corresponding to the number of processes, and the performance data is determined based on the start timestamp and the end timestamp, including but not limited to: average value, standard deviation, maximum value and minimum value. Exemplarily, if three processes are used to test the first target test script, three processes are executed simultaneously at the start time, and the end time 1 of process 1, the end time 2 of process 2, and the end time 3 of process 3 are finally recorded. At this time, the latest end time can be selected as the end timestamp, and then the start time and the end timestamp are estimated to determine the duration of this time; furthermore, the three processes are used to test the first target test script multiple times to obtain multiple durations, so as to determine the average value, standard deviation, maximum value and minimum value based on multiple durations.

[0150] In the embodiment of the present application, the BMC performance in a multi-process environment of a single test script is evaluated by gradually increasing the number of processes of the first target test script.

[0151] Specifically, the multi-process automated testing process: In the multi-process running environment under the OS, one or more test scripts in the test script module are executed concurrently. The number of processes can be flexibly set according to actual needs. Before all processes are executed and after all processes are completed, the timestamp recording module records the current time and generates a timestamp. The performance data analysis module calculates the completion time of all processes based on the timestamp and obtains the overall performance index of the BMC in the multi-process environment. This process simulates the multi-task concurrent scenarios that the BMC may face in actual operation, thereby verifying its performance in a complex environment.

[0152] Code implementation example:

[0153] #! / bin / bash

[0154] # Multi-process automated test script, add performance time record

[0155] # Test script directory

[0156] TEST_SCRIPTS_DIR=". / test_scripts"

[0157] # Output file

[0158] OUTPUT_FILE=". / multi_process_test_results_with_timing.txt"

[0159] # Clear the output file

[0160] > $OUTPUT_FILE

[0161] # Record start time

[0162] SUITE_START_TIME=$(date +%Y-%m-%d\ %H:%M:%S.%3N)

[0163] echo "Multi-process test suite started at: $SUITE_START_TIME" >> $OUTPUT_FILE

[0164] # Initialize an array to store the P IDs of background jobs

[0165] JOBS=()

[0166] # Traverse the test script directory and start a background job for each script

[0167] for script in $TEST_SCRIPTS_DIR / *.sh; do

[0168] if [ -x "$script" ]; then

[0169] SCRIPT_NAME=$(basename "$script")

[0170] echo "Starting $SCRIPT_NAME in the background..." >> $OUTPUT_FILE

[0171] # Record script start timestamp (in background job)

[0172] {

[0173] SCRIPT_START_TIME=$(date +%Y-%m-%d\ %H:%M:%S.%3N)

[0174] echo "$SCRIPT_NAME started at: $SCRIPT_START_TIME" >> $OUTPUT_FILE

[0175] # Execute the test script and redirect the output to a file

[0176] bash "$script" >> "$OUTPUT_FILE.tmp" 2>&1

[0177] # Record script end timestamp (in background job)

[0178] SCRIPT_END_TIME=$(date +%Y-%m-%d\ %H:%M:%S.%3N)

[0179] echo "$SCRIPT_NAME completed at: $SCRIPT_END_TIME" >> $OUTPUT_FILE

[0180] # Calculate execution time (in background job)

[0181] SCRIPT_EXECUTION_TIME=$(echo "$SCRIPT_END_TIME $SCRIPT_START_TIME" |awk '{print ($1-$2) / 86400000000000}') # Convert to seconds

[0182] echo "$SCRIPT_NAME executed in: $SCRIPT_EXECUTION_TIME seconds." >> $OUTPUT_FILE

[0183] # Append the contents of the temporary output file to the main output file (this is for demonstration purposes, usually it should be redirected directly to the correct log file)

[0184] cat "$OUTPUT_FILE.tmp" >> "$OUTPUT_FILE"

[0185] # Delete temporary output files

[0186] rm "$OUTPUT_FILE.tmp"

[0187] } &

[0188] # Add the PID of the background job to the array

[0189] JOBS+=($!)

[0190] else

[0191] echo "$script is not executable. Skipping..." >> $OUTPUT_FILE

[0192] fi

[0193] done

[0194] # Wait for all background jobs to complete

[0195] for job in "${JOBS[@]}"; do

[0196] wait $job

[0197] done

[0198] # Record end time

[0199] SUITE_END_TIME=$(date +%Y-%m-%d\ %H:%M:%S.%3N)

[0200] echo "Multi-process test suite completed at: $SUITE_END_TIME" >> $OUTPUT_FILE

[0201] # Optional: Calculate the total execution time of the entire test suite

[0202] SUITE_EXECUTION_TIME=$(echo "$SUITE_END_TIME $SUITE_START_TIME" | awk'{print ($1-$2) / 86400000000000}') # Convert to seconds

[0203] echo "Total test suite execution time: $SUITE_EXECUTION_TIMEseconds." >> $OUTPUT_FILE.

[0204] S102, during the multi-script multi-process test process, gradually increase the number of processes of each second target test script of at least two second target test scripts for testing, and obtain second performance data of at least two second target test scripts;

[0205] The at least two second target test scripts are at least two scripts that are not associated with each other among the plurality of test scripts;

[0206] Among them, the multi-script multi-process test process means running multiple test scripts at the same time, and each test script can be executed in parallel in multiple processes. The number of processes is used to indicate the number of processes occupied by each second target test script during execution. Increasing the number of processes can simulate higher concurrency scenarios. The second performance data refers to the performance indicators of the second target test script collected in the process of gradually increasing the number of processes. The absence of at least two associated scripts indicates that the selected second target test scripts are independent of each other, and there is no dependency in the execution process and they will not affect each other.

[0207] At least two second target test scripts are taken as a test group, and several test groups can be determined based on multiple test scripts; in the embodiment of the present application, multi-script multi-process testing can be performed based on each test group, and multi-script multi-process testing can also be performed on a certain test group, which is no longer limited in the embodiment of the present application.

[0208] A possible implementation of the embodiment of the present application is to gradually increase the number of processes of each of at least two second target test scripts for testing to obtain second performance data of at least two second target test scripts, including:

[0209] Determine at least two second target test scripts; test the at least two second target test scripts multiple times with a second target process number to obtain multiple execution times of each second target test script, wherein the second target process number is any number from 1 to the second maximum process number.

[0210] Specifically, taking a test group as an example, the second performance data includes performance data corresponding to the test group, including but not limited to: average value, standard deviation, maximum value and minimum value. Exemplarily, if there are three test scripts in the test group, three processes are used to test the three test scripts, each process corresponds to a test script, and the three processes are executed simultaneously at the start time, and finally the end time 1 of test script 1 corresponding to process 1, the end time 2 of test script 2 corresponding to process 2, and the end time 3 of test script 3 corresponding to process 3 are recorded; then, the three-process test is used multiple times to obtain multiple durations corresponding to each test script, so as to determine the average value, standard deviation, maximum value and minimum value based on the multiple durations. Then, increase the number of processes by multiples, use 6 processes to test three test scripts, and each two processes correspond to one test script; for one of the test scripts, finally record the end time 1 of process 1 and the end time 2 of process 2; select the latest time as the end timestamp and record the duration of this time; continue testing until the stop condition is reached, which can be the maximum number of processes set by the user, or when it is detected that any data in the performance data of a script corresponding to the current number of processes exceeds the corresponding threshold, it means that the BMC performance is significantly reduced, and it is determined that the number of processes will not be increased.

[0211] Specifically, in actual applications, the system often needs to process requests from different modules or functions at the same time, and these requests are usually not directly related. Therefore, in the multi-script multi-process test process, in order to comprehensively evaluate the performance and stability of the system, at least two unrelated second target test scripts are selected for testing; by gradually increasing the number of processes of these scripts, the performance of the system under gradually increasing loads can be simulated, thereby obtaining more accurate second performance data.

[0212] S103: Determine the performance of the baseboard management controller according to the first performance data and the second performance data.

[0213] In a possible scenario, the average value, standard deviation, maximum value, and minimum value in the first performance data and the second performance data are used as the performance representation of the BMC, so that the user can fully understand the performance.

[0214] In another possible case, for each average value, standard deviation, maximum value, and minimum value in the first performance data and the second performance data, corresponding preset thresholds are stored in the database for comparison. Only when the threshold requirements are met, the test is determined to be qualified. If any of them is not met, the test result is determined to be failed, indicating that the current BMC performance is unqualified, thereby achieving quantitative evaluation of BMC performance. Each preset threshold is determined by technicians based on actual experience and is not limited in this embodiment.

[0215] Furthermore, for both single-process automated testing processes and multi-process automated testing processes, a loop mechanism can be added to perform multiple tests, and multiple sets of data results can be processed to obtain more accurate data.

[0216] It can be seen that in the embodiment of the present application, when evaluating the performance of the baseboard management controller for multiple processes, a single-script multi-process test and a multi-script multi-process test are set; in the single-script multi-process test process, the number of processes of the first target test script is gradually increased to obtain the first performance data that accurately reflects the execution efficiency of the test script under different numbers of processes; in the multi-script multi-process test process, the number of processes of at least two unrelated second target test scripts is gradually increased at the same time for testing. Multiple independent test scripts may run at the same time and compete with each other for system resources. Through the obtained second performance data, the performance of the baseboard management controller when processing multiple independent tasks at the same time can be more comprehensively determined. Not only the performance under a single test script is considered, but also the performance when multiple test scripts are executed concurrently, so that the performance of the baseboard management controller can be more accurately reflected.

[0217] A possible implementation of the embodiment of the present application is to determine at least two second target test scripts, including:

[0218] Get dependencies between multiple test scripts;

[0219] Combining multiple test scripts according to the dependency relationship to determine multiple combinations, each combination including at least two test scripts without dependency relationship;

[0220] A target combination is determined from the plurality of combinations, the target combination including at least two second target test scripts.

[0221] The dependency relationship means that the execution of one test script depends on the successful execution of another test script. For example, if test script A tests a basic function, and test script B tests an extended function based on the basic function, then test script B depends on test script A.

[0222] Specifically, each test script is regarded as a node in the graph, and the dependency relationship is regarded as a directed edge between nodes; the relationship graph is traversed, and all test script combinations that do not have dependencies and can be executed in parallel are found. Then, a target combination is determined from them, and the target combination can be the combination with the most test scripts, so that only this combination can be tested; or it can be any combination, so that the combinations can be tested in sequence.

[0223] It can be seen that in the embodiment of the present application, the test scripts are grouped according to the dependencies between each other, so that the test scripts in each combination have no dependencies, which can improve the efficiency of determining the combination.

[0224] A possible implementation of the embodiment of the present application also includes:

[0225] During the single-script multi-process test, if the first performance data of the first target test script exceeds the preset data threshold, the number of processes is stopped from being increased.

[0226] In the embodiment of the present application, for each data in the first performance data corresponding to each number of processes, the corresponding preset data threshold can be the same, or can increase as the number of processes increases. In the process of gradually increasing the number of processes, when the test data is obtained, it is compared with the preset data threshold. As long as there is any situation that exceeds the threshold, it means that the system may have reached its performance limit, and the number of processes stops increasing.

[0227] Exemplarily, the first target test script is measured multiple times according to one process to obtain the average value, standard deviation, maximum value and minimum value, all of which do not exceed the corresponding threshold value; the number of processes is increased, and the first target test script is measured multiple times according to two processes (each process executes the first target test script) to obtain the average value, standard deviation, maximum value and minimum value, all of which do not exceed the corresponding threshold value; the number of processes is increased, and the first target test script is measured multiple times according to three processes to obtain the average value, standard deviation, maximum value and minimum value, and if one of them exceeds the corresponding threshold value, the number of processes is stopped from being increased; or, the first target test script is measured according to three processes, and if the execution time obtained by measuring once is much longer than the preset standard execution time, the number of processes is stopped from being increased.

[0228] It can be seen that in the embodiment of the present application, during the single-script multi-process testing process, by monitoring performance data and making decisions based on preset thresholds, the testing system can automatically stop increasing the number of processes when necessary, thereby improving the efficiency of the test.

[0229] A possible implementation of the embodiment of the present application also includes:

[0230] During the multi-script multi-process test, if the second performance data of any second target test script of at least two second target test scripts exceeds a preset data threshold, the number of processes is stopped from being increased.

[0231] Similarly, referring to the single script multi-process test, the embodiments of the present application will not be described in detail.

[0232] It can be seen that in the embodiment of the present application, during the multi-script multi-process testing process, by monitoring performance data and making decisions based on preset thresholds, the testing system can automatically stop increasing the number of processes when necessary, thereby improving the efficiency of the test.

[0233] A possible implementation of the embodiment of the present application may be to set a single process mode and a multi-process mode in order to comprehensively determine the performance of the BMC, and specifically, also includes:

[0234] Get the test mode selected by the user;

[0235] When the test mode is multi-process test, single script multi-process test and multi-script multi-process test are executed;

[0236] When the test mode is single-process testing, each test script is tested in turn.

[0237] For the single-process automated test process: In the single-process running environment under the OS, execute each test script in the test script module in turn. Before and after each test script is executed, the timestamp recording module records the current time and generates a timestamp. The performance data analysis module calculates the execution time of each test script based on these timestamps and obtains the basic performance indicators of the BMC in the single-process environment. This process ensures that the performance of the BMC in the single-process environment is comprehensively and accurately evaluated.

[0238] Code implementation example:

[0239] #! / bin / bash

[0240] # Single process automated test script

[0241] # Test script directory

[0242] TEST_SCRIPTS_DIR=". / test_scripts"

[0243] # Output file

[0244] OUTPUT_FILE=". / single_process_test_results.txt"

[0245] # Clear the output file

[0246] > $OUTPUT_FILE

[0247] # Record start time

[0248] echo "Single process test suite started at: $(date +%Y-%m-%d\ %H:%M:%S.%3N)" >> $OUTPUT_FILE

[0249] # Traverse the test script directory and execute each script

[0250] for script in $TEST_SCRIPTS_DIR / *.sh; do

[0251] if [ -x "$script" ]; then

[0252] echo "Executing $script..." >> $OUTPUT_FILE

[0253] # Record the script start time

[0254] START_TIME=$(date +%s%3N)

[0255] # Execute the test script

[0256] bash $script >> $OUTPUT_FILE 2>&1

[0257] # Record the end time of the script

[0258] END_TIME=$(date +%s%3N)

[0259] # Calculate execution time and add to output file

[0260] EXECUTION_TIME=$((END_TIME - START_TIME))

[0261] echo "$script completed in $EXECUTION_TIME nanoseconds." >> $OUTPUT_FILE

[0262] else

[0263] echo "$script is not executable. Skipping..." >> $OUTPUT_FILE

[0264] fi

[0265] done

[0266] # Record end time

[0267] echo "Single process test suite completed at: $(date +%Y-%m-%d\ %H:%M:%S.%3N)" >> $OUTPUT_FILE

[0268] It can be seen that in the embodiments of the present application, multiple modes can be set to comprehensively evaluate the BMC performance.

[0269] A possible implementation of the embodiment of the present application also includes:

[0270] Obtaining performance data corresponding to each of multiple rounds of target tests, where the target tests include: single-script multi-process testing and / or multi-script multi-process testing;

[0271] Calculate the target standard deviation based on the performance data corresponding to each of the multiple rounds of target tests;

[0272] Evaluate BMC stability based on target standard deviation.

[0273] Taking the single script multi-process test as an example, multiple rounds of tests can be performed on the test process of each number of processes tested by each first target test script to obtain the performance data corresponding to each number of processes in multiple rounds of tests, and then calculate the standard deviation of each performance data. For example, the first target test script is tested with 3 processes to obtain the average value, standard deviation, maximum value, and minimum value of the execution time corresponding to the first round; then the second round of tests is performed to obtain the average value, standard deviation, maximum value, and minimum value of the execution time corresponding to the second round until the preset number of rounds is reached. The standard deviation corresponding to each dimension is calculated with each dimension (average value, standard deviation, maximum value, and minimum value); then the average value of the standard deviation can be used as the standard deviation corresponding to the 3 processes. If the standard deviation is less than the preset standard deviation threshold, it means that the BMC has high stability, otherwise, it is determined that the BMC has low stability. If the target standard deviation is small, it means that the performance data of multiple rounds of tests are relatively stable, and the performance of the BMC under long-term operation or complex loads is also relatively stable. On the contrary, if the target standard deviation is large, it means that the performance data of multiple rounds of tests have large fluctuations, and there may be problems with the stability of the BMC.

[0274] It can be seen that in the embodiment of the present application, the standard deviation is calculated through multiple rounds of tests to evaluate the stability of the BMC, so that the performance of the BMC under long-term operation or complex loads can be comprehensively evaluated, assisting technicians to discover stability problems in a timely manner.

[0275] Based on any of the above embodiments, the present invention proposes a timestamp-based BMC basic performance comprehensive evaluation system and automated testing method, which aims to comprehensively and accurately evaluate the basic performance indicators of BMC in single-process and multi-process environments by recording and analyzing the time data during the test. Figure 2 ,The system includes a test script module, a timestamp recording module, a ,performance data analysis module and a report generation module, which can automatically ,execute the test script, record the time data during the ,testing process, and generate a detailed performance evaluation report.

[0276] The automated testing process reduces the workload of manual testing and improves testing efficiency. At the same time, the modularly designed test scripts are easy to expand and maintain, reducing testing costs. By recording and analyzing the time data during the test, the basic performance indicators of the BMC in single-process and multi-process environments can be comprehensively and accurately evaluated. In addition, the performance data analysis module supports a variety of data analysis methods, which helps users gain a deep understanding of the performance of the BMC. The system architecture is simple and clear, easy to implement and expand; users can add new test scripts and performance analysis indicators as needed to meet the needs of different test scenarios.

[0277] Furthermore, the performance data can be organized into intuitive reports for easy viewing and analysis by users. At the same time, multiple report formats and output methods meet the needs of different users.

[0278] Furthermore, the method of the embodiment of the present application can be combined with a real-time performance monitoring system to construct a performance monitoring and early warning system. When the BMC performance is lower than the preset threshold, the system automatically triggers the early warning mechanism and promptly notifies the operation and maintenance personnel to handle it. This helps to promptly discover and solve performance problems and improve the stability and reliability of the system.

[0279] Furthermore, targeted optimization suggestions can be put forward based on the test results and performance bottleneck analysis, which will help improve the design and implementation of BMC and enhance the overall performance of the server or embedded system. At the same time, the optimization suggestions can also provide reference and reference for the performance testing and optimization of other similar systems.

[0280] The following is an introduction to a performance testing device provided in an embodiment of the present application. The device described below and the method described above can be referred to each other. The device of this embodiment is set in an electronic device. Refer to, 3, Figure 3 It is a structural block diagram of a device of one embodiment of the present application, comprising:

[0281] A first test module 210 is used to gradually increase the number of processes of a first target test script for testing during a single-script multi-process test, and obtain first performance data of the first target test script, where the first target test script is any script among the multiple test scripts, and the first performance data is determined based on a start timestamp and an end timestamp;

[0282] The second test module 220 is used to gradually increase the number of processes of each of at least two second target test scripts during the multi-script multi-process test process to perform the test, and obtain second performance data of at least two second target test scripts, where the at least two second target test scripts are at least two scripts that are not associated with each other in the multiple test scripts;

[0283] The performance data analysis module 230 is used to determine the performance of the baseboard management controller according to the first performance data and the second performance data.

[0284] In an achievable manner, the first test module 210 is used to gradually increase the number of processes of the first target test script for testing to obtain first performance data of the first target test script, including:

[0285] Testing the first target test script multiple times with a first target number of processes to obtain multiple execution times, wherein the first target number of processes is any number from 1 to a first maximum number of processes;

[0286] Determine first performance data corresponding to the target process quantity according to the plurality of execution times, the first performance data comprising: an average value, a standard deviation, a maximum value, and a minimum value of the execution time;

[0287] Obtaining first performance data of a first target test script according to first performance data corresponding to each first target process quantity;

[0288] Accordingly, the second testing module 220 is used to:

[0289] determining at least two second target test scripts;

[0290] At least two second target test scripts are tested multiple times with a second target process number to obtain multiple execution times of each second target test script, where the second target process number is any number from 1 to a second maximum process number.

[0291] In one achievable manner, the first testing module 210 is used to:

[0292] Testing the first target test script with the first target process quantity, and recording the start timestamp and the end timestamp of each test;

[0293] The execution time is determined according to the start timestamp and the end timestamp, so as to obtain multiple execution times of testing the first target test script multiple times with the first target process number.

[0294] In an achievable manner, the first test module 210 is further configured to stop increasing the number of processes if the first performance data of the first target test script exceeds a preset data threshold during the single-script multi-process test;

[0295] The second test module 220 is also used to stop increasing the number of processes if the second performance data of any second target test script of at least two second target test scripts exceeds a preset data threshold during the multi-script multi-process test.

[0296] In one achievable manner, the present invention further includes:

[0297] A mode selection module is used to: obtain a test mode selected by a user;

[0298] When the test mode is multi-process test, single script multi-process test and multi-script multi-process test are executed;

[0299] When the test mode is single-process testing, each test script is tested in turn.

[0300] In an achievable manner, the performance data analysis module 230 is further used to stably obtain the performance data corresponding to each of multiple rounds of target tests, the target tests including: single-script multi-process tests, and / or multi-script multi-process tests;

[0301] Calculate the target standard deviation based on the performance data corresponding to each of the multiple rounds of target tests;

[0302] Evaluate the stability of the baseboard management controller based on the target standard deviation.

[0303] In an achievable manner, the second test module 220 is further used to: obtain dependency relationships among multiple test scripts;

[0304] Combining multiple test scripts according to the dependency relationship to determine multiple combinations, each combination including at least two test scripts without dependency relationship;

[0305] A target combination is determined from the plurality of combinations, the target combination including at least two second target test scripts.

[0306] Figure 4 A structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 4 As shown, the electronic device includes: a memory 60 for storing a computer program;

[0307] The processor 61 is used to implement the steps of the performance testing method in the above embodiment when executing a computer program.

[0308] The electronic device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0309] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 61 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0310] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 60 is at least used to store the following computer program 601, wherein, after the computer program is loaded and executed by the processor 61, it can implement the relevant steps of the performance testing method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 602 may include Windows, Unix, Linux, etc.

[0311] In some embodiments, the electronic device may further include a display screen 62 , an input / output interface 63 , a communication interface 64 , a power supply 65 , and a communication bus 66 .

[0312] Those skilled in the art will understand that Figure 4 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure.

[0313] It is understandable that if the performance testing method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the current technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk or optical disk and other media that can store program codes.

[0314] Based on this, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0315] An embodiment of the present application provides a computer program product, including a computer program or instructions, which implement the corresponding contents of the aforementioned method embodiment when the computer program or instructions are executed by a processor.

[0316] The above is a detailed introduction to a performance testing method, device, equipment and medium provided in an embodiment of the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0317] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0318] The above is a detailed introduction to a performance testing method, device, equipment and medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A performance testing method, characterized in that: include: In a single-script multi-process test process, gradually increase the number of processes of a first target test script for testing, and obtain first performance data of the first target test script, wherein the first target test script is any script among a plurality of test scripts, and the first performance data is determined based on a start timestamp and an end timestamp; In a multi-script multi-process test process, gradually increase the number of processes of each second target test script of at least two second target test scripts for testing, and obtain second performance data of the at least two second target test scripts, wherein the at least two second target test scripts are at least two scripts that are not associated with each other in the multiple test scripts; The performance of the baseboard management controller is determined according to the first performance data and the second performance data.

2. The method according to claim 1, characterized in that Gradually increase the number of processes of the first target test script to perform a test, and obtain first performance data of the first target test script, including: Testing the first target test script multiple times with a first target number of processes to obtain multiple execution times, wherein the first target number of processes is any number from 1 to a first maximum number of processes; Determine first performance data corresponding to the target process quantity according to the plurality of execution times, the first performance data comprising: an average value, a standard deviation, a maximum value, and a minimum value of the execution time; Obtaining first performance data of a first target test script according to first performance data corresponding to each first target process quantity; Accordingly, gradually increasing the number of processes of each of the at least two second target test scripts for testing to obtain second performance data of the at least two second target test scripts includes: determining at least two second target test scripts; The at least two second target test scripts are tested multiple times with a second target process number to obtain multiple execution times of each second target test script, where the second target process number is any number from 1 to a second maximum process number.

3. The method according to claim 2, characterized in that The first target test script is tested multiple times with a first target number of processes to obtain multiple execution times, including: Testing the first target test script with a first target number of processes, and recording a start timestamp and an end timestamp of each test; The execution time is determined according to the start timestamp and the end timestamp, so as to obtain multiple execution times of testing the first target test script multiple times with the first target process number.

4. The method according to claim 2, characterized in that: The determining of at least two second target test scripts comprises: Get dependencies between multiple test scripts; Combining the multiple test scripts according to the dependency relationship to determine multiple combinations, each combination including at least two test scripts that have no dependency relationship; A target combination is determined from the plurality of combinations, the target combination including at least two second target test scripts.

5. The method according to claim 1, characterized in that: Also includes: During the single-script multi-process test, if the first performance data of the first target test script exceeds the preset data threshold, stop increasing the number of processes; and / or, During the multi-script multi-process test, if the second performance data of any second target test script of the at least two second target test scripts exceeds a preset data threshold, the number of processes is stopped from being increased.

6. The method according to claim 1, characterized in that Also includes: Get the test mode selected by the user; When the test mode is a multi-process test, a single-script multi-process test and a multi-script multi-process test are performed; When the test mode is a single-process test, each test script is tested in sequence.

7. The method according to claim 1, characterized in that Also includes: Acquire performance data corresponding to each of multiple rounds of target tests, wherein the target tests include: single-script multi-process testing and / or multi-script multi-process testing; Calculate the target standard deviation based on the performance data corresponding to each of the multiple rounds of target tests; Evaluate the stability of the baseboard management controller based on the target standard deviation.

8. A testing device, characterized in that: include: A first test module is used to gradually increase the number of processes of a first target test script for testing during a single-script multi-process test, and obtain first performance data of the first target test script, wherein the first target test script is any script among a plurality of test scripts, and the first performance data is determined based on a start timestamp and an end timestamp; A second test module is used to gradually increase the number of processes of each of at least two second target test scripts during a multi-script multi-process test to perform a test, and obtain second performance data of the at least two second target test scripts, wherein the at least two second target test scripts are at least two scripts that are not associated with each other in the multiple test scripts; The performance data analysis module is used to determine the performance of the baseboard management controller according to the first performance data and the second performance data.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the method according to any one of claims 1 to 7 when running the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program code, and the program code is loaded and executed by a processor to implement the method according to any one of claims 1 to 7.

Images