A test method for LTP pressure test monitoring and early warning

Abstract

The application provides a kind of LTP pressure test monitoring early warning test method, belong to operating system test verification field, the application can guarantee that pressure test reaches standard value and is full load stable operation in time monitoring by monitoring early warning mechanism and pressurization, reaches final test purpose;Through the monitoring early warning mechanism, the running situation of pressure test is monitored in time, the scene of excessively high CPU utilization is monitored in time and early warning information is thrown, the time of abnormal scene occurrence and CPU utilization abnormal value are recorded, so as to notify tester in time for timely analysis, positioning problem in the abnormal CPU utilization of later period.

Description

Technical Field

[0001] This invention relates to the field of operating system testing and verification, and in particular to a testing method for monitoring and early warning of LTP stress testing. Background Technology

[0002] In the development of domestically produced server operating systems, stability testing is a crucial performance indicator. Currently, domestic Linux systems are constantly being updated and iterated, and the Linux kernel is also continuously being updated. Although the overall stability of Linux systems is relatively high, with new versions being optimized and upgraded from older systems, in recent years, large websites with high business volume and high traffic have frequently experienced system crashes and lags. Therefore, system stability testing still needs further strengthening. LTP, developed by the Linux Test Project, is an open-source system testing software used to perform stability testing on Linux systems. ltpstress.sh is part of the LTP test suite; it is a parallel stress test script developed based on system resource utilization statistics. It runs a wide range of kernel components in parallel while using network and memory management, generating high stress loads on the test system to ultimately verify the robustness of the product under high system resource utilization.

[0003] In practice, the stress testing environment is often unstable during LTP stress testing: an excessively high CPU idle percentage means that the stress value is too low during the test and cannot achieve the expected full-load test. This problem often occurs in multi-processor servers.

[0004] High CPU utilization means that the stress level is too high during the test, which may lead to abnormal scenarios such as server lag and crashes. Since each LTP stress test cycle is 72 hours, high CPU utilization will seriously affect the success rate of the stress test.

[0005] The LTP stress test duration can be controlled via parameters, with the industry standard typically set at 72 hours. However, due to this large duration, it's impossible for testers to monitor the system continuously throughout the entire test. Furthermore, the LTP stress environment is unstable for different test samples, leading to issues such as excessive stress causing lag, crashes, and failure to monitor in real-time, or insufficient utilization of hardware resources due to insufficient stress, which fails to meet our testing requirements. Additionally, anomalies during testing are difficult to detect, and the results are not presented intuitively. Therefore, a monitoring and early warning testing method is urgently needed to automatically monitor the stress test process, provide real-time alerts and interventions for abnormal scenarios, ensure stable stress test operation, and provide intuitive test results. Summary of the Invention

[0006] To address the above technical problems, this invention provides a testing method for monitoring and early warning of LTP stress testing.

[0007] The technical solution of this invention is:

[0008] A test method for monitoring and early warning of LTP pressure test, through timely monitoring and pressurization by the monitoring and early warning mechanism, can ensure that the pressure test reaches the standard value and runs stably at full load, thus achieving the final test objective;

[0009] By using a monitoring and early warning mechanism, the stress test operation can be monitored in a timely manner. Scenarios with excessively high CPU utilization can be detected and early warning information can be issued in a timely manner. The time of occurrence of abnormal scenarios and abnormal CPU utilization values ​​can be recorded, so that testers can be notified in a timely manner to analyze and locate the problem when abnormal CPU utilization occurs later.

[0010] Furthermore,

[0011] In Linux, the `isProcessExisting` method can be used to determine if an LTP process exists. First, `getAllProcess("ps")` retrieves the names of all processes in the system and writes the process names to a string. A loop is then used to check if an LTP process exists among all processes. A return value of TRUE or FALSE indicates that the LTP process is running normally during the test and has not been killed, displaying the message "LTP test in progress!", which triggers the LTP monitoring mechanism. If the LTP process is not found, it means that the process was automatically killed by the LTP test mechanism, at which point the LTP stress test ends. Information related to the test's completion is output for testers' reference, specifically the exact time the LTP test ended and the completion message.

[0012] The LTP test process starts normally and enters the monitoring mechanism. During the test, it obtains the real-time CPU utilization based on the LTP stress process ID. After the stress test stabilizes, it monitors the maximum CPU utilization at set intervals and outputs the data to the log file in real time. At this point, it makes a judgment based on the actual CPU utilization. If the CPU utilization is lower than 90%, it proves that the LTP stress value is insufficient, and the script stress-increasing mechanism is automatically triggered. If the CPU utilization is higher than 90%, it will continue to maintain the current stress test environment until the LTP test process is killed and the test exits.

[0013] The test log file records the real-time CPU utilization at different time points, showing the changes in CPU utilization throughout the test. To obtain the CPU utilization of a specific process's I / O, you first need to use `ps` to find process information, then filter out lines with %CPU using `grep -v`, and finally use `awk` to find the integer part of the CPU utilization percentage.

[0014] Furthermore,

[0015] When the actual CPU utilization is below 90%, an automatic pressure-boosting mechanism is triggered by the script to further increase CPU utilization: Nohup / opt / ltp / testcases / bin / genload --vm 8 --vm-bytes 443547648& The CPU utilization is gradually increased to above 90% through loop checks.

[0016] There are also scenarios where CPU utilization reaches 100%. In this case, we can see from the LTP logs that when the code runs to quicklist_trim, it calls preempt_disable, which makes the current task's preempt count greater than 0, thus preventing other tasks from preempting the current task. Immediately afterwards, it calls rt_spin_lock_slowpath, entering a busy-wait state. There is no task preemption or switching in between. So when the code calls rt_spin_lock_slowpath to request resources, if the resource is currently occupied by another task, the current task will relinquish the CPU, actively call schedule to relinquish the CPU, and enter a sleep state to wait for the resource to be released. This generates a warning. While recording the warning information, the config_quicklist function is turned off. Then, if the resource requested by the code calling rt_spin_lock_slowpath is occupied, the task will actively yield the CPU, and the scheduler will schedule other tasks normally, instead of sleeping indefinitely waiting for the originally requested resource to be released, thus avoiding the LTP stress crash problem.

[0017] After each LTP stress test is completed, the output information at each time point can be recorded in the log file, which can intuitively reflect the changes in test parameters throughout the entire test process.

[0018] The beneficial effects of this invention are

[0019] This invention addresses the issue of unstable stress testing environment during LTP stress testing by identifying problems early, assessing the actual situation of the stress testing environment, and implementing corresponding intervention measures to maintain stable operation of the stress testing environment. This prevents test failures caused by testing tools and allows for real-time recording of test data through logs, preventing crashes caused by excessive stress that make it impossible to pinpoint the cause of the problem. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the workflow of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] This invention provides a test method for monitoring and early warning of LTP stress testing, such as... Figure 1 As shown, the present invention includes:

[0023] 1. Install and run LTP variant

[0024] LTP is a dynamic engineering project, and LTP source packages are generally named in the format ltp-yyyymmdd. The currently commonly used version is ltp-20180515, and its official website is http: / / ltp.sourceforge.net / . Download the corresponding version of the LTP test suite package from the official website, extract it, and navigate to the LTP source directory. After successful compilation, it will automatically install to the / opt / ltp directory.

[0025] Compile and install: . / build.sh

[0026] . / configure --build="platform architecture"

[0027] Phytium platform: ARM, Loongson platform: MIPS, Shenwei platform: Alpha

[0028] After `make && make install` successfully compiles LTP, it consists of three main parts: test cases, test scripts, and test drivers. The test cases directory is `testcases / `, which contains test cases and their scripts. The main test script directory is `testscripts / `, which contains test scripts for the file system, disk, kernel, memory, etc. The test driver directory is `bin / `, which stores various LTP driver scripts, such as `ltp-pan` and `ltp-scanner`. To execute an LTP test: `. / testscripts / ltpstress.sh –t 72` (executes a 72-hour stress test).

[0029] 2. The LTP process was killed.

[0030] In Linux, the `isProcessExisting` method can be used to determine if an LTP process exists. First, `getAllProcess("ps")` retrieves the names of all processes in the system and writes the process names to a string. A loop is then used to check if an LTP process exists among all processes. A return value of TRUE or FALSE indicates that the LTP process is running normally during the test and has not been killed, displaying the message "LTP test in progress!", which triggers the LTP monitoring mechanism. If the LTP process is not found, it means that the process was automatically killed by the LTP test mechanism, at which point the LTP stress test ends. Information related to the test's completion is output for testers' reference, specifically the exact time the LTP test ended and the completion message.

[0031] 3. CPU utilization is greater than 90%.

[0032] The LTP test process starts normally and enters the monitoring mechanism. During the test, it obtains real-time CPU utilization based on the LTP stress process ID. After the stress test stabilizes, it monitors the maximum CPU utilization at regular intervals and records it to the log file in real time. At this point, a judgment is made based on the actual CPU utilization. If the CPU utilization is below 90%, it indicates that the LTP stress value is insufficient, and the script stress-increasing mechanism is automatically triggered. If the CPU utilization is above 90%, the current stress test environment will continue to run until the LTP test process is killed and the test exits. The test log file records the real-time CPU utilization at different time points, objectively displaying the changes in CPU utilization throughout the test. It clearly shows the time of occurrence of abnormal scenarios such as excessively low or high CPU utilization, allowing for targeted problem localization.

[0033] The following function can obtain the CPU utilization of a specified process's I / O. First, it uses `ps` to find process information, then filters out lines with `%CPU` using `grep -v`, and finally uses `awk` to find the integer part of the CPU utilization percentage. `function GetCpu{`

[0034] CpuValue=`ps–p$1–o pcpu|grep–v CPU|awk`{print$1}`|awk–F.`{print$1}`

[0035] Echo$CpuValue

[0036] }

[0037] 4. Script automatically applies pressure

[0038] When the actual CPU utilization is too low, an automatic stress-boosting mechanism is triggered by the script to further increase CPU utilization: Nohup / opt / ltp / testcases / bin / genload --vm 8 --vm-bytes 443547648&. Through loop checks, the CPU utilization is gradually increased to over 90%, meeting the stress environment standard of LTP testing and avoiding failure to achieve the final test objective due to insufficient test pressure.

[0039] Meanwhile, there are also scenarios where CPU utilization reaches 100%. In this case, we can see from the LTP logs that when the code runs to quicklist_trim, it calls preempt_disable, which makes the current task's preempt count greater than 0, meaning it prevents other tasks from preempting the current task. Immediately afterwards, it calls rt_spin_lock_slowpath, entering a busy-wait state. There is no task preemption or switching in between. So when the code calls rt_spin_lock_slowpath to request resources, if the resource is currently occupied by another task, the current task will relinquish the CPU, actively call schedule to relinquish the CPU, and enter a sleep state to wait for the resource to be released. This generates a warning, the most obvious manifestation of which is an LTP stress crash, which is unreasonable. To address the aforementioned issues, while recording warning messages, we can disable the `config_quicklist` function. Then, if the resources requested by the code via `rt_spin_lock_slowpath` are occupied, the task will proactively relinquish the CPU, and the scheduler will schedule other tasks normally. This prevents the task from continuously sleeping and waiting for the originally requested resources to be released, thus avoiding the `ltpstress` crash problem and enabling early detection and avoidance of crashes caused by code logic issues.

[0040]

[0041] 5. LTP test completed

[0042] After each round of LTP stress test, the output information at each time point can be recorded in the log file, which can intuitively reflect the changes in test parameters throughout the test process. Test problems can also be addressed in a targeted manner. In addition, this testing method eliminates some common LTP test problems, which can maximize the protection of the test environment and improve test efficiency.

[0043] The above description is merely a preferred embodiment of the present invention and is used only to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A test method for monitoring and early warning of LTP stress testing, characterized in that, The monitoring and early warning mechanism ensures timely monitoring and stress increase: The LTP test process starts normally and enters the monitoring mechanism. During the test, it obtains real-time CPU utilization based on the LTP stress process ID. After the stress test stabilizes, it monitors the maximum CPU utilization at set intervals and outputs the data to a log file in real time. At this point, a judgment is made based on the actual CPU utilization. If the CPU utilization is below 90%, it indicates that the LTP stress value is insufficient. In this case, the script stress increase mechanism is automatically intervened: `nohup / opt / ltp / testcases / bin / genload --vm 8 --vm-bytes 443547648 &`. This process iterates through the process, gradually increasing the CPU utilization to above 90%. If the CPU utilization is above 90%, the current stress test environment will continue to run until the LTP test process is killed and the test exits. By monitoring and early warning mechanisms, the stress test operation is monitored in a timely manner. Scenarios with excessively high CPU utilization are detected and early warning information is issued in a timely manner. The time of occurrence of abnormal scenarios and abnormal CPU utilization values ​​are recorded, which facilitates timely notification of testers to analyze and locate problems when abnormal CPU utilization occurs later. When CPU utilization reaches 100%, the LTP logs show that when the code reaches `quicklist_trim`, it calls `preempt_disable`, ensuring the current task's `preempt_count` is greater than 0, thus preventing other tasks from preempting the current task. Immediately afterwards, it calls `rt_spin_lock_slowpath`, entering a busy-wait state. Since there is no task preemption or switching in between, when the code calls `rt_spin_lock_slowpath` to request resources, if those resources are currently occupied by another task, the current task relinquishes the CPU, actively calls `schedule` to relinquish the CPU, and enters a sleep state to wait for the resource to be released. This generates a warning. While recording the warning, `config_quicklist` is disabled. Then, if the resource requested by the code using `rt_spin_lock_slowpath` is occupied, the task will voluntarily yield the CPU, and the scheduler will schedule other tasks normally.

2. The method according to claim 1, characterized in that, In Linux, the `isProcessExisting` method is used to determine if an LTP process exists. First, `getAllProcess("ps")` is used to obtain the names of all processes in the system and write the process names into a string. Then, a loop is used to check if an LTP process exists among all processes. If the result is TRUE or FALSE, it indicates that the LTP process is running normally and has not been killed during the test. A message "LTP test in progress!" is displayed, and the LTP monitoring mechanism is then activated. If the LTP process is not found, it means that the process was automatically killed by the LTP test mechanism. At this time, the LTP stress test ends. The test end information is output for the testers to refer to, and the specific time of LTP test end and test end prompt message are located.

3. The method according to claim 2, characterized in that, The test log file records the real-time CPU utilization at different time points, showing the changes in CPU utilization throughout the test.

4. The method according to claim 2, characterized in that, To obtain the CPU utilization of a specific process's I / O, you first need to use ps to find process information, then use grep -v to filter out the %CPU lines, and finally use awk to find the integer part of the CPU utilization percentage.

5. The method according to claim 4, characterized in that, After each LTP stress test is completed, the output information at each time point is recorded in the log file, which intuitively reflects the changes in test parameters throughout the entire test process.

Images