Low-intrusive environmental anomaly test method and system

Abstract

The invention relates to a low-intrusive environment anomaly testing method and system, and belongs to the technical field of software testing. A current environment anomaly test method has the problems of uncontrollable resource occupation, inflexible tool deployment and the like, and the stability and the test efficiency of a system are influenced. The invention aims to solve the problem by constructing a test architecture in which a server side is separated from a test executor, the server side is responsible for generating and sending a test case, the test executor executes a corresponding test on a tested machine, and a control group technology is adopted to accurately manage resources, so that the test execution hardly affects the original system performance, and the test efficiency is improved. Besides, the method supports the combined configuration of fixed and random test parameters, so as to generate test cases with wider coverage, thereby greatly improving the comprehensiveness and depth of the test. Compared with the prior art, the method has obvious flexibility and expandability, the automation level of the testing process can be effectively improved, the testing deployment is simplified, and the early-stage preparation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a low-intrusive environmental anomaly testing method and system. Background Art

[0002] Software environment abnormality testing is a key technology to ensure system stability and reliability. Software environment abnormality testing mainly includes two categories: network environment failure and operating environment failure. Network environment failure involves network connection interruption, high network latency, limited network bandwidth, etc.; operating environment failure includes insufficient memory, exhausted hard disk space, high CPU usage, etc. These failures not only affect the normal operation of the software, but may also cause system crashes, causing serious losses to the business.

[0003] Existing abnormal testing technologies usually adopt an invasive testing method, which requires the deployment of multiple testing tools and their dependencies on the running machine. Although this method can achieve testing, it has obvious disadvantages. First, the deployment process is cumbersome and inflexible. Each tool has a specific installation configuration and complex dependencies. The testing team needs to spend a lot of time and energy to prepare, which increases the complexity of the test and reduces the deployment efficiency. In addition, in the face of rapidly changing testing requirements, existing methods are difficult to adjust and optimize quickly, further affecting the efficiency and accuracy of the test. Secondly, it is difficult to accurately control the occupation of system resources by abnormal testing tools. When executing tests, these tools usually consume a lot of memory, CPU and storage resources, which may cause the system to respond slowly, cause out of memory errors, and even affect the performance of other businesses.

[0004] Therefore, existing testing methods need to be improved in terms of resource occupancy control and deployment flexibility. Summary of the invention

[0005] The purpose of the present invention is to solve the problems of uncontrollable resource usage and inflexible test tool deployment caused by invasive abnormal testing in the prior art, and to provide a low-invasive software environment abnormal testing method and system.

[0006] According to the design scheme provided by the present invention, on the one hand, a low-intrusive environmental anomaly testing method is provided.

[0007] include:

[0008] Construct a test architecture with a server and a test execution body separated, wherein the server is used to generate and send test cases and coordinate the test process; the test execution body is used to receive the test request sent by the server and perform fixed tests or random tests according to the test request;

[0009] Configure test parameters on the server and generate corresponding test cases, wherein the test parameters include fixed test parameters and random test parameters, wherein the fixed test parameters are defined by the user and include request type, request frequency and abnormal test type; and the random test parameters are generated by randomly selecting abnormal test types and their corresponding parameters from a preset test case library;

[0010] According to the test parameters and the corresponding test cases, a test suite including multiple test scenarios is constructed through the server, each test scenario corresponding to an exception type and parameter combination;

[0011] According to the test request, the server runs the corresponding test suite and automatically deploys the test executor to the machine under test. The test executor uses the control group technology to control the resources of various abnormal test components, receives the test instructions sent by the server, executes the test script, and calls the corresponding test components to execute the test;

[0012] After the test is completed, the test executor uploads the test results to the server, which generates a test report and restores the environment of the tested service.

[0013] As a low-intrusive software environment anomaly testing method of the present invention, further, the server is a microservice architecture deployed in a container, and has a visual interface for configuring the IP address, port number, test type and random test parameters of the machine under test; the server integrates logging and monitoring alarm functions, and introduces a failure retry mechanism to automatically retry failed test scenarios.

[0014] As a low-intrusive software environment anomaly testing method of the present invention, further, the generation of the test suite includes:

[0015] Based on the test parameters configured by the user and the generated test cases, a test suite containing multiple test scenarios is automatically built, and an optimized test script is written for each test scenario. The test script is updated through the version management system, and the priority and weight of the test scenario can be dynamically adjusted through the visual interface.

[0016] As a low-intrusive software environment abnormality testing method of the present invention, further, the test execution body includes various abnormality testing components, and the abnormality testing components include an abnormality injection module, a system monitoring module, an abnormality testing tool and related dependencies; the test execution body uses a control group technology to configure differentiated resource restriction strategies for each abnormality testing component according to different abnormality test types, including:

[0017] Limit the memory and I / O resources of components tested for CPU anomalies;

[0018] Limit the CPU and I / O resources of the components tested for memory anomalies;

[0019] Limit the CPU and memory resources of the components that are tested for disk I / O anomalies.

[0020] As a low-intrusive software environment anomaly testing method of the present invention, further, the anomaly injection module is used to simulate and trigger specific anomaly scenarios, and the system monitoring module is used to collect system operation indicators in real time, including CPU usage, memory occupancy and network delay data.

[0021] As a low-intrusive software environment anomaly testing method of the present invention, further, the step of restoring the tested service environment includes:

[0022] The server cleans up the test executable and related files on the machine under test;

[0023] Release the system resources occupied during the test, and verify whether the resources are completely released by comparing the system logs and network status;

[0024] The tested environment is restored to a preset state through an automated recovery mechanism, wherein the automated recovery mechanism supports snapshot recovery or script recovery strategies.

[0025] As a low-intrusive software environment exception testing method of the present invention, further, the exception test types of the preset test case library include CPU occupancy, memory leakage, disk I / O exception, network delay, and database connection exception; the test case library supports users to customize new exception test types and their parameter ranges, and is managed using a version control system.

[0026] As a low-intrusive software environment anomaly testing method of the present invention, further, the test report supports visual display and can be exported to PDF or Excel format; the test report includes a detailed log of the test scenario, an exception processing time comparison chart and system stability evaluation suggestions.

[0027] On the other hand, the present invention provides a low-intrusive environment anomaly testing method and system for implementing the above-mentioned testing method. The testing system includes a server and a test execution body. The server is used to generate and send test cases and coordinate the test process, which includes:

[0028] Configuration management module, used to provide a visual interface to configure the IP address, port number, test type and test parameters of the machine under test;

[0029] The test suite generation module is used to automatically build a test suite containing multiple test scenarios based on the user-configured test parameters and the preset test case library, and write an optimized test script for each test scenario;

[0030] The test execution module is used to run the test suite according to the preset test strategy, monitor the test progress and results in real time, and introduce an automatic retry mechanism for failed test scenarios;

[0031] The result collection and analysis module is used to collect the test results uploaded by the test executor, organize and compile data, and generate a visual test report containing test scenarios, test parameters, and system stability evaluation;

[0032] Logging module, used to record detailed logs during the test process, and supports version management of test scripts and test cases;

[0033] Monitoring and alarm module, used to monitor the test process in real time and trigger alarms when abnormalities occur;

[0034] The test executor is deployed on the machine under test, and is used to receive the test request sent by the server, and use the control group technology to isolate and finely manage the resources of abnormal test components, execute the test script and upload the test results.

[0035] The beneficial effects of the present invention are:

[0036] Compared with the existing technology, the low-intrusive software environment abnormality testing method proposed in this invention has significant technical advantages and innovative value. By using the control group (cgroup) technology to accurately limit the resources of the test components, it is effectively ensured that the test execution process will not have a significant impact on the performance and stability of the original software system.

[0037] The server-side and test executor separation architecture designed by the present invention greatly improves the flexibility of test deployment. Users can easily configure test parameters through the server without complex tool chain deployment and environment adaptation. The test executor can be quickly deployed to different test environments to achieve rapid iteration and reuse of test cases, significantly reducing the deployment complexity and pre-preparation cost of the test system.

[0038] By supporting the combination of fixed and random parameters on the server side, test cases with wider coverage can be generated, which can simulate more realistic and diverse abnormal situations, greatly improving the comprehensiveness and depth of abnormal testing.

[0039] Based on the microservice architecture, the testing method of the present invention has excellent scalability. The server can easily access new test modules, exception injection plug-ins and monitoring components to quickly adapt to the testing requirements of different types of software systems. The test case library supports continuous expansion and version management, providing a technical foundation for continuous integration and continuous testing.

[0040] Through automatic retry upon failure, detailed logging and monitoring mechanisms, and automated test report generation, the test executor can accurately capture and record the detailed scene of the exception, significantly improving the accuracy and credibility of the test results and providing rich information for problem location and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the architecture of the low-intrusive environmental anomaly testing system of the present invention;

[0042] Figure 2 It is a flow chart of the low-intrusive environmental anomaly testing method of the present invention. DETAILED DESCRIPTION

[0043] In order to better understand the technical solution, the method of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0044] Embodiments of the present invention, see Figure 2 As shown, a low-intrusive environmental anomaly testing method is proposed, comprising:

[0045] Construct a test architecture with a server and a test execution body separated, wherein the server is used to generate and send test cases and coordinate the test process; the test execution body is used to receive the test request sent by the server and perform fixed tests or random tests according to the test request;

[0046] Configure test parameters on the server and generate corresponding test cases, wherein the test parameters include fixed test parameters and random test parameters, wherein the fixed test parameters are defined by the user and include request type, request frequency and abnormal test type; and the random test parameters are generated by randomly selecting abnormal test types and their corresponding parameters from a preset test case library;

[0047] According to the test parameters and the corresponding test cases, a test suite including multiple test scenarios is constructed through the server, each test scenario corresponding to an exception type and parameter combination;

[0048] According to the test request, the server runs the corresponding test suite and automatically deploys the test executor to the machine under test. The test executor uses the control group technology to control the resources of various abnormal test components, receives the test instructions sent by the server, executes the test script, and calls the corresponding test components to execute the test;

[0049] After the test is completed, the test executor uploads the test results to the server, which generates a test report and restores the environment of the tested service;

[0050] In the above method, the separation architecture of the server and the test executor (agent) is one of the core designs of this embodiment. The server is responsible for the generation of test cases, the configuration of test parameters, the construction of test suites, and the control of test execution, while the test executor is responsible for executing specific test tasks on the machine under test. The advantages of this separation architecture are flexibility, scalability, and resource isolation. The server can be deployed independently of the tested environment, and the test executor can be quickly deployed to different machines under test as needed to adapt to different test environments. The server adopts a microservice architecture, and the components are loosely coupled, which is convenient for the subsequent expansion of new test modules or monitoring components. The test executor isolates the resources of the test components through the control group (cgroup) technology to ensure that the test process will not have a significant impact on the normal operation of the system under test.

[0051] The server provides two test parameter configuration methods: fixed parameter configuration and random test parameter generation. Fixed parameter configuration allows users to set fixed test parameters, such as request type, request frequency, exception type, etc., through the server's visual interface or configuration file. The server introduces a parameter verification mechanism to ensure that the parameters entered by the user are legal and valid, avoiding errors or exceptions during the test process. Random test parameter generation randomly selects abnormal test types and parameters from the preset test case library to simulate abnormal sudden scenarios in the production environment. The test case library contains a variety of abnormal scenarios, such as CPU usage, memory leaks, disk I / O abnormalities, network delays, database connection abnormalities, etc. Users can also customize the test case library to meet specific testing requirements. The test case library is managed using a version control system to ensure the accuracy and consistency of test cases.

[0052] The generation of test suites is a key step in the testing process. The server automatically builds a test suite containing multiple test scenarios based on the test parameters configured by the user and the generated test cases. Each test scenario corresponds to an exception type and parameter combination. The test suite generation process includes test scenario construction and test script writing and optimization. Test scenario construction automatically builds multiple test scenarios based on the test parameters configured by the user and the generated test cases to ensure the comprehensiveness and pertinence of the test. The server also introduces a priority and weight mechanism for test scenarios, so that users can flexibly adjust the execution order of test scenarios according to test requirements. Write corresponding test scripts for each test scenario to ensure the accuracy and efficiency of the scripts. The test scripts are optimized to reduce unnecessary resource consumption and improve the efficiency of test execution. The test scripts also support version management to ensure the accuracy and consistency of the scripts.

[0053] The test executor (agent) is the specific executor of the test task. Its core function is to isolate the resources of the test components through the control group (cgroup) technology to ensure that the test process will not have a significant impact on the normal operation of the system under test. The test executor uses the control group (cgroup) technology to configure different CPU, memory and I / O restriction policies according to different abnormal test types. For example, for components of CPU abnormality testing, the test executor (agent) limits resources other than the CPU (such as memory, storage, etc.) to ensure that the resource usage of the test component is controlled to prevent affecting the operation of the service under test. The test executor contains an exception injection module to simulate and trigger specific abnormal scenarios, such as simulating network delays, packet loss, and high CPU usage. The test executor also contains a system monitoring module to collect system operation indicators, such as CPU usage, memory usage, disk I / O, etc. These indicators can help testers analyze the performance of the system under abnormal conditions.

[0054] After the test is completed, the test executor uploads the test results to the server, and the server generates a detailed test report. The test report generation process includes result collection and collation, data cleaning and deduplication, and report generation and uploading. The server collects the test results sent back by the test executor, and organizes and compiles them. The statistical results include key indicators such as response time, error rate, exception handling time, as well as detailed test logs and screenshots. The server introduces data cleaning and deduplication mechanisms to ensure the accuracy and consistency of the test results. The server generates a detailed test report based on the collated test results. The report content covers information such as test scenarios, test parameters, test results, system stability assessment, and improvement suggestions. The report adopts a visual design to facilitate users to quickly understand and analyze test results. The report supports export and sharing in multiple formats (such as PDF, Excel, etc.).

[0055] After the test is completed, the server is responsible for restoring the tested environment to the state before the test to ensure the normal operation of the tested system. The recovery process includes environment cleanup, resource release and verification, and automated recovery mechanism. The server cleans up the test executive (agent) program and related files on the tested machine to ensure the cleanliness and security of the tested service environment. Introduce an automated cleanup mechanism to reduce the cost and risk of manual intervention. The server releases the system resources occupied during the test, such as memory, CPU, network bandwidth, etc., and introduces a resource release verification mechanism to ensure that the system resources have been completely released and restored to the state before the test. The server introduces an automated recovery mechanism to ensure that the tested environment can be automatically restored to the preset state after the test is completed. The automated recovery mechanism supports multiple recovery strategies (such as snapshot recovery, script recovery, etc.) to meet different test scenarios and requirements.

[0056] Furthermore, a low-intrusive environmental anomaly testing system based on the above method is implemented, such as Figure 1 As shown, it includes a server and a test execution body. The server is used to generate and send test cases and coordinate the test process, which includes:

[0057] Configuration management module, used to provide a visual interface to configure the IP address, port number, test type and test parameters of the machine under test;

[0058] The test suite generation module is used to automatically build a test suite containing multiple test scenarios based on the user-configured test parameters and the preset test case library, and write an optimized test script for each test scenario;

[0059] The test execution module is used to run the test suite according to the preset test strategy, monitor the test progress and results in real time, and introduce an automatic retry mechanism for failed test scenarios;

[0060] The result collection and analysis module is used to collect the test results uploaded by the test executor, organize and compile data, and generate a visual test report containing test scenarios, test parameters, and system stability evaluation;

[0061] Logging module, used to record detailed logs during the test process, and supports version management of test scripts and test cases;

[0062] Monitoring and alarm module, used to monitor the test process in real time and trigger alarms when abnormalities occur;

[0063] The test executor is deployed on the machine under test, and is used to receive the test request sent by the server, and use the control group technology to isolate and finely manage the resources of abnormal test components, execute the test script and upload the test results.

[0064] The key technical points of the present invention include an architecture that separates the server from the test executor, the application of control group (cgroup) technology, random test parameter generation and an automated recovery mechanism. Through the separation architecture, flexible deployment of test tasks and resource isolation are achieved to ensure that the impact of the test process on the system under test is minimized. Through the control group (cgroup) technology, resource isolation is performed on the test components to ensure that the resource occupancy of the test components is controlled, preventing the test process from having a significant impact on the normal operation of the system under test. Through random test parameter generation, abnormal and sudden scenarios in the production environment are simulated, which improves the comprehensiveness and depth of the test. Through the automated recovery mechanism, it is ensured that the tested environment can be quickly restored to the state before the test after the test is completed, reducing the cost and risk of manual intervention.

Claims

1. A low-intrusive environmental anomaly testing method, characterized in that: include: Build a test architecture that separates the server from the test executor, where the server is used to generate and send test cases and coordinate the test process; The test execution body is used to receive the test request sent by the server and execute a fixed test or a random test according to the test request; Configuring test parameters on the server and generating corresponding test cases, wherein the test parameters include fixed test parameters and random test parameters, and the fixed test parameters are defined by the user and include request type, request frequency, and abnormal test type; The random test parameters are generated by randomly selecting abnormal test types and their corresponding parameters from a preset test case library; According to the test parameters and the corresponding test cases, a test suite including multiple test scenarios is constructed through the server, each test scenario corresponding to an exception type and parameter combination; According to the test request, the server runs the corresponding test suite and automatically deploys the test executor to the machine under test. The test executor uses the control group technology to control the resources of various abnormal test components, receives the test instructions sent by the server, executes the test script, and calls the corresponding test components to execute the test; After the test is completed, the test executor uploads the test results to the server, which generates a test report and restores the environment of the tested service.

2. The low-intrusive environmental anomaly testing method according to claim 1, characterized in that: The server is a microservice architecture deployed in a container and has a visual interface for configuring the IP address, port number, test type and random test parameters of the machine under test; the server integrates logging and monitoring alarm functions, and introduces a failure retry mechanism to automatically retry failed test scenarios.

3. The low-intrusive environmental anomaly testing method according to claim 2, characterized in that: The generation of the test suite includes: Based on the test parameters configured by the user and the generated test cases, a test suite containing multiple test scenarios is automatically built, and an optimized test script is written for each test scenario. The test script is updated through the version management system, and the priority and weight of the test scenario can be dynamically adjusted through the visual interface.

4. The low-intrusive environmental anomaly testing method according to claim 1, characterized in that: The test execution body includes various abnormal test components, including abnormal injection modules, system monitoring modules, abnormal test tools and related dependencies; The test executor uses control group technology to configure differentiated resource restriction strategies for each abnormal test component according to different abnormal test types, including: Limit the memory and I / O resources of components tested for CPU anomalies; Limit the CPU and I / O resources of the components tested for memory anomalies; Limit the CPU and memory resources of the components that are tested for disk I / O anomalies.

5. The low-intrusive environmental anomaly testing method according to claim 4, characterized in that: The exception injection module is used to simulate and trigger specific exception scenarios, and the system monitoring module is used to collect system operation indicators in real time, including CPU usage, memory occupancy and network delay data.

6. The low-intrusive environmental anomaly testing method according to claim 1, characterized in that: The steps of restoring the tested service environment include: The server cleans up the test executable and related files on the machine under test; Release the system resources occupied during the test, and verify whether the resources are completely released by comparing the system logs and network status; The tested environment is restored to a preset state through an automated recovery mechanism, wherein the automated recovery mechanism supports snapshot recovery or script recovery strategies.

7. The low-intrusive environmental anomaly testing method according to claim 1, characterized in that: The abnormal test types of the preset test case library include CPU usage, memory leakage, disk I / O abnormality, network delay, and database connection abnormality; the test case library supports users to customize new abnormal test types and their parameter ranges, and is managed using a version control system.

8. The low-intrusive environmental anomaly testing method according to claim 1, characterized in that: The test report supports visual display and can be exported to PDF or Excel format; the test report includes a detailed log of the test scenario, an exception handling time comparison chart, and system stability assessment recommendations.

9. A low-intrusive environmental anomaly testing system, characterized in that: The method for implementing any one of claims 1 to 8, wherein the test system comprises a server and a test execution body, wherein the server is used to generate and send test cases and coordinate the test process, and comprises: Configuration management module, used to provide a visual interface to configure the IP address, port number, test type and test parameters of the machine under test; The test suite generation module is used to automatically build a test suite containing multiple test scenarios based on the user-configured test parameters and the preset test case library, and write an optimized test script for each test scenario; The test execution module is used to run the test suite according to the preset test strategy, monitor the test progress and results in real time, and introduce an automatic retry mechanism for failed test scenarios; The result collection and analysis module is used to collect the test results uploaded by the test executor, organize and compile data, and generate a visual test report containing test scenarios, test parameters, and system stability evaluation; Logging module, used to record detailed logs during the test process, and supports version management of test scripts and test cases; Monitoring and alarm module, used to monitor the test process in real time and trigger alarms when abnormalities occur; The test executor is deployed on the machine under test, and is used to receive the test request sent by the server, and use the control group technology to isolate and finely manage the resources of abnormal test components, execute the test script and upload the test results.

Images