A system testing method, device, equipment and storage medium

Abstract

The present application discloses a system testing method, device, equipment and storage medium, which relates to the field of system testing technology, including: determining the fault testing process of the target system, and constructing multiple test devices based on the fault testing process; determining the operating mode between the test devices, and determining the corresponding target testing method based on the operating mode; the target testing method includes fault testing or chaos engineering; and performing system testing on the target system through the target testing method. The present application constructs multiple test devices based on the determined fault testing process of the target system, and then determines the target testing method according to the operating mode between the test devices, thereby realizing the integration of chaos engineering and fault testing, achieving a high degree of code reuse, and then performing system testing on the target system through the target testing method. The above-mentioned test device can automatically generate user data, avoiding the operation of chaos engineering in the actual production environment, thereby avoiding the risk of user data loss.

Description

Technical Field

[0001] The present invention relates to the technical field of system testing, and in particular to a system testing method, device, equipment and storage medium. Background Art

[0002] Currently, chaos engineering is described as "a discipline that conducts experiments on distributed systems with the goal of building confidence in the system's ability to withstand turbulent conditions in a production environment." In other words, chaos engineering is a complex technical means to improve the resilience of technical architecture. Chaos engineering can ensure the stability of the system through experiments. It aims to nip failures in the bud, that is, identify failures before they cause interruptions. Or it can actively create failures, test the system's behavior under various pressures, identify and fix failure problems, and avoid serious consequences. However, chaos engineering generally runs on an actual production environment, which may cause loss of user data. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a system testing method, device, equipment and storage medium, which can prevent chaos engineering from running in an actual production environment, thereby reducing the risk of user data loss. The specific scheme is as follows:

[0004] Determine a fault test process of a target system, and construct a plurality of test devices based on the fault test process;

[0005] Determine the operation mode between the test devices, and determine a corresponding target test method based on the operation mode; the target test method includes fault testing or chaos engineering;

[0006] The target system is subjected to system testing by means of the target testing method.

[0007] Optionally, constructing a plurality of test devices based on the fault test process includes:

[0008] Analyze the fault testing process to obtain corresponding analysis results, and divide the key steps in the fault testing process based on the analysis results to obtain corresponding process division results;

[0009] A plurality of test devices are constructed according to the process division result, each of which is used to execute the key steps in the fault test process.

[0010] Optionally, constructing a plurality of test devices based on the fault test process includes:

[0011] Constructing a background constructor, a fault trigger, a test point constructor, and an environment checker based on the fault test process;

[0012] Among them, the background constructor is a test device for creating a virtual machine on the target system and for building actual business on the virtual machine. The fault trigger is a test device for triggering a system fault of the target system and for judging whether the processing result obtained by the target system for processing the system fault meets the expected processing result. The test point constructor is a test device for simulating configuration operations on the target system and for judging whether the configuration operation is abnormal. The environment checker is a test device for detecting the status of each virtual machine in the target system and judging whether the virtual machine is abnormal.

[0013] Optionally, after performing system testing on the target system in the target testing manner, the method further includes:

[0014] By using the fault trigger, detecting whether the fault handling of the target system for the system fault is normal, obtaining a corresponding fault detection result, and by using the environment checker to monitor whether the current stable state of each virtual machine on the target system meets the test hypothesis when the system fault occurs, obtaining a corresponding steady-state monitoring result; the test hypothesis is a hypothesis that the current stable state is consistent with the initial stable state defined for the target system;

[0015] An operating state of the target system is determined based on the fault detection result and the steady-state monitoring result.

[0016] Optionally, determining the operating state of the target system based on the fault detection result and the steady-state monitoring result includes:

[0017] If the fault detection result indicates that the fault handling is abnormal, determining that the target system is operating abnormally;

[0018] If the fault detection result indicates that the fault processing is normal and the steady-state monitoring result indicates that the current steady state of each virtual machine on the target system does not satisfy the test hypothesis, it is determined that the target system has not processed the problem caused by the system fault;

[0019] If the fault detection result indicates that the fault handling is normal and the steady-state monitoring result indicates that the current steady state of each of the virtual machines on the target system meets the test assumption, it is determined that the target system is operating normally.

[0020] Optionally, the operating modes include: serial mode and parallel mode.

[0021] Optionally, determining a corresponding target test method based on the operation mode; the target test method includes fault testing or chaos engineering, including:

[0022] If the operation mode is a serial mode, the test device is operated according to the serial mode, and the target test mode is determined as a fault test;

[0023] If the operation mode is a parallel mode, the test device is operated according to the parallel mode, and the target test method is determined as chaos engineering.

[0024] In a second aspect, the present application discloses a system testing device, comprising:

[0025] A test process determination module is used to determine the fault test process of the target system;

[0026] A test device construction module, used to construct a plurality of test devices based on the fault test process;

[0027] An operation mode determination module, used to determine the operation mode between the test devices;

[0028] A test mode determination module, used to determine a corresponding target test mode based on the operation mode; the target test mode includes fault testing or chaos engineering;

[0029] The system testing module is used to perform system testing on the target system through the target testing method.

[0030] In a third aspect, the present application discloses an electronic device, comprising:

[0031] Memory, used to store computer programs;

[0032] The processor is used to execute the computer program to implement the steps of the system testing method disclosed above.

[0033] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned disclosed system testing method are implemented.

[0034] It can be seen that the present application provides a system testing method, including: determining the fault testing process of the target system, and constructing multiple test devices based on the fault testing process; determining the operating mode between the test devices, and determining the corresponding target testing method based on the operating mode; the target testing method includes fault testing or chaos engineering; and performing a system test on the target system through the target testing method. It can be seen that the present application constructs multiple test devices based on the determined fault testing process of the target system, and then determines the target testing method according to the operating mode between the test devices, implements fault testing or chaos engineering, and then performs a system test on the target system through the target testing method. The technical solution of the present application realizes the integration of chaos engineering and fault testing, and achieves a high degree of code reuse. The above-mentioned test device constructed based on the fault testing process can automatically generate user data, avoiding the operation of chaos engineering in the actual production environment, thereby avoiding the risk of user data loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0036] Figure 1 A flow chart of a system testing method disclosed in this application;

[0037] Figure 2 A specific system testing method flow chart disclosed in this application;

[0038] Figure 3 A specific system testing method flow chart disclosed in this application;

[0039] Figure 4 A schematic diagram of a fault testing process disclosed in this application;

[0040] Figure 5 A schematic diagram of a fault test process division method disclosed in this application;

[0041] Figure 6 This is a schematic diagram of a serial mode operation disclosed in this application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Currently, chaos engineering is described as "a discipline that conducts experiments on distributed systems to build confidence in the system's ability to withstand turbulent conditions in the production environment." That is, chaos engineering can ensure the stability of the system through experiments, aiming to nip failures in the bud, that is, to identify failures before they cause interruptions, or to actively create failures, test the behavior of the system under various pressures, identify and fix failure problems, and avoid serious consequences. However, chaos engineering generally runs on the actual production environment, which may cause the loss of user data. To this end, this application provides a system testing solution that can prevent chaos engineering from running in the actual production environment, thereby reducing the risk of user data loss.

[0044] The embodiment of the present invention discloses a system testing method, see Figure 1 As shown, the method includes:

[0045] Step S11: determine a fault test process of the target system, and construct a plurality of test devices based on the fault test process.

[0046] It should be pointed out that the key points of the current chaos engineering experiment are: first, define and measure the stable state of the system; second, create a hypothesis, that is, assume that the system can continue to maintain this stable state when a fault occurs, and the stable state does not change; third, simulate possible faults in the real world; fourth, compare the steady-state indicators with the indicators collected after the interference is injected into the system. It can be seen that the chaos engineering experiment focuses on the overall performance of the system, and the fault test focuses on the system's handling of faults. Compared with the fault test, the focus of chaos is broader. If the fault test cannot pass, then the chaos engineering experiment cannot pass. Therefore, according to the above key points of chaos engineering, the fault test process is transformed, and the inspection of the stable state is added to achieve chaos engineering while achieving fault testing. In order to adapt chaos engineering to the actual needs of cloud platform operation, the key points of chaos engineering are modified and expanded accordingly, so that the key points of cloud platform chaos engineering implemented in this application can be: first, steady state. For cloud platforms, the most intuitive reaction to steady state is the state of virtual machines, that is, checking whether the computing, storage, network, etc. of the virtual machines meet the standards; second, assumptions. Since some faults in fault testing may cause the system to be unavailable, assumptions need to be classified according to whether the faults will affect the steady state; third, triggering faults and checking the system's handling of faults, that is, the fault test of the system can be compatible with chaos engineering; fourth, checking the status of all virtual machines on the cloud platform; fifth, the status of chaos engineering experiment results. If the fault check fails, it is directly judged that the chaos engineering experiment has failed. If the fault check passes and the steady state check also passes, it is directly judged that the chaos engineering experiment has passed. If the fault check passes but the steady state check fails, further inspection is required to determine whether the fault has an impact on the steady state of the system. If the fault affects the steady state of the system, the chaos engineering experiment is judged to have passed, otherwise it is judged to have failed. The above stable state is mainly for distributed systems, so the target system can be a distributed system.

[0047] In this embodiment, for the current target system, a corresponding fault test process is determined, that is, for the distributed system, a fault test process corresponding to the distributed system is determined, and the fault test process is a modified fault test process that can realize all the key points of both fault testing and chaos engineering, and then multiple test devices are constructed based on the fault test process. In other words, the switching between fault testing and chaos engineering can be realized by running the constructed test devices in different operation modes.

[0048] Step S12: determining the operation mode between the test devices, and determining a corresponding target test method based on the operation mode; the target test method includes fault testing or chaos engineering.

[0049] In this embodiment, the operation mode includes a serial mode and a parallel mode, so before determining the target test mode, the operation model between the test devices must be determined first. It is understandable that if the current operation mode is a serial mode, the test device is run according to the serial mode, and the target test mode is determined as a fault test, that is, running the test device in sequence in a serial mode is a fault test. If the operation mode is a parallel mode, the test device is run according to the parallel mode, and the target test mode is determined as chaos engineering, that is, running the test device in a parallel mode is chaos engineering, that is, adding an inspection of the fault handling process to the failure of the chaos engineering helps to locate the problem point and improve the inspection of the fault handling process.

[0050] Step S13: performing a system test on the target system by using the target test method.

[0051] In this embodiment, if the current operating mode of the above-mentioned test device is serial mode, it indicates that the current execution is fault testing, and the target system is system tested through the fault testing method. If the current operating mode of the above-mentioned test device is parallel mode, it indicates that the current execution is chaos engineering, and the target system is system tested through the chaos engineering experiment method.

[0052] It can be seen that the embodiment of the present application constructs multiple test devices based on the determined fault test process of the target system, and then determines the target test method according to the operation mode between the test devices to implement fault testing or chaos engineering, and then performs system testing on the target system through the target test method. The technical solution of the present application realizes the integration of chaos engineering and fault testing, and achieves a high degree of code reuse. The above-mentioned test devices constructed based on the fault test process can automatically generate user data, avoiding the operation of chaos engineering in the actual production environment, thereby avoiding the risk of user data loss.

[0053] See also Figure 2 As shown, the embodiment of the present invention discloses a specific system testing method. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution.

[0054] Step S21: Determine the fault testing process of the target system.

[0055] Step S22: Analyze the fault testing process to obtain corresponding analysis results, and divide the key steps in the fault testing process based on the analysis results to obtain corresponding process division results.

[0056] In this embodiment, after determining the fault testing process of the target system, the fault testing process is analyzed to obtain corresponding analysis results, and then the key steps in the fault testing process are divided according to the analysis results to obtain corresponding process division results.

[0057] Step S23: constructing a plurality of test devices for executing each of the key steps in the fault test process according to the process division result.

[0058] In this embodiment, a plurality of test devices respectively used to execute the key steps in the fault test process are constructed according to the process division result. That is to say, the key steps in the fault test process can be executed by the test devices.

[0059] Step S24: determining the operation mode between the test devices, and determining a corresponding target test method based on the operation mode; the target test method includes fault testing or chaos engineering.

[0060] Step S25: performing system testing on the target system by using the target testing method.

[0061] For the specific contents of the above-mentioned step S21, step S24 to step S25, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0062] It can be seen that the embodiment of the present application analyzes the fault test process of the determined target system and divides the key steps to construct multiple test devices, and then determines the target test method according to the operation mode between the test devices to implement fault testing or chaos engineering, and then performs system testing on the target system through the target test method. The technical solution of the present application realizes the integration of chaos engineering and fault testing, and achieves a high degree of code reuse. The above-mentioned test device constructed based on the fault test process can automatically generate user data, avoiding the operation of chaos engineering in the actual production environment, thereby avoiding the risk of user data loss.

[0063] See also Figure 3 As shown, the embodiment of the present invention discloses a specific system testing method. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution.

[0064] Step S31: determine the fault test process of the target system, and construct a background constructor, a fault trigger, a test point constructor, and an environment checker based on the fault test process.

[0065] In this embodiment, the fault test process of the target system is determined, such as Figure 4 As shown, Figure 4The fault test process for the current target system is shown, and then the background constructor, fault trigger, test point constructor, and environment checker are constructed based on the fault test process, such as Figure 5 As shown, the key steps in the fault test process are divided according to the analysis results obtained by analyzing the fault test process, and then the background constructor, fault trigger, test point constructor, and environment checker are constructed respectively according to the obtained process division results, which is equivalent to planning and dividing the fault test process, and then constructing four key components corresponding to the fault test process according to the segmentation results, that is, dividing each key step into each constructed component, wherein the background constructor is a test component for creating a virtual machine on the target system and for building actual business on the virtual machine, the fault trigger is a test component for triggering a system fault for the target system and for judging whether the processing result obtained by the target system processing the system fault meets the expected processing result, the test point constructor is a test component for simulating the configuration operation of the target system and for judging whether the configuration operation is abnormal, and the environment checker is a test component for detecting the status of each virtual machine in the target system and judging whether the virtual machine is abnormal. That is to say, the above-mentioned background constructor is mainly used to create a virtual machine on the target system and build actual business on the virtual machine. The actual business may include but is not limited to database business, etc., so as to simulate the real virtual machine operation scenario. The above-mentioned fault trigger is mainly used to trigger the fault, and check the system at each stage of triggering the fault to determine whether the system handles the fault as expected. The above-mentioned test point constructor is mainly used to simulate some configuration operations of the user on the system, including all configuration operations that may be performed on the system, such as creating a virtual machine, backing up a virtual machine, etc., and determine whether the configuration operation is abnormal. The above-mentioned environment checker is used to detect the status of all virtual machines on the system, that is, to detect the stable state of all virtual machines on the system, and determine whether the virtual machines on the system have intolerable abnormalities.

[0066] Step S32: Determine the operating mode among the background constructor, the fault trigger, the test point constructor and the environment checker, and determine the corresponding target test method based on the operating mode; the target test method includes fault testing or chaos engineering.

[0067] In this embodiment, the background constructor, the fault trigger, the test point constructor, and the environment checker may be run in a serial mode or a parallel mode. If it is determined that the background constructor, the fault trigger, the test point constructor, and the environment checker are run in a serial mode, Figure 6As shown, all key steps in the fault testing process are performed in series. First, the background construction is performed by the background constructor, and then the test point constructor and the fault trigger are executed crosswise. Finally, the environment checker performs the environment check. At this time, running the background constructor, the fault trigger, the test point constructor and the environment checker in a serial mode is to perform the fault test. At this time, the faults that can be run are all the same. If it is determined that the background constructor, the fault trigger, the test point constructor and the environment checker are running in parallel mode, that is, the background constructor, the fault trigger, the test point constructor and the environment checker continue to run in parallel, then the background constructor can simulate the internal business of the user virtual machine to generate the internal data of the virtual machine; the test point constructor can simulate the configuration operation of the user environment to generate the environment operation data, and the test point constructor checks each stage of the operation to determine the operating state of the system; the fault constructor continuously triggers different faults, and the selected faults are faults that will not affect the steady state, so in an ideal case, the parallel mode can continue to run without problems; the environment checker will continuously monitor the steady state of the system, and once the environment checker detects that the steady state of the system does not meet the standard, it will throw an error and stop the system test. In other words, running the background constructor, the fault trigger, the test point constructor and the environment checker in serial mode is fault testing, and running the background constructor, the fault trigger, the test point constructor and the environment checker in serial-parallel mode is chaos engineering. This embodiment can implement all key points of chaos engineering by running the background constructor, the fault trigger, the test point constructor and the environment checker in parallel mode, and automatically generates user data during the operation, avoiding the risk of user data loss that may occur when running in a production environment.

[0068] Step S33: performing system testing on the target system by using the target testing method.

[0069] Step S34: Detect through the fault trigger whether the target system's fault handling for the system fault is normal, and obtain the corresponding fault detection result; and when the system fault occurs, monitor through the environment checker whether the current stable state of each virtual machine on the target system meets the test hypothesis, and obtain the corresponding steady-state monitoring result; the test hypothesis is an assumption that the current stable state is consistent with the initial stable state defined for the target system.

[0070] In this embodiment, after the target system is tested by the fault test or chaos engineering in the above target test method, the fault trigger can be used to detect whether the fault handling of the target system for the system fault is normal, and the corresponding fault detection result can be obtained. When the system fault occurs, the current stable state of each virtual machine on the target system is monitored by the environment checker to see whether it meets the test assumptions, and the corresponding steady-state monitoring result can be obtained.

[0071] Step S35: Determine the operating state of the target system based on the fault detection result and the steady-state monitoring result.

[0072] In this embodiment, when the fault detection result shows that the fault handling is abnormal, it is determined that the target system is operating abnormally; when the fault detection result shows that the fault handling is normal and the steady-state monitoring result shows that the current stable state of each of the virtual machines on the target system does not meet the test hypothesis, it is determined that the target system has not handled the problem caused by the system fault; when the fault detection result shows that the fault handling is normal and the steady-state monitoring result shows that the current stable state of each of the virtual machines on the target system meets the test hypothesis, it is determined that the target system is operating normally.

[0073] For the specific content of the above step S33, reference may be made to the corresponding content disclosed in the above embodiments, which will not be repeated here.

[0074] It can be seen that the embodiment of the present application constructs a background constructor, a fault trigger, a test point constructor, and an environment checker based on the determined fault test process of the target system, and then determines the target test method according to the operating mode between the background constructor, the fault trigger, the test point constructor, and the environment checker to implement fault testing or chaos engineering, and then performs system testing on the target system through the target test method, and determines the operating state of the target system based on the fault detection results and the steady-state monitoring results obtained during the system testing process. The technical solution of the present application realizes the integration of chaos engineering and fault testing, and achieves a high degree of code reuse. The above-mentioned test device constructed based on the fault test process can automatically generate user data, avoiding the operation of chaos engineering in an actual production environment, thereby avoiding the risk of user data loss.

[0075] Accordingly, the embodiment of the present application further discloses a system testing device, which includes:

[0076] A test process determination module 11 is used to determine the fault test process of the target system;

[0077] A test device construction module 12, configured to construct a plurality of test devices based on the fault test process;

[0078] An operation mode determination module 13, used to determine the operation mode between the test devices;

[0079] A test mode determination module 14 is used to determine a corresponding target test mode based on the operation mode; the target test mode includes fault testing or chaos engineering;

[0080] The system testing module 15 is used to perform system testing on the target system through the target testing method.

[0081] In this embodiment, the corresponding fault test process is determined by the test process determination module 11. For the distributed system, the fault test process corresponding to the distributed system is determined by the test process determination module 11. The fault test process is a modified fault test process that can implement fault testing and all key points of chaos engineering. Then, a plurality of test devices are constructed based on the fault test process by the test device construction module 12, and the operation mode determination module 13 is used to determine the operation mode between the test devices, wherein the operation mode includes a serial mode and a parallel mode. If the current operation mode determined by the above-mentioned operation mode determination module 13 is a serial mode, the test device is operated according to the serial mode, and the target test mode is determined as a fault test. That is to say, the test devices are operated sequentially in a serial mode, which is a fault test. If the current operation mode determined by the operation mode determination module 13 is a parallel mode, the test device is run according to the parallel mode, and the target test mode is determined as chaos engineering. That is to say, running the test device in parallel mode is chaos engineering, that is, adding an inspection of the fault handling process to the failure of chaos engineering helps to locate the problem point and improve the inspection of the fault handling process. Then, the target test mode can be determined by the test mode determination module 14, and the target test mode includes fault testing and chaos engineering testing. In other words, the switching between fault testing and chaos engineering can be achieved by running the constructed test device in different operation modes. When the current operation mode of the test device is a serial mode, it indicates that the current execution is a fault test, and the system test module 15 performs a system test on the target system through a fault test mode. If the current operation mode of the test device is a parallel mode, it indicates that the current execution is a chaos engineering, and the system test module 15 performs a system test on the target system through a chaos engineering experiment.

[0082] As can be seen from the above, the embodiment of the present application constructs multiple test devices based on the determined fault test process of the target system, and then determines the target test method according to the operation mode between the test devices to implement fault testing or chaos engineering, and then performs system testing on the target system through the target test method. The technical solution of the present application realizes the integration of chaos engineering and fault testing, and achieves a high degree of code reuse. The above-mentioned test devices constructed based on the fault test process can automatically generate user data, avoiding the operation of chaos engineering in the actual production environment, thereby avoiding the risk of user data loss.

[0083] In some specific embodiments, the test device construction module 12 may specifically include:

[0084] A test process analysis unit, used to analyze the fault test process and obtain corresponding analysis results;

[0085] A step division unit, used for dividing the key steps in the fault testing process based on the analysis result to obtain corresponding process division results;

[0086] The first construction unit is used to construct a plurality of test devices respectively used to execute the key steps in the fault test process according to the process division result.

[0087] In some specific embodiments, the test device construction module 12 may specifically include:

[0088] A second construction unit is used for testing the device to construct a background constructor, a fault trigger, a test point constructor, and an environment checker based on the fault test process;

[0089] Among them, the background constructor is a test device for creating a virtual machine on the target system and for building actual business on the virtual machine. The fault trigger is a test device for triggering a system fault of the target system and for judging whether the processing result obtained by the target system for processing the system fault meets the expected processing result. The test point constructor is a test device for simulating configuration operations on the target system and for judging whether the configuration operation is abnormal. The environment checker is a test device for detecting the status of each virtual machine in the target system and judging whether the virtual machine is abnormal.

[0090] In some specific embodiments, the system testing device may further include:

[0091] A fault detection module, used to detect whether the fault handling of the target system for the system fault is normal through the fault trigger, and obtain a corresponding fault detection result;

[0092] A steady-state monitoring module, used for monitoring whether the current steady state of each virtual machine on the target system satisfies a test hypothesis through an environment checker when the system failure occurs, and obtaining a corresponding steady-state monitoring result; the test hypothesis is a hypothesis that the current steady state is consistent with an initial steady state defined for the target system;

[0093] An operating status determination module is used to determine the operating status of the target system based on the fault detection result and the steady-state monitoring result.

[0094] In some specific embodiments, the operation status determination module may specifically include:

[0095] A first determination unit, configured to determine that the target system is operating abnormally if the fault detection result indicates that the fault processing is abnormal;

[0096] A second determination unit is configured to determine that the target system has not processed the problem caused by the system failure if the fault detection result indicates that the fault processing is normal and the steady-state monitoring result indicates that the current steady state of each virtual machine on the target system does not satisfy the test hypothesis;

[0097] A third determination unit is configured to determine that the target system is operating normally if the fault detection result indicates that the fault processing is normal and the steady-state monitoring result indicates that the current steady state of each virtual machine on the target system satisfies the test hypothesis.

[0098] In some specific embodiments, the test mode determination module 14 may specifically include:

[0099] A first determining unit, configured to, when the operating mode is a serial mode, operate the test device according to the serial mode and determine the target test mode as a fault test;

[0100] The second determination unit is used to run the test device according to the parallel mode when the operation mode is the parallel mode, and determine the target test method as chaos engineering.

[0101] Furthermore, an embodiment of the present application also provides an electronic device. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the system testing method disclosed in any of the above embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0102] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0103] In addition, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0104] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the system testing method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0105] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the system testing method steps disclosed in any of the aforementioned embodiments are implemented.

[0106] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0107] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0108] The above is a detailed introduction to a system testing method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A system testing method, characterized in that: include: Determine a fault test process of a target system, and construct a plurality of test devices based on the fault test process; Determine the operation mode between the test devices, and determine a corresponding target test method based on the operation mode; the target test method includes fault testing or chaos engineering; Performing system testing on the target system by using the target testing method; Wherein, the constructing of multiple test devices based on the fault test process includes: Constructing a background constructor, a fault trigger, a test point constructor, and an environment checker based on the fault test process; Wherein, the background constructor is a test device for creating a virtual machine on the target system and for building actual business on the virtual machine, the fault trigger is a test device for triggering a system fault for the target system and for judging whether the processing result obtained by the target system for processing the system fault meets the expected processing result, the test point constructor is a test device for simulating the configuration operation of the target system and for judging whether the configuration operation is abnormal, and the environment checker is a test device for detecting the status of each virtual machine in the target system and judging whether the virtual machine is abnormal; Wherein, the operation modes include: serial mode and parallel mode; Accordingly, the corresponding target test mode is determined based on the operation mode; the target test mode includes fault testing or chaos engineering, including: If the operation mode is a serial mode, the test device is operated according to the serial mode, and the target test mode is determined as a fault test; a system test is performed on the target system in the fault test mode; If the operation mode is a parallel mode, the test device is operated according to the parallel mode, and the target test method is determined to be chaos engineering; and a system test is performed on the target system through a chaos engineering experiment method.

2. The system testing method according to claim 1, characterized in that: The constructing of multiple test devices based on the fault test process includes: Analyze the fault testing process to obtain corresponding analysis results, and divide the key steps in the fault testing process based on the analysis results to obtain corresponding process division results; A plurality of test devices are constructed according to the process division result, each of which is used to execute the key steps in the fault test process.

3. The system testing method according to claim 1, characterized in that: After the target system is tested in the target testing mode, the method further includes: By using the fault trigger, detecting whether the fault handling of the target system for the system fault is normal, obtaining a corresponding fault detection result, and by using the environment checker to monitor whether the current stable state of each virtual machine on the target system meets the test hypothesis when the system fault occurs, obtaining a corresponding steady-state monitoring result; the test hypothesis is a hypothesis that the current stable state is consistent with the initial stable state defined for the target system; An operating state of the target system is determined based on the fault detection result and the steady-state monitoring result.

4. The system testing method according to claim 3, characterized in that: The determining the operating state of the target system based on the fault detection result and the steady-state monitoring result includes: If the fault detection result indicates that the fault handling is abnormal, determining that the target system is operating abnormally; If the fault detection result indicates that the fault processing is normal and the steady-state monitoring result indicates that the current steady state of each virtual machine on the target system does not satisfy the test hypothesis, it is determined that the target system has not processed the problem caused by the system fault; If the fault detection result indicates that the fault handling is normal and the steady-state monitoring result indicates that the current steady state of each of the virtual machines on the target system meets the test assumption, it is determined that the target system is operating normally.

5. A system testing device, characterized in that: include: A test process determination module is used to determine the fault test process of the target system; A test device construction module, used to construct a plurality of test devices based on the fault test process; An operation mode determination module, used to determine the operation mode between the test devices; A test mode determination module, used to determine a corresponding target test mode based on the operation mode; The target testing method includes fault testing or chaos engineering; A system testing module, used for performing a system test on the target system by using the target testing method; Wherein, the test device building module includes: A second construction unit is used to construct a background constructor, a fault trigger, a test point constructor, and an environment checker based on the fault test process; wherein the background constructor is a test device for creating a virtual machine on the target system and for building actual business on the virtual machine, the fault trigger is a test device for triggering a system fault for the target system and for judging whether the processing result obtained by the target system for processing the system fault meets the expected processing result, the test point constructor is a test device for simulating the configuration operation of the target system and for judging whether the configuration operation is abnormal, and the environment checker is a test device for detecting the status of each virtual machine in the target system and judging whether the virtual machine is abnormal; Wherein, the operation mode includes: serial mode and parallel mode; accordingly, the test mode determination module includes: A first determining unit is used for, if the operation mode is a serial mode, running the test device according to the serial mode, and determining the target test mode as a fault test; and performing a system test on the target system in the fault test mode; The second determination unit is used to, if the operation mode is a parallel mode, run the test device according to the parallel mode, and determine the target test method as chaos engineering; and perform system testing on the target system through a chaos engineering experiment method.

6. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the system testing method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the system testing method according to any one of claims 1 to 4 are implemented.

Images