Abnormal scene testing method and electronic equipment

Abstract

The present invention discloses an abnormal scenario testing method and electronic equipment, the method comprising: obtaining a system to be tested, the system to be tested comprising at least one module to be tested; determining fault information corresponding to each module to be tested and at least one server to be tested; testing each server to be tested according to test task information and fault information, and determining a test result; performing abnormality detection on implicit correlation indicators of the system to be tested according to the test result, and determining a detection result, wherein the implicit correlation indicators are determined based on historical operation data, thereby solving the problem that observation indicators can only be determined based on user selection during abnormal scenario testing, and realizing automatic abnormality detection by analyzing historical operation data to determine implicit correlation indicators, without the need for users to select observation indicators, thereby avoiding the problem of improper selection of observation indicators due to users' unfamiliarity with the system to be tested, and rationally selecting observation indicators, timely discovering system abnormalities, and improving the efficiency of abnormal scenario testing.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an abnormal scenario testing method and electronic equipment. Background Art

[0002] With the development of microservices and cloud-native technologies, distributed systems have become prevalent across the industry. However, distributed system architectures contain numerous service components and complex dependencies between them, making it difficult to assess the impact of a single failure on the entire system. Furthermore, long request chains can make identifying and locating problems more difficult if fundamental services like monitoring, alerting, and logging are inadequate. Furthermore, with rapid business and technology iterations, ensuring the stability and high availability of the system remains a significant challenge. Implementing exception scenario testing is essential for building a highly available distributed architecture system. Existing exception scenario testing methods can only determine test and observation metrics based on user selections and are unable to detect anomalies based on the actual system operation. Summary of the Invention

[0003] The present invention provides an abnormal scenario testing method and electronic equipment to solve the problem that observation indicators can only be determined according to user selection.

[0004] According to one aspect of the present invention, a method for abnormal scenario testing is provided, comprising:

[0005] Acquire a system to be tested, where the system to be tested includes at least one module to be tested;

[0006] Determining fault information corresponding to each of the modules to be tested and at least one server to be tested;

[0007] Testing each of the servers to be tested according to the test task information and the fault information, and determining the test results;

[0008] According to the test results, an abnormality detection is performed on the implicit correlation indicators of the system to be tested to determine the detection results, wherein the implicit correlation indicators are determined according to historical operation data.

[0009] According to another aspect of the present invention, an electronic device is provided, comprising:

[0010] at least one processor; and

[0011] a memory communicatively connected to the at least one processor; wherein,

[0012] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the abnormal scenario testing method described in any embodiment of the present invention.

[0013] The technical solution of the embodiment of the present invention is to obtain a system to be tested, which includes at least one module to be tested; determine the fault information corresponding to each module to be tested and at least one server to be tested; test each server to be tested according to the test task information and the fault information to determine the test result; perform anomaly detection on the implicit correlation indicators of the system to be tested according to the test result to determine the test result, wherein the implicit correlation indicator is determined according to historical operation data, which solves the problem that the observation indicator can only be determined according to the user's selection during the abnormal scenario test, tests the server to be tested according to the test task information and the fault information to determine the test result, performs anomaly detection on the implicit correlation indicators of the system to be tested according to the test result, determines the implicit correlation indicator by analyzing the historical operation data, realizes automatic selection of implicit correlation indicators, and then realizes automated anomaly detection, does not require the user to select the observation indicator, and avoids the problem of improper selection of observation indicators due to the user's unfamiliarity with the system to be tested, reasonably selects observation indicators, discovers system anomalies in time, and improves the efficiency of abnormal scenario testing.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 This is a flowchart of an abnormal scenario testing method provided according to the first embodiment of the present invention;

[0017] Figure 2 This is a flowchart of an abnormal scenario testing method provided according to the second embodiment of the present invention;

[0018] Figure 3 It is a structural diagram of an electronic device for implementing the abnormal scenario testing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] Example 1

[0022] Figure 1 A flowchart of an abnormal scenario testing method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of performing abnormal scenario testing on a system. The method can be executed by an abnormal scenario testing device. The abnormal scenario testing device can be implemented in the form of hardware and / or software. The abnormal scenario testing device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0023] S101: Acquire a system to be tested, where the system to be tested includes at least one module to be tested.

[0024] In this embodiment, the system to be tested can be specifically understood as a system with test requirements, and the module to be tested can be specifically understood as a module in the system with test requirements. A system contains one or more modules. When testing, only one module can be tested, or multiple modules can be tested. When testing multiple modules, they can be tested simultaneously, in a set order, or according to the order in which the modules work together. This application does not limit this.

[0025] The system to be tested and the module to be tested can be selected by the user himself, or a test document can be generated in advance according to the test requirements, and the system to be tested and the module to be tested can be determined by parsing the test document.

[0026] S102: Determine the fault information corresponding to each module to be tested and at least one server to be tested.

[0027] In this embodiment, fault information can be specifically understood as information used to simulate a fault, such as fault start time, fault duration, and key fault parameters. The number of faults can be one or more. The server to be tested can be specifically understood as a server with testing requirements. The system cannot process business without servers, and the system can be deployed through servers. Therefore, when testing the system, it is necessary to select servers that actually process business.

[0028] Different fault types can be pre-set, with different parameters corresponding to different faults, or different parameters can be directly set to describe the fault. Users can select the parameter type based on actual project requirements and set the corresponding parameters to generate fault information. Parameter type, parameter value, and other information can also be defined in the test document, and fault information can be automatically generated by parsing the test document.

[0029] S103: Test each server to be tested according to the test task information and the fault information, and determine the test result.

[0030] In this embodiment, test task information can be specifically understood as information used to simulate system operation and conduct system testing. For example, for a trading system, test task information can be a transaction executed by the server under test. Test results can be data generated during the test or after the test. Fault information is injected into the test server, and the test task is executed by running the test task information to test the server under test and obtain test results. Test results can be collected at regular intervals during the operation of the server under test, such as CPU, memory, and disk usage. Fault information injection can be achieved by calling ChaosBlade via HTTP requests to perform operations such as fault creation, destruction, and querying. Other chaos experiment tools can also be used to complete fault information injection. Test task information is implemented using a .jmx script. The script's concurrency and execution time can be set to execute the test task. This can be achieved by integrating relevant Xmeter functions. Since there may be multiple modules under test and multiple faults to be injected, different modules under test and different faults can be tested sequentially or simultaneously.

[0031] S104: Perform anomaly detection on implicit correlation indicators of the system to be tested according to the test results to determine the detection results, wherein the implicit correlation indicators are determined according to historical operation data.

[0032] In this embodiment, the implicit correlation indicator can be specifically understood as an indicator associated with the fault, for example, the packet loss rate, and the number of implicit correlation indicators can be one or more. By analyzing the historical operation data, the indicators that the system is easily affected by when a fault occurs are determined, and they are used as implicit correlation indicators to realize the automatic determination of implicit correlation indicators. When testing abnormal scenarios, the implicit correlation indicators are automatically focused on. According to the test results, the test data corresponding to the implicit correlation indicators of the system to be tested are determined, and by analyzing the test data, it is determined whether it meets the requirements, and the test results are obtained to realize the abnormal detection of the implicit correlation indicators. When analyzing the test data corresponding to the implicit correlation indicators, the control standard can be a pre-set expected value, or it can be the data during normal operation determined based on historical operation data. In addition to performing abnormal detection on implicit correlation indicators, the present application can also allow users to specify indicators that need to be focused on, and perform abnormal detection on these indicators.

[0033] An embodiment of the present invention provides an abnormal scenario testing method, which obtains a system to be tested, wherein the system to be tested includes at least one module to be tested; determines fault information corresponding to each module to be tested and at least one server to be tested; tests each server to be tested according to test task information and fault information to determine a test result; performs abnormality detection on implicit correlation indicators of the system to be tested according to the test result to determine the detection result, wherein the implicit correlation indicator is determined according to historical operation data, which solves the problem that observation indicators can only be determined according to user selection during abnormal scenario testing. The server to be tested is tested according to test task information and fault information to determine the test result, and the implicit correlation indicator of the system to be tested is detected according to the test result. The implicit correlation indicator is determined by analyzing the historical operation data, and the implicit correlation indicator is automatically selected, thereby realizing automated abnormality detection. The user does not need to select the observation indicator, and the problem of improper selection of observation indicators due to the user's unfamiliarity with the system to be tested is avoided. The observation indicator is reasonably selected, the system abnormality is discovered in time, and the efficiency of abnormal scenario testing is improved.

[0034] Example 2

[0035] Figure 2 This is a flow chart of an abnormal scenario testing method provided by the second embodiment of the present invention. This embodiment is refined based on the above embodiment. Figure 2 As shown, the method includes:

[0036] S201: Receive query conditions input by a user.

[0037] In this embodiment, the query conditions can be specifically understood as information used to filter systems or modules. The query conditions can be system names, module names, etc. The module name can be used to directly locate a specific module in the system, and the system name can be used to locate the system. Since the system includes one or more modules, the modules it locates are all modules in the system. By setting a search box, the user can enter the query conditions in the search box to query and select the system or module that needs to be tested for abnormal scenarios. The query conditions can be information such as the system name that the user freely enters, or different query conditions can be set and displayed to the user. The user selects the query conditions by single-clicking, double-clicking, sliding, etc., and automatically enters them into the search box.

[0038] S202: Determine a candidate system and at least one candidate module in the candidate system according to the query condition.

[0039] In this embodiment, the candidate system can be specifically understood as a system that matches the query condition; the candidate module can be specifically understood as a module in the system that matches the query condition.

[0040] The system is screened by query conditions, and the alternative systems that match the query conditions are determined, and at least one alternative module in the alternative system is determined. The query condition entered by the user can be one or multiple query conditions entered in batches. For example, the query condition entered by the user first is the system name, and the alternative systems that match the system name are determined. The modules in the alternative system are displayed, and the user can select the alternative module by entering the query condition again. Alternatively, if the query condition entered by the user is the module name, the module that matches the module name is the alternative module, and the system where the alternative module is located is the alternative system. The present application can also display the alternative system or alternative module according to the query conditions, for example, by displaying it in a list, and the displayed content can be basic information such as system number, module number, system name, system English name, module name, module English name, development department, testing department, etc. It is also possible to display the abnormal scenario test status recently performed by the currently selected alternative system or alternative module by means of charts and the like.

[0041] S203: If it is determined according to the user ID that the user has the authority over the candidate system and each candidate module, the candidate system is determined as the system to be tested, and the candidate module is determined as the module to be tested.

[0042] In the present embodiment, the user identification is used to identify the identity of the user, which can be a name, ID, etc. The user identifications of different users are different, and a unique user can be locked by the user identification. The user's operating authority is determined according to the user's identification, and whether the user has the authority of the alternative system and each alternative module is determined according to the user's identification. If the user has the authority, the alternative system is determined as the system to be tested, and the alternative module is determined as the module to be tested. Authority can be divided according to department. For example, the alternative system can be operated by the user of the test department. Therefore, when the user is determined to be an employee of the test department according to the user identification, it is determined that the user has the authority of the alternative system. Authority can also be directly divided according to the user's accuracy. For example, user A of the test department has the authority of the alternative system, and user B does not have the authority of the alternative system.

[0043] S204: For each module to be tested, determine the fault parameter type and display it.

[0044] In this embodiment, the fault parameter type can be specifically understood as the parameter type used to create the fault. Fault parameter types can include fault name, fault start time, fault duration, and key fault parameters. Key fault parameters can include CPU load ratio, network packet loss ratio, network delay duration, and so on. For each module to be tested, a fault parameter type can be pre-set. The number of fault parameter types can be one or more, and different fault parameter types can correspond to different modules to be tested. Each fault parameter type is displayed in a list, chart, or other format.

[0045] S205: Receive fault parameters input by the user for each fault parameter type.

[0046] In this embodiment, fault parameters can be understood as specific parameter value information, for example, fault duration: 20 minutes. For each fault parameter type, the user can select the fault parameter type based on actual needs and set the corresponding fault parameter by manually entering the parameter, clicking on pre-set options, or providing options. For each fault parameter type, the user can select and set the corresponding fault parameter, or choose not to select or set the fault parameter for that type. Important fault parameter types can also be set as required to prevent users from missing them.

[0047] S206: Generate fault information according to each fault parameter.

[0048] Fault information is generated based on each fault parameter, and the fault information is used to create a fault. The fault parameters can be used to generate fault information in a pre-set format to distinguish fault parameter types corresponding to different fault parameters, or the fault information may include the fault parameter type.

[0049] S207: For each module to be tested, determine the environment type according to the module to be tested, and display the candidate environments in the environment type.

[0050] In this embodiment, the environment type can be a development environment, a test environment, a quasi-production environment, a production environment, etc., and each environment type can include one or more environments. The alternative environment can be specifically understood as a specific environment in the environment type. The environment type corresponding to each module in the system can be pre-set, or the environment type can be displayed to the user, and the user can select it, and the environment type of the module to be tested is determined based on the user's operation. After determining the environment type, the alternative environment in the environment type is displayed so that the user can select the environment for specific testing. The specific environment information in the alternative environment can be system architecture, server type and quantity, hardware configuration, and software installation status, etc.

[0051] S208: Determine the target environment according to the user's selection operation.

[0052] In this embodiment, the target environment can be specifically understood as specific environment information for abnormal scenario testing. After the alternative environments are displayed to the user, the user selects the target environment from the alternative environments through a selection operation so as to perform testing according to the target environment.

[0053] S209: Determine the server type and quantity according to the target environment.

[0054] After the user determines the target environment, the type and number of servers included in the target environment are determined accordingly.

[0055] S210: Screen servers according to server type and quantity to determine at least one server to be tested.

[0056] The number of servers can be one or more, and the server types can be one or more. When selecting servers to test, you can choose to inject faults into all servers in the test environment, or you can specify a specific server, for example, by entering its IP address. Select matching servers based on server type, and select a certain number of servers based on quantity, to obtain at least one server to test.

[0057] There is no strict order in determining the server to be tested and the fault information in this application. This application takes parallel execution as an example, that is, S204-S206 and S207-S210 are executed in parallel.

[0058] S211 . Test each server to be tested according to the test task information and the fault information, and determine the test result.

[0059] When injecting faults into the server to be tested, the target environment can also be used to build the operating environment of the server to be tested. The fault information of this application can also be displayed through a fault list, and fault tasks can be generated based on the fault information so that when performing fault injection, operations such as pause and delete can be performed to stop fault injection.

[0060] S212: For each implicit correlation indicator, determine the normal operation result corresponding to the implicit correlation indicator according to historical operation data.

[0061] By analyzing historical operating data, determine the normal operating results of the implicitly correlated indicators when the system under test is operating normally. For example, if the implicitly correlated indicator is response time, the normal operating result can be determined by taking the average of the historical operating data. The historical operating data in this step uses data from when the system is operating normally to avoid errors caused by fault information. The historical operating data can be data from a period of time, such as one day, one week, or one month.

[0062] S213. Determine the operation result corresponding to the implicit correlation indicator according to the test result.

[0063] Filter the corresponding operation results from the test results based on the type of implicit correlation indicator. For example, if the test results include TPS, response time, and transaction success rate, and the implicit correlation indicator is response time, filter the response time from the test results as the operation result of the implicit correlation indicator. Because the test may last for a long time, a large number of test results may be generated for the same type of data during the test process. Therefore, the operation results in this application can be one or more.

[0064] S214: Compare the operation result with the normal operation result to determine the test result.

[0065] The run result is compared with the normal run result. If the discrepancy is significant, the test result is abnormal; otherwise, the test result is normal. For example, if the implicit correlation metric is TPS (transactions per second) greater than 10,000, the test result is normal. An abnormal test result indicates that the module under test in the system under test is abnormal. The cause of the abnormality can be hardware or software anomalies involved in the system. After confirming the abnormality, staff can promptly conduct troubleshooting and optimize the system.

[0066] The normal operation results are determined by historical operation data, without the need for user-specified results, thus reducing test errors.

[0067] As an optional embodiment of this embodiment, this optional embodiment is further optimized to include:

[0068] A1. Compare historical operation data with test results to determine candidate correlation indicators.

[0069] In this embodiment, candidate correlation indicators can be specifically understood as indicators that can be used for user observation. Historical operating data is compared with test results to identify data types that have significantly changed relative to the historical operating data. For example, a threshold is set to determine data changes. Data types with significantly changed data are more likely to be affected by the fault and are selected as candidate correlation indicators.

[0070] A2. When the candidate correlation indicator meets the preset conditions, the candidate correlation indicator is determined as an implicit correlation indicator, and the implicit correlation indicator of the system to be tested is updated.

[0071] In this embodiment, the preset condition can be set based on the time, number of times, etc., when the candidate association indicator appears. For example, after each test, the candidate association indicator is determined and the number of times the candidate association indicator appears is recorded. The number is accumulated by 1 after each appearance. When the number is greater than a certain threshold, it is determined that the candidate association indicator meets the preset condition. The candidate association indicator that meets the preset condition is determined as an implicit association indicator, and is used as a new implicit association indicator to update the implicit association indicator of the system to be tested. If the original implicit association indicator of the system to be tested does not contain the new implicit association indicator, the association relationship between the new implicit association indicator and the system to be tested is directly established. If the original implicit association indicator of the system to be tested contains the new implicit association indicator, the association relationship remains unchanged and there is no need to establish the association relationship again.

[0072] The implicit correlation indicators in this application can also be analyzed and determined through machine learning to achieve automated anomaly detection, making anomaly detection more comprehensive.

[0073] As an optional embodiment of this embodiment, this optional embodiment is further optimized to include:

[0074] B1. Determine at least one observation indicator and corresponding expected data for each module to be tested.

[0075] In this embodiment, the observation indicators can be specifically understood as indicators of concern during the abnormal scenario testing process. The observation indicators can be selected automatically or set by the user. The observation indicators corresponding to the module to be tested are determined based on the correspondence between the module to be tested and each observation indicator. Alternatively, for each module to be tested, the observation indicators are displayed to the user, and the user selects the observation indicators that need to be paid attention to. For each observation indicator, the corresponding expected data is set. The expected data can be a specific value or a range of values. The expected data can be manually entered by the user or determined by parsing the test document.

[0076] B2. Generate a test report based on the test results and expected data.

[0077] In the test results, determine the corresponding operating results for each observed indicator and write the operating results and expected data into the test report so that the test results can be displayed according to the test report. The operating data can also be displayed in the form of charts. You can also compare the operating results with the corresponding expected data to determine the results of anomaly detection and write them into the test report.

[0078] This application can also pre-set monitoring and management indicators, such as basic server operation indicators (such as CPU, memory, disk) and transaction operation indicators (such as TPS, response time, success rate), and display them in graphical form. They can also be written into the test report to facilitate fault analysis. The detection results of implicitly associated indicators can also be written into the test report and displayed in graphical form.

[0079] As an optional embodiment of this embodiment, this optional embodiment is further optimized to include: after determining that the system to be tested is abnormal based on the detection results or test report and repairing the defects of the system to be tested, receiving the defect resolution strategy uploaded by the user and storing the defect resolution strategy in the expert database.

[0080] In this embodiment, the expert database can be specifically understood as a storage space for storing data, which can be a database or a local storage space. The defect resolution strategy can be specifically understood as a solution proposed for system failures. When the test results or test report determine that the system to be tested is abnormal, the staff will perform defect repair on the system to be tested. After the defect is repaired, the above-mentioned fault injection can be repeated to determine whether the system failure has disappeared. If the system failure has disappeared, it can be determined that the defect resolution strategy is reasonable. The user uploads the defect resolution strategy, receives the defect resolution strategy uploaded by the user and stores it in the expert database, and provides a defect resolution solution. When storing the defect resolution strategy, it can be stored in correspondence with the defect, or a description of the defect can be added to facilitate query or use by other users.

[0081] The expert library in the embodiment of the present application can also store the full-process best practices of various abnormal scenario tests, including abnormal scenario design, observation indicator selection, test report analysis, etc. Other users can refer to and reuse them according to their own needs.

[0082] As an optional embodiment of this embodiment, this optional embodiment is further optimized to include:

[0083] C1. Determine the stop-loss indicator and filter out the stop-loss data from the test results based on the stop-loss indicator.

[0084] In this embodiment, the stop-loss indicator can be specifically understood as an indicator used to determine whether to issue an alarm or stop the test, for example, the CPU usage rate; and the stop-loss data can be specifically understood as the specific data corresponding to the stop-loss indicator, for example, the CPU usage rate is 80%. The stop-loss indicator is pre-set, and the corresponding type of stop-loss data is filtered from the test results based on the stop-loss indicator.

[0085] It is important to note that test results can be determined as the test progresses, that is, results can be obtained while the test is in progress; for certain types of data, the corresponding data can only be obtained after the test is completed. For such data, test results are obtained after the test is completed.

[0086] C2. When the stop-loss data meets the stop-loss conditions, an alarm message is generated and an alarm prompt is given according to the alarm message.

[0087] In this embodiment, the stop-loss condition can be understood as a pre-set condition used to determine whether the stop-loss data meets the requirements. For example, the stop-loss condition is that the CPU usage is greater than 90%. The alarm information can be understood as information used to issue an alarm, for example, "CPU usage is too high, please stop using the device." The stop-loss data is judged to determine whether the stop-loss condition is met. When the stop-loss condition is met, an alarm message is generated and an alarm prompt is issued. The alarm prompt can be a voice prompt, a text prompt, or other means, and can be sent via email, text message, or other means.

[0088] As an optional embodiment of this embodiment, this optional embodiment is further optimized to include:

[0089] D1. Generate test cases based on the system to be tested, the module to be tested, the fault information and the server to be tested.

[0090] Abnormal scenario testing can be performed based on the system to be tested, the module to be tested, the fault information and the server to be tested. To facilitate repeated testing, this application generates corresponding test cases based on the system to be tested, the module to be tested, the fault information and the server to be tested, which can be used for repeated calls.

[0091] D2. When the trigger conditions of the test case are met, perform automated testing according to the test case.

[0092] In this embodiment, the trigger condition can be either automatic triggering upon reaching a preset time point, or manual triggering upon receiving a user click or call. Different trigger conditions are set for different test cases. When the trigger condition of a test case is determined to be met, the test case is automatically called to perform abnormal scenario testing based on the system to be tested, the module to be tested, the fault information, and the server to be tested specified in the test case.

[0093] The test cases in this application can be stored in the case library. For the tested test cases, not all of them need to be stored. Commonly used and typical test cases can be selected for storage. That is, when the test case meets the conditions, the test case is stored, and when the trigger conditions are met, automated testing is performed according to the test case.

[0094] This application can also provide a test history query function, which can view all abnormal scenario test information and corresponding test reports performed by the current user, and can perform chart statistics on the test history from the time dimension and system dimension.

[0095] The embodiment of the present invention provides an abnormal scenario testing method, which solves the problem that observation indicators can only be determined based on user selection during abnormal scenario testing. The test task information and fault information are used to test the server to be tested, and the test results are determined. Abnormality detection is performed on the implicit correlation indicators of the system to be tested based on the test results. The implicit correlation indicators are determined by analyzing historical operation data, and the implicit correlation indicators are automatically selected, thereby realizing automated abnormality detection. The user does not need to select observation indicators, and the problem of improper selection of observation indicators due to the user's unfamiliarity with the test system is avoided. The observation indicators are reasonably selected, system abnormalities are discovered in a timely manner, and the efficiency of abnormal scenario testing is improved. By forming a test case and automatically calling it when the trigger condition is met, the test case is automatically executed.

[0096] Example 3

[0097] Figure 3 A schematic diagram of the structure of an electronic device 30 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0098] like Figure 3As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc., which is communicatively connected to the at least one processor 31. The memory stores a computer program that can be executed by the at least one processor. The processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 32 or the computer program loaded from the storage unit 38 into the random access memory (RAM) 33. Various programs and data required for the operation of the electronic device 30 can also be stored in the RAM 33. The processor 31, ROM 32, and RAM 33 are connected to each other via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0099] Multiple components in the electronic device 30 are connected to the I / O interface 35, including an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a magnetic disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0100] The processor 31 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 31 executes the various methods and processes described above, such as the abnormal scenario testing method.

[0101] In some embodiments, the abnormal scenario testing method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 38. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 30 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the abnormal scenario testing method described above can be performed. Alternatively, in other embodiments, the processor 31 can be configured to perform the abnormal scenario testing method in any other appropriate manner (e.g., by means of firmware).

[0102] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0104] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0106] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0107] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0108] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0109] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. An abnormal scenario testing method, characterized in that: include: Acquire a system to be tested, where the system to be tested includes at least one module to be tested; Determining fault information corresponding to each of the modules to be tested and at least one server to be tested; Injecting the fault information into the test server, executing the test task by running the test task information, implementing the test on the server to be tested, and obtaining the test result; Performing anomaly detection on implicit correlation indicators of the system to be tested according to the test results to determine the detection results, wherein the implicit correlation indicators are determined based on historical operation data; The step of determining at least one server to be tested corresponding to each module to be tested includes: For each module to be tested, determine the environment type according to the module to be tested, and display the alternative environments in the environment type; Determine the target environment based on the user's selection operation; Determine the server type and quantity based on the target environment; Screening the servers according to the server types and quantities to determine at least one server to be tested; The step of determining the fault information corresponding to each module to be tested includes: For each module to be tested, determine the fault parameter type and display it; receiving fault parameters input by the user for each of the fault parameter types; generating fault information according to each of the fault parameters; The step of performing abnormality detection on the implicit correlation indicators of the system to be tested according to the test results to determine the detection results includes: For each implicit correlation indicator, determine the normal operation result corresponding to the implicit correlation indicator based on historical operation data; Determine the operating result corresponding to the implicit correlation indicator according to the test result; The operation result is compared with the normal operation result to determine the detection result.

2. The method according to claim 1, characterized in that The obtaining of the system to be tested includes: Receive query conditions entered by the user; Determine an alternative system and at least one alternative module in the alternative system according to the query condition; If it is determined according to the user identifier of the user that the user has the authority over the candidate system and each of the candidate modules, the candidate system is determined as the system to be tested, and the candidate modules are determined as modules to be tested.

3. The method according to claim 1, characterized in that Also includes: Compare historical operating data with test results to determine candidate correlation indicators; When the candidate correlation indicator meets a preset condition, the candidate correlation indicator is determined as an implicit correlation indicator, and the implicit correlation indicator of the system to be tested is updated.

4. The method according to claim 1, wherein Also includes: Determining at least one observation indicator and corresponding expected data for each of the modules to be tested; A test report is generated based on the test results and expected data.

5. The method according to any one of claims 1 to 4, characterized in that Also includes: After determining that the system to be tested is abnormal according to the detection result or the test report and performing defect repair on the system to be tested, a defect resolution strategy uploaded by a user is received and the defect resolution strategy is stored in an expert database.

6. The method according to claim 1, characterized in that Also includes: determining a stop-loss indicator, and filtering stop-loss data from the test results according to the stop-loss indicator; When the stop-loss data meets the stop-loss condition, an alarm message is generated and an alarm prompt is given according to the alarm message.

7. The method according to claim 1, characterized in that Also includes: Generate test cases based on the system to be tested, the module to be tested, the fault information and the server to be tested; When the triggering condition of the test case is met, automated testing is performed according to the test case.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the abnormal scenario testing method according to any one of claims 1 to 7.

Images