Automatic judgment method and system based on chaos experiment fault reproduction and storage medium
By using automated evaluation methods based on chaos experiments in a cloud-native environment, the problem of low efficiency of traditional fault reproduction evaluation is solved, and more efficient fault reproduction evaluation and lower average fault repair time are achieved.
Patent Information
- Application Number
- CN202510222461.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-27
- Publication Date
- 2025-06-10
AI Technical Summary
In the cloud-native environment, in the process of reproducing production failures of securities business systems, the traditional method relies on the experience of operation and maintenance personnel to determine the efficiency is low, resulting in a high average fault repair time MTTR.
An automated evaluation method based on chaos experiment is adopted, by obtaining production fault indicator data, simulating traffic injection and configuring observation indicators, performing chaos experiments, calculating index fit, and determining whether the fault recursive is successful.
It improves the accuracy and efficiency of fault reproducing judgment, reduces the average fault repair time MTTR, and enhances the objectivity and accuracy of judgment.
Smart Images
Figure CN120123211A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of chaos engineering, and in particular, to an automated evaluation method, system, and storage medium for fault reproduction based on chaos experiments. Background Art
[0002] With the booming rise of the Chinese securities trading market, the number of market investors and trading scale have been continuously expanding, and the pace of business innovation and fintech innovation has been accelerating day by day. This has put forward more stringent requirements for aspects such as the business response speed, concurrent processing efficiency, and disaster recovery ability of the core trading system. As the core trading system gradually migrates to the cloud-native architecture, the technology stack has become increasingly complex. After the production deployment of the core trading system in the cloud-native environment, the risks faced have also increased. Especially in the context of the continuous expansion of system complexity and scale, the occurrence frequency and impact scope of production failures have also increased significantly, which is meaningful for investigating the root causes of production failures; After a production failure occurs, large-scale fault reproduction experiments are usually required to investigate the root cause of the failure. Therefore, how to automatically determine whether the fault has been reproduced, so as to reduce the mean time to repair (MTTR), has become a typical problem. At present, for the investigation of production failures in the securities business in the cloud-native environment, fault reproduction is the key work in the investigation process. The traditional method is based on the experience of operation and maintenance personnel to inject a large number of simulated faults, and manually determine whether the fault has been reproduced by comparing relevant indicators. The main problems are: first, the determination result has a large error and is highly dependent on personnel experience and ability; second, there is no automation ability, the determination efficiency is low, resulting in a high MTTR. Summary of the Invention
[0003] The purpose of the present invention is to provide an automated evaluation method, system, and storage medium for fault reproduction based on chaos experiments, which can automatically evaluate whether the fault reproduction is successful through index fitting degree analysis, improve the accuracy and efficiency of reproduction evaluation, enhance the fault reproduction efficiency, and provide necessary support for effectively reducing the MTTR.
[0004] To achieve the above object, the present invention is implemented by the following technical solutions: In a first aspect, the present invention provides an automated evaluation method for fault reproduction based on chaos experiments, including: Obtain the production fault index data of the securities business system to be reproduced; Based on the production fault index data, inject simulated traffic into the securities business system to be reproduced in the reproduction environment, and configure observation indicators; Inject simulated production faults into the securities business system to be reproduced through the chaos platform, so as to execute a chaos experiment and obtain reproduction index data; Calculate the index fitting degree between the reproduced fault index data and the production fault index data to obtain the fitting degree of fault reproduction; Determine whether the chaos experiment for the securities business system to be reproduced is successfully reproduced according to the fitting threshold and the fitting degree of fault reproduction.
[0005] Optionally, obtaining the production fault index data of the securities business system to be reproduced includes: Determine the time period when the production fault of the securities business system to be reproduced occurs; Use the cloud native monitoring and application performance management system to obtain the index data during the time period when the production fault occurs, that is, the production fault index data of the securities business system to be reproduced.
[0006] Optionally, injecting simulated traffic into the securities business system to be reproduced includes: Determine a traffic generation tool according to the characteristics of the production fault index data of the securities business system to be reproduced; the traffic generation tool includes a traffic stress testing platform or a traffic simulation tool; Generate simulated traffic based on the traffic generation tool and inject the simulated traffic into the securities business system to be reproduced.
[0007] Optionally, the observation indicators are configured according to the production fault index data of the securities business system to be reproduced, including: general performance indicators and securities business indicators; the general performance indicators include five indicator types, namely CPU type, memory type, disk type and network type; the securities business indicators include: asset transfer TPS, asset transfer RT, asset transfer success rate, order placement TPS, order placement RT, pending order volume and cancellation volume.
[0008] Optionally, injecting simulated production faults into the securities business system to be reproduced through the chaos platform, so as to execute a chaos experiment and obtain reproduction index data, includes: Analyze the production fault index data of the securities business system to be reproduced to determine the simulated production fault; Use the chaos engineering tool to select the atomic fault types that match the simulated production fault and configure the corresponding fault parameters, so as to create a chaos experiment; Execute the chaos experiment in the reproduction environment to inject the simulated production fault; After the injection is completed, obtain the reproduction index data with the help of the observation tool.
[0009] Optionally, the calculation of the index fitting degree between the reproduced fault index data and the production fault index data to obtain the fitting degree of fault reproduction includes: Perform embedding processing on the reproduced fault index data and the production fault index data to obtain reproduced fault index vector data and production fault index vector data; Calculate the similarity between the reproduced fault index vector data and the production fault index vector data to obtain the fitting degree of fault reproduction; wherein, if there are multiple vectors in the reproduced fault index vector data and the production fault index vector data, calculate the similarity of the vectors in the reproduced fault index vector data and the production fault index vector data one by one, and then calculate the average value to obtain the fitting degree of fault reproduction.
[0010] Optionally, the similarity calculation uses any one of the calculation methods of cosine similarity, Pearson correlation coefficient, Euclidean distance, and Jaccard distance.
[0011] Optionally, determining whether the chaos experiment for the securities business system to be reproduced is successfully reproduced according to the fitting threshold and the fitting degree of fault reproduction includes: Obtain the fitting threshold; If the fitting degree of fault reproduction is greater than the fitting threshold, the chaos experiment for the securities business system to be reproduced is successfully reproduced; If the fitting degree of fault reproduction is not greater than the fitting threshold, the chaos experiment for the securities business system to be reproduced fails.
[0012] In a second aspect, the present invention provides an automated evaluation system for fault reproduction based on chaos experiments, including: An acquisition module for acquiring production fault index data of the securities business system to be reproduced; A traffic injection and index configuration module for injecting simulated traffic into the securities business system to be reproduced in the reproduction environment based on the production fault index data and configuring observation indexes; A fault injection module for injecting simulated production faults into the securities business system to be reproduced through a chaos platform, thereby performing a chaos experiment to obtain reproduced index data; A fitting degree calculation module for calculating the index fitting degree between the reproduced fault index data and the production fault index data to obtain the fitting degree of fault reproduction; A reproduction evaluation module for determining whether the chaos experiment for the securities business system to be reproduced is successfully reproduced according to the fitting threshold and the fitting degree of fault reproduction.
[0013] In a third aspect, the present invention provides a computer-readable storage medium storing a computer program, and when the computer program is executed, it implements the automated evaluation method for fault reproduction based on chaos experiments described in the first aspect of the claims.
[0014] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention provides an automated evaluation method, system and storage medium for fault reproduction based on chaos experiments. The method is based on chaos engineering experiments, and combines technologies such as traffic stress testing and index observation. In the reproduction environment, simulated traffic injection is carried out, observation indexes are configured, and at the same time, simulated fault injection is executed to conduct chaos experiments to obtain reproduction fault index data; then, using the index fitting degree analysis algorithm, the fitting degree between the production fault index data and the reproduction fault index data is determined; finally, whether the chaos experiment fault reproduction is successful is judged through the fitting threshold and the fitting degree result; this method can automatically evaluate whether the fault reproduction is successful, effectively reduce the misjudgment caused by human factors before, and is more objective and accurate; this method can also improve the reproduction evaluation accuracy and efficiency, improve the fault reproduction efficiency, and provide necessary support for effectively reducing the mean time to repair MTTR. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 The flowchart of an automated evaluation method for fault reproduction based on chaos experiments in an embodiment of the present invention is shown; Figure 2 The curve diagram of the production fault index data and the reproduction fault index data for the observation index "Pod memory usage" in an embodiment of the present invention is shown; Figure 3 The curve diagram of the production fault index data and the reproduction fault index data for the observation index "Pod CPU usage" in an embodiment of the present invention is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0016] The present invention will be further described below with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and cannot be used to limit the protection scope of the present invention.
[0017] Embodiment 1
[0018] As Figure 1 shown, an embodiment of the present invention introduces an automated evaluation method for fault reproduction based on chaos experiments, including the following steps: S1: Obtain the production fault index data of the securities business system to be reproduced; S2: Based on the production fault index data, inject simulated traffic into the securities business system to be reproduced in the reproduction environment and configure observation indexes; S3: Inject simulated production faults into the securities business system to be reproduced through a chaos platform, thereby performing chaos experiments to obtain reproduction index data; S4: Calculate the index fitness between the reproduced fault index data and the production fault index data to obtain the fitness of fault reproduction. S5: Determine whether the chaos experiment for the securities business system to be reproduced is successful based on the fitness threshold and the fitness of fault reproduction.
[0019] Specifically, before conducting the above reproduction experiment, that is, the chaos experiment, it is necessary to first prepare a reproduction environment similar to the production environment. For the cloud-native application system, the reproduction environment is usually the corresponding kubernetes cluster and the application's namespace. Among them, namespace is a resource isolation mechanism in Kubernetes, which divides the resources in the cluster into multiple logical parts, and each part has its own resource set and isolation environment.
[0020] An automated evaluation method for fault reproduction based on chaos experiment introduced in this embodiment can analyze through index fitness, thereby automatically evaluating whether the fault reproduction is successful, improving the accuracy and efficiency of reproduction evaluation, enhancing the fault reproduction efficiency, and providing necessary support for effectively reducing the mean time to repair (MTTR).
[0021] In this embodiment, in step S1, obtaining the production fault index data of the securities business system to be reproduced specifically includes: Determine the time period when the production fault of the securities business system to be reproduced occurs. Specifically, first discover the fault through user feedback, system alarms or abnormal notifications of the monitoring system, and initially locate the securities business system that may be affected according to the fault phenomenon (such as transaction failure, response timeout, etc.); then query the abnormal start time and recovery time of the production fault through the monitoring data to obtain the time period when the production fault occurs; in addition, when confirming the time period when the production fault occurs, multi-source data comparison can be further combined to verify the accuracy of the time range, and the time range can also be appropriately extended to ensure data integrity.
[0022] Use cloud-native monitoring and application performance management systems to obtain the index data for the time period when the production fault occurs, that is, the production fault index data of the securities business system to be reproduced.
[0023] Specifically, cloud-native monitoring uses Prometheus or Grafana, and the application performance management system (APM system) uses SkyWalking, Pinpoint, New Relic, Dynatrace, etc.; the process of obtaining index data includes: Input the time period of the determined production failure, select the metrics (performance metrics / business metrics) related to the production failure, and finally obtain the metric data through the API query interface; among them, the metrics related to the production failure are the same as the configured observed metrics, which will not be repeated here.
[0024] In this embodiment, in step S2, based on the production failure metric data, simulated traffic is injected into the securities business system to be reproduced in the reproduction environment, specifically including: Determine the traffic generation tool according to the characteristics of the production failure metric data of the securities business system to be reproduced; the traffic generation tool includes a traffic stress testing platform or a traffic simulation tool; Generate simulated traffic based on the traffic generation tool and inject the simulated traffic into the securities business system to be reproduced, so as to reproduce the traffic when the production failure occurs and achieve the effect of restoring the traffic scene; provide support for subsequent observation and analysis of the accuracy of performance metrics and business metrics; Specifically, analyze the production failure metric data of the securities business system to be reproduced to determine the traffic characteristics at the time of failure; select a suitable traffic generation tool according to the traffic characteristics; for example, when simulating scenarios with high concurrency and large traffic, use traffic stress testing platforms such as JMeter, LoadRunner, Gatling, etc. for traffic generation; when simulating specific types of transaction traffic, use traffic simulation tools such as Tsung, Locust, k6, etc. for traffic generation; Specifically, for the heterogeneous protocols of securities trading, through the traffic injection tool for heterogeneous transaction protocols, it is possible to support the injection of traffic for Http protocol, Dubbo protocol, and QStep protocol. In this embodiment, we use the traffic injection tool for heterogeneous transaction protocols to inject traffic for the asset query interface into the reproduction environment. At the same time, in the cloud-native environment, the traffic injection tool for heterogeneous transaction protocols will create a Job resource object, and the heterogeneous protocol control program inside the Job will implement transaction protocol verification, message conversion, and traffic injection operations in the form of dynamic plugins.
[0025] In this embodiment, in step S2, configuring the observed metrics specifically includes: Determine the observation target according to the production failure metric data of the securities business system to be reproduced; Select a suitable monitoring tool based on the technology stack and requirements of the securities business system to be reproduced; among them, common monitoring tools include: cloud-native monitoring, APM system, log monitoring, custom monitoring, etc.; According to the observation objectives, specific observation indicators are configured, including general performance indicators and securities business indicators; the general performance indicators include five indicator types, namely CPU type, memory type, disk type, and network type; the securities business indicators include: asset transfer TPS (Transactions Per Second, number of transactions per second), asset transfer RT (Response Time, response time), asset transfer success rate, order placement TPS, order placement RT, number of orders to be reported, and number of order cancellations; Among them, the general performance indicators are shown in the following table:
[0026] Among them, the securities business indicators are shown in the following table:
[0027] In this embodiment, step S3 injects simulated production faults into the securities business system to be reproduced through the chaos platform, thereby performing a chaos experiment to obtain reproduced indicator data, specifically including: Analyze the production fault indicator data of the securities business system to be reproduced to determine the simulated production faults, including single fault scenarios such as "Pod CPU usage" and "Pod memory usage", or a combination of multiple fault scenarios injected into the test environment; Use the chaos engineering tool chaosblade to select the atomic fault types that match the simulated production faults and configure the corresponding fault parameters, thereby creating a chaos experiment; Execute the chaos experiment in the reproduction environment to inject the simulated production faults; After the injection is completed, obtain the reproduced indicator data with the help of the observation tool.
[0028] In this embodiment, step S4 calculates the index fitting degree between the reproduced fault indicator data and the production fault indicator data to obtain the fitting degree of fault reproduction, including: Perform embedding processing (embedding) on the reproduced fault indicator data and the production fault indicator data, that is, map high-dimensional data (such as text, pictures, audio) to a low-dimensional space to obtain the reproduced fault indicator vector data and the production fault indicator vector data; Calculate the similarity between the reproduced fault indicator vector data and the production fault indicator vector data to obtain the fitting degree of fault reproduction; among them, if there are multiple vectors in the reproduced fault indicator vector data and the production fault indicator vector data, calculate the similarity of the vectors in the reproduced fault indicator vector data and the production fault indicator vector data one by one, and then calculate the average value to obtain the fitting degree of fault reproduction.
[0029] Specifically, any one of the calculation methods such as cosine similarity, Pearson correlation coefficient, Euclidean distance, and Jaccard distance can be used in similarity calculation; in this embodiment, the cosine similarity algorithm with the widest range of use, the most stability, and relatively simple calculation is adopted; Specifically, as Figure 2 and 3 shown, the "Pod CPU usage" and "Pod memory usage" are used for goodness-of-fit calculation; As Figure 2 shown, for the observed index "Pod CPU usage", vectors A and B are obtained; Among them, vector A represents the production fault index vector data, that is, the vector of the Pod CPU usage displayed at the time points within 2 minutes and 30 seconds during production faults, denoted as vector A = a 1, a 2,…, an ; where n represents the total number of time points within 2 minutes and 30 seconds; Vector B represents the reproduced fault index vector data, that is, the vector of the Pod CPU usage displayed at the time points within 2 minutes and 30 seconds during reproduced faults, denoted as vector B = b 1, b 2,…, bn ; According to the cosine similarity calculation formula:
[0030] Among them, and are the norms of vectors A and B respectively; is the goodness-of-fit for the "Pod CPU usage" index; According to the above method and Figure 3 shown content, the goodness-of-fit for the "Pod memory usage" index can be obtained ; Taking the average of and , the goodness-of-fit for fault reproduction can be obtained as 96.3%.
[0031] In this embodiment, step s5 determines whether the chaos experiment for the securities business system to be reproduced is successful according to the goodness-of-fit threshold and the goodness-of-fit for fault reproduction, including: Obtain the goodness-of-fit threshold; in this embodiment, the goodness-of-fit threshold is set to 80%; If the goodness-of-fit for fault reproduction is greater than the goodness-of-fit threshold, the chaos experiment for the securities business system to be reproduced is successful; If the fitness of the fault reproduction is not greater than the fitness threshold, the chaotic experiment reproduction of the securities business system to be reproduced fails.
[0032] Specifically, in this embodiment, the fitness threshold is set to 80%. Since the calculated fitness of the fault reproduction is 96.3% which is greater than the fitness threshold, it is determined that the chaotic experiment reproduction is successful.
[0033] Embodiment 2
[0034] This embodiment provides an automated evaluation system for fault reproduction based on chaotic experiments, including: An acquisition module, configured to acquire production fault index data of the securities business system to be reproduced; A traffic injection and index configuration module, configured to inject simulated traffic into the securities business system to be reproduced in a reproduction environment based on the production fault index data, and configure observation indexes; A fault injection module, configured to inject simulated production faults into the securities business system to be reproduced through a chaos platform, so as to perform a chaotic experiment and obtain reproduction index data; A fitness calculation module, configured to calculate the index fitness of the reproduced fault index data and the production fault index data to obtain the fitness of the fault reproduction; A reproduction evaluation module, configured to determine whether the chaotic experiment for the securities business system to be reproduced is successfully reproduced according to the fitness threshold and the fitness of the fault reproduction.
[0035] Embodiment 3
[0036] This embodiment provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed, it implements the automated evaluation method for fault reproduction based on chaotic experiments described in Claim Embodiment 1.
[0037] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0038] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.
[0039] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.
[0040] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit and scope protected by the present invention and the claims. All of these fall within the protection scope of the present invention.
Claims
1. An automated evaluation method based on chaos experiment fault reproduction, characterized in that: include: Obtain production fault indicator data of the securities business system to be reproduced; Based on the production fault indicator data, simulated traffic is injected into the securities business system to be reproduced in the reproduction environment, and observation indicators are configured; Through the chaos platform, simulated production failures are injected into the securities business system to be reproduced, so as to perform chaos experiments and obtain reproduction indicator data; The index fitting degree is calculated for the reproduction fault index data and the production fault index data to obtain the fitting degree of the fault reproduction; Whether the chaos experiment on the securities business system to be reproduced is successfully reproduced is determined based on the fitting threshold and the fitting degree of the fault reproduction.
2. The automatic evaluation method based on chaos experiment fault reproduction according to claim 1 is characterized in that: The obtaining of production fault indicator data of the securities business system to be reproduced includes: Determine the time period during which the production failure of the securities business system to be reproduced occurred; By using the cloud-native monitoring and application performance management system, the indicator data of the time period in which the production failure occurred is obtained, that is, the production failure indicator data of the securities business system to be reproduced.
3. The automatic evaluation method based on chaos experiment fault reproduction according to claim 2 is characterized in that: The injecting of simulated traffic into the securities business system to be reproduced includes: Determine a traffic generation tool according to the characteristics of the production fault indicator data of the securities business system to be reproduced; the traffic generation tool includes a traffic stress testing platform or a traffic simulation tool; Generate simulated traffic based on the traffic generation tool, and inject the simulated traffic into the securities business system to be reproduced.
4. The automatic evaluation method based on chaos experiment fault reproduction according to claim 3 is characterized in that: The observation indicators are configured according to the production fault indicator data of the securities business system to be reproduced, including: general performance indicators and securities business indicators; the general performance indicators include five indicator types, namely CPU, memory, disk and network; the securities business indicators include: asset transfer TPS, asset transfer RT, asset transfer success rate, order TPS, order RT, number of pending orders and number of cancelled orders.
5. The automatic evaluation method based on chaos experiment fault reproduction according to claim 4 is characterized in that: The chaos platform is used to inject simulated production failures into the securities business system to be reproduced, thereby performing a chaos experiment and obtaining reproduction indicator data, including: Analyze the production failure indicator data of the securities business system to be reproduced and determine the simulated production failure; Use chaos engineering tools to select atomic fault types that match simulated production faults and configure corresponding fault parameters to create chaos experiments. Perform chaos experiments in a reproducible environment to inject simulated production failures; After the injection is completed, the reproducible indicator data is obtained with the help of observation tools.
6. The automatic evaluation method based on chaos experiment fault reproduction according to claim 5 is characterized in that: The calculation of the index fitting degree of the reproduction fault index data and the production fault index data to obtain the fitting degree of the fault reproduction includes: Embedding the recurrence fault index data and the production fault index data to obtain recurrence fault index vector data and production fault index vector data; The similarity between the reproduced fault indicator vector data and the production fault indicator vector data is calculated to obtain the fit of the fault reproduction; if there are multiple vectors in the reproduced fault indicator vector data and the production fault indicator vector data, the similarity is calculated one by one for the vectors in the reproduced fault indicator vector data and the production fault indicator vector data, and then the average value is calculated to obtain the fit of the fault reproduction.
7. The automatic evaluation method based on chaos experiment fault reproduction according to claim 6 is characterized in that: The similarity calculation adopts any one of the following calculation methods: cosine similarity, Pearson correlation coefficient, Euclidean distance and Jaccard distance.
8. The automatic evaluation method based on chaos experiment fault reproduction according to claim 6 is characterized in that: Whether the chaos experiment for the securities business system to be reproduced is successfully reproduced is determined based on the fitting threshold and the fitting degree of the fault reproduction, including: Get the fitting threshold; If the fitting degree of the fault reproduction is greater than the fitting threshold, the chaos experiment for the securities business system to be reproduced is successfully reproduced; If the fitting degree of the fault reproduction is not greater than the fitting threshold, the chaos experiment reproduction of the securities business system to be reproduced fails.
9. An automated evaluation system based on chaos experiment fault reproduction, characterized in that: include: An acquisition module, used to acquire production fault indicator data of the securities business system to be reproduced; A traffic injection and index configuration module, which is used to inject simulated traffic into the securities business system to be reproduced in a reproduction environment based on the production fault index data, and configure observation indicators; The fault injection module is used to inject simulated production faults into the securities business system to be reproduced through the chaos platform, thereby executing chaos experiments and obtaining reproduction indicator data; A fitting degree calculation module is used to calculate the index fitting degree of the reproduction fault index data and the production fault index data to obtain the fitting degree of the fault reproduction; The reproduction evaluation module is used to determine whether the chaos experiment for the securities business system to be reproduced is successful based on the fitting threshold and the fitting degree of the fault reproduction.
10. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed, implements the automated evaluation method based on chaos experiment fault reproduction as described in any one of claims 1 to 8.