System fault analysis method and device, electronic equipment and computer program product

Abstract

The invention discloses a system fault analysis method and device, electronic equipment and a computer program product. The method relates to the field of artificial intelligence, and comprises the following steps: acquiring a system architecture and historical system data of a target system, and detecting fault data related to the target system from a target knowledge base based on the system architecture and the historical system data; inputting the fault data into a first target model to obtain an experiment model of a chaotic engineering experiment of the target system; injecting a preset fault under the target fault type into the target system based on a fault injection strategy in the experiment model, and collecting a fault experiment index of the target system in which the preset fault is injected; and inputting the fault experiment index into the second target model to obtain a fault analysis result. Through the method and the device, the problem of low fault analysis efficiency of a software system caused by dependence on manual setting and expert experience in a chaos engineering experiment in related technologies is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular, to a system fault analysis method and device, electronic equipment and computer program product. BACKGROUND

[0002] With the rapid development and popularization of information technology, large-scale distributed systems and cloud infrastructures have gradually grown in complexity. This surge in complexity presents a significant challenge, namely, how to ensure that these systems remain highly available, reliable, and resilient in the face of external disturbances and internal defects. In the related art, a method of chaos engineering is used to analyze the faults of a system.

[0003] However, the chaos engineering method in the related art has obvious limitations in experimental design, fault scenario coverage, data analysis efficiency, and real-time response capability. The design of chaos engineering experiments and the fault analysis process are highly dependent on manual setting and expert experience. The selection of fault scenarios, the setting of experimental parameters, and subsequent analysis all require the direct involvement of domain experts. Due to the limitations of manual setting, experiments can only be based on known fault types and expert-pre-set scenarios, which may result in insufficient testing of the system's performance under unknown or rare faults. After the experiment is executed, a large amount of monitoring data, system logs, and performance indicators are generated. Manually analyzing these massive amounts of data is not only time-consuming, but also prone to missing critical information, affecting the accurate assessment of system vulnerability. In a complex and dynamic environment, the speed and accuracy of manual decision-making are difficult to meet the needs of immediate response and rapid optimization, especially in large-scale distributed systems, where a fault can instantly affect multiple components, requiring immediate analysis and countermeasures. The process of manual analysis and decision-making may be influenced by personal experience and bias, resulting in less objective analysis results and optimization suggestions, which may miss the best practices.

[0004] Currently, there is no effective solution to the problem of low efficiency of software system fault analysis caused by the dependence of chaos engineering experiments on manual setting and expert experience in the related art. SUMMARY

[0005] The main purpose of the present application is to provide a system fault analysis method, device, electronic equipment and computer program product to solve the problem of low efficiency of software system fault analysis caused by the dependence of chaos engineering experiments on manual setting and expert experience in the related art.

[0006] In order to achieve the above object, according to one aspect of the present application, a system fault analysis method is provided. The method comprises: acquiring a system architecture and historical system data of a target system, detecting fault data related to the target system from a target knowledge base based on the system architecture and the historical system data; inputting the fault data into a first target model to obtain an experiment model of a chaos engineering experiment of the target system, wherein the experiment model is used for M types of fault scenarios of the target system, and M is a positive integer; injecting a preset fault under a target fault type into the target system based on a fault injection strategy in the experiment model, and collecting fault experiment indicators of the target system after the preset fault is injected, wherein the fault experiment indicators are used to indicate the performance and state of the target system in the fault scenario, and the fault injection strategy comprises the target fault type and fault parameters; inputting the fault experiment indicators into a second target model to obtain a fault analysis result, wherein the fault analysis result is used to indicate an abnormal state and a risk assessment result of the target system in the fault scenario of the preset fault.

[0007] Optionally, the first target model is obtained by training in the following manner: acquiring a training sample set in the field of chaos engineering experiments, wherein the training sample set comprises N training samples, each training sample comprises historical fault data and a historical experiment model corresponding to the historical fault data, N is greater than M, and N is a positive integer; initializing a pre-training model, fine-tuning model parameters of the pre-training model based on the training sample set, and obtaining the trained first target model.

[0008] Optionally, injecting the preset fault under the target fault type into the target system based on the fault injection strategy in the experiment model comprises: determining the target fault type and the fault parameters in the fault injection strategy, wherein the target fault type comprises at least one of the following: network interruption, service downtime, data loss and delay, and the fault parameters comprise at least one of the following: injection time, duration, frequency and influence range on the target system; inputting the target fault type and the fault parameters into a preset chaos engineering tool, and injecting the preset fault into a test environment of the target system through the preset chaos engineering tool.

[0009] Optionally, collecting the fault experiment indicators of the target system after the preset fault is injected comprises: collecting performance indicators, log information and error rates of the target system through a preset monitoring tool, wherein the error rate is a proportion of the number of response errors to the total number of responses of the target system in the process of providing services; and performing data cleaning and normalization processing on the performance indicators, the log information and the error rates to obtain the fault experiment indicators.

[0010] Optionally, the fault experiment index is input into the second target model to obtain a fault analysis result, including: processing the fault experiment index by an abnormal behavior identification module of the second target model to obtain an abnormal behavior identification result of the target system, wherein the abnormal behavior includes at least one of performance degradation, service unavailability and resource exhaustion; identifying a fault point from the fault experiment index by a fault point identification module of the second target model, wherein the fault point is a service of the target system that has an abnormal behavior in the fault scene; processing the fault experiment index by a risk assessment module of the second target model to obtain a risk assessment result, wherein the risk assessment result is used to indicate stability and a predicted fault mode of the target system in the fault scene; and determining at least one of the abnormal behavior identification result, the fault point and the risk assessment result as the fault analysis result.

[0011] Optionally, after obtaining the fault analysis result, the method further includes: extracting the fault point and the abnormal behavior from the fault analysis result, retrieving a target repair strategy for repairing the fault point and the abnormal behavior from the target knowledge base based on the fault point and the abnormal behavior; and executing the target repair strategy on the target system.

[0012] Optionally, the target knowledge base is constructed in the following manner: acquiring system-related data of a plurality of data sources, wherein the plurality of data sources include at least one of system logs, historical fault data, monitoring information, operation manuals and fault repair documents; converting each piece of data into a preset format by a preprocessing manner to obtain preprocessed data; creating an index for each piece of preprocessed data, and storing all data and the index corresponding to each piece of data to a preset database to obtain the target knowledge base.

[0013] To achieve the above object, according to another aspect of the present application, a system fault analysis device is provided. The device includes: an acquisition unit configured to acquire a system architecture and historical system data of a target system, and detect fault data related to the target system from a target knowledge base based on the system architecture and the historical system data; a first input unit configured to input the fault data into a first target model to obtain an experiment model of a chaos engineering experiment of the target system, wherein the experiment model is used to experiment on M types of fault scenes of the target system, and M is a positive integer; an injection unit configured to inject a preset fault under a target fault type into the target system based on a fault injection strategy in the experiment model, and collect fault experiment index of the target system after the preset fault is injected, wherein the fault experiment index is used to indicate performance and state of the target system in the fault scene, and the fault injection strategy includes the target fault type and fault parameters to be injected; and a second input unit configured to input the fault experiment index into a second target model to obtain a fault analysis result, wherein the fault analysis result is used to indicate an abnormal state and a risk assessment result of the target system in the fault scene of the preset fault.

[0014] In the embodiment of the present application, the system architecture and historical system data of the target system are acquired, and the fault data related to the target system is detected from the target knowledge base based on the system architecture and the historical system data; the fault data is input into the first target model to obtain an experiment model of the chaos engineering experiment of the target system, wherein the experiment model is used for M types of fault scenes of the experimental target system, and M is a positive integer; a preset fault under a target fault type is injected into the target system based on a fault injection strategy in the experiment model, and a fault experiment index of the target system after the preset fault is injected is collected, wherein the fault experiment index is used to indicate the performance and state of the target system in the fault scene, and the fault injection strategy includes the target fault type and the fault parameter injected; the fault experiment index is input into the second target model to obtain a fault analysis result, wherein the fault analysis result is used to indicate the abnormal state and the risk assessment result of the target system in the preset fault scene, and the automatic process from fault data retrieval to fault injection strategy generation and then to system performance state index analysis is realized by combining information retrieval, model generation and chaos engineering, the purpose of more accurately identifying system vulnerability and improving the overall stability and robustness of the system is achieved, thereby realizing the technical effect of improving the fault analysis efficiency of the software system, and further solving the technical problem that the chaos engineering experiment relies on manual setting and expert experience, resulting in low fault analysis efficiency of the software system. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which form a part of this application, are intended to provide further understanding of the application and are incorporated herein in their entirety, and the illustrative embodiments thereof and description thereof serve to explain the application. In the drawings: Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a fault analysis method of a system is shown; Figure 2 A flowchart of a fault analysis method of a system according to an embodiment of the present application is shown; Figure 3 A schematic diagram of a fault analysis system according to an embodiment of the present application is shown; Figure 4 A schematic diagram of a fault analysis device of a system according to an embodiment of the present application is shown; Figure 5 A structural block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0016] In order to better understand the technical scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0017] It should be noted that the terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units need not be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0018] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present application are information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary security measures are taken, do not violate public order and good customs, and provide corresponding operation portal for user to choose authorization or refusal. For example, the system and the interface between the related users or institutions provide the corresponding operation portal for the user to choose to agree or refuse the automatic decision result; if the user chooses to refuse, the expert decision process is entered.

[0019] Embodiment 1 According to the embodiments of the present application, a method for analyzing the fault of the system is also provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0020] The method provided by the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing the method for analyzing the fault of the system is shown. As shown inFigure 1 As shown, the computer terminal 10 (or mobile device) can include one or more processors 102 (the processor 102 can include, but not limited to, a processing device such as a Microcontroller Unit (MCU) or a Field-Programmable Gate Array (FPGA), etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it can also include a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which can be included as one of the ports in the BUS (Business)), a network interface, a power supply and / or a camera. Those skilled in the art can understand that, Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or less components than those shown, or have a different configuration from that shown. Figure 1 For example, the computer terminal 10 can include more or less components than those shown, or have a different configuration from that shown. Figure 1 For example, the computer terminal 10 can include more or less components than those shown, or have a different configuration from that shown.

[0021] It should be noted that the one or more processors 102 and / or other data processing circuits described above can be generally referred to herein as "data processing circuits". The data processing circuits can be embodied in whole or in part as software, hardware, firmware or any combination thereof. In addition, the data processing circuits can be a single independent processing module, or any one of the other elements incorporated into the computer terminal 10 (or mobile device) in whole or in part. As referred to in the embodiments of the present application, the data processing circuits serve as a processor control (for example, selection of a variable resistance terminal path connected to an interface).

[0022] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage means corresponding to the fault analysis method of the system in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned fault analysis method of the system. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 can further include a memory remotely disposed with respect to the processor 102, which can be connected to the computer terminal 10 through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0023] The transmission device 106 is configured to receive or send data via a network. The network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module configured to communicate with the Internet wirelessly.

[0024] The display can be a liquid crystal display, for example, a touch screen, which can enable a user to interact with the user interface of the computer terminal 10 (or a mobile device).

[0025] In the above operating environment, the present application provides a system fault analysis method. Figure 2 is a flowchart of the system fault analysis method provided by the embodiments of the present application, as shown in Figure 2 The method comprises the following steps. In step S201, the system architecture and historical system data of the target system are obtained, and the fault data related to the target system is detected from the target knowledge base based on the system architecture and the historical system data.

[0026] In step S201, the target system is comprehensively analyzed, including system architecture, service components, network topology, hardware configuration, software stack, and system running environment, etc. The historical system data is extracted from the running records of the target system, including but not limited to system logs, performance monitoring data, alarm records, known fault events, recovery time data, etc. The target knowledge base can be a pre-constructed knowledge base including fault patterns, historical cases, repair strategies, and preventive measures of different systems. After obtaining the system architecture and historical data of the target system, the retrieval enhancement technology is used to intelligently filter out the fault data associated with the system architecture and historical system data of the target system.

[0027] In step S202, the fault data is input into a first target model to obtain an experimental model of the chaos engineering experiment of the target system, wherein the experimental model is used to experiment on M types of fault scenarios of the target system, and M is a positive integer.

[0028] In step S202, the first target model can be a natural language generation model for generating an experiment model of a chaos engineering experiment. Based on the system architecture, historical system data and retrieved failure data, M types of failure scenarios are intelligently generated. Each failure scenario design includes the location, type, intensity and expected impact area of the fault injection. For example, if the target system is a large-scale distributed database cluster, the model may generate experiments including scenarios such as simulating partial node network interruption, master node downtime, data partition access delay, etc. The first target model can intelligently recommend the optimal parameter settings for each failure scenario, including the duration, frequency and specific components affected by the fault injection. Optimization of these parameters can ensure that the chaos experiment is both challenging and not overly disruptive to the system, while enabling the experiment to reflect the system's real behavior and recovery capabilities as much as possible.

[0029] In step S203, based on the fault injection strategy in the experiment model, a preset fault of a target fault type is injected into the target system, and a fault experiment index of the target system after the preset fault is injected is collected, wherein the fault experiment index is used to indicate the performance and state of the target system in the fault scenario, and the fault injection strategy includes the target fault type and fault parameters to be injected.

[0030] In step S203, the experiment model can include M preset fault types, such as network delay, service downtime, database connection failure, disk space depletion, resource overload, etc. For each fault type, the experiment model also specifies a series of fault parameters, such as fault duration, occurrence frequency, impact range, etc. For example, for network delay injection, the model may suggest injecting a delay on a certain key link in the system every few minutes for a certain period of time, with a delay time of hundreds of milliseconds. Before implementing the chaos engineering experiment, ensure that the experimental environment is ready. According to the fault type and parameters in the experiment model, select appropriate fault injection tools and set them according to the model's suggested parameters. For example, if the target fault is to simulate resource overload, select a tool that can simulate resource occupation and set the percentage of resource occupation and the duration.

[0031] According to the guidance of the experiment model, the preset fault is injected into the target system. During the fault injection, the integrated monitoring tool is used to collect the fault experiment index of the target system after the preset fault is injected, such as performance index and state information. These indicators can include central processing unit utilization, memory usage, network throughput, service response time, transaction processing rate, system error rate, etc. Real-time data collection helps to analyze the impact of the fault on the system in real time and evaluate the system's recovery capability.

[0032] In step S204, the fault experiment index is input into the second target model to obtain a fault analysis result, wherein the fault analysis result is used to indicate abnormal states and risk assessment results of the target system in the fault scene of the preset fault.

[0033] In step S204, the second target model can be a machine learning model specially used for analyzing the fault experiment index. It is trained based on the characteristics (such as system architecture, service type, and hardware configuration) of the target system and historical data, and can identify and understand the behavior patterns of the system in different fault scenes. The preprocessed fault experiment index is input into the second target model. The model identifies abnormal states in the system according to the input index. The abnormal states can be mutations or deviations of certain performance indicators, such as sudden spikes in central processing unit utilization, significant increases in network latency, and prolonged service response time. At the same time, the model can assess the risk levels behind these abnormal states, including short-term and long-term risks, and potential impacts on the overall stability of the system. Based on the analysis results of the model, the system will generate a fault analysis report, which describes the abnormal states of the system in different fault scenes and the risk levels of each state. The risk assessment results can be divided into various levels of warnings, such as low risk, medium risk, and high risk, to guide subsequent optimization measures and fault repair strategies.

[0034] The system fault analysis method provided by the embodiment of the present application acquires the system architecture and historical system data of the target system, detects fault data related to the target system from the target knowledge base based on the system architecture and historical system data, inputs the fault data into the first target model to obtain an experiment model of the chaos engineering experiment of the target system, wherein the experiment model is used to experiment on M types of fault scenes of the target system, and M is a positive integer; based on a fault injection strategy in the experiment model, a preset fault under a target fault type is injected into the target system, and fault experiment indexes of the target system after injecting the preset fault are collected, wherein the fault experiment indexes are used to indicate the performance and state of the target system in the fault scene, and the fault injection strategy includes the target fault type and fault parameters injected; the fault experiment indexes are input into the second target model to obtain a fault analysis result, wherein the fault analysis result is used to indicate abnormal states and risk assessment results of the target system in the fault scene of the preset fault, and the problem that the chaos engineering experiment in the related art relies on manual setting and expert experience, resulting in low fault analysis efficiency of the software system is solved. By combining information retrieval, generation model, and chaos engineering, an automatic process from fault data retrieval to fault injection strategy generation and then to system performance state index analysis is realized, the purpose of more accurately identifying system vulnerabilities and improving the overall stability and robustness of the system is achieved, thereby realizing the technical effect of improving the fault analysis efficiency of the software system, and further solving the technical problem that the chaos engineering experiment relies on manual setting and expert experience, resulting in low fault analysis efficiency of the software system.

[0035] In order to more comprehensively and efficiently conduct chaos engineering experiments on the system, a first target model needs to be trained to generate an experimental model. Optionally, in the system fault analysis method provided in the embodiments of the present application, the first target model is trained in the following manner: a training sample set in the field of chaos engineering experiments is obtained, wherein the training sample set contains N training samples, each training sample includes historical fault data and a historical experimental model corresponding to the historical fault data, N is greater than M, N is a positive integer; a pre-training model is initialized, and the model parameters of the pre-training model are fine-tuned through the training sample set to obtain the first target model after training.

[0036] In some embodiments, the training sample set in the field of chaos engineering experiments can be obtained from historical chaos engineering experiments, containing specific fault data and historical experimental models designed for these faults. The fault data can include fault type, fault parameter, fault duration, fault impact range, etc.; and the historical experimental model records the design ideas, execution process, expected results and actual result analysis of the experiment in detail. The pre-training model can be a natural language processing model based on deep learning. The model has been pre-trained on large-scale text data and has strong language understanding and generation capabilities, providing a solid foundation for subsequent fine-tuning. Before training the first target model, the parameters of the pre-training model need to be loaded into the model, i.e., initialization.

[0037] With the initialized pre-training model and the training sample set in the field of chaos engineering experiments, the next step is fine-tuning. The goal of fine-tuning is to enable the model to learn how to generate effective experimental models based on fault data. During fine-tuning, the model will learn to extract key information from the input fault data and generate matching experimental models as output. A loss function is designed to measure the difference between the historical experimental model generated by the model and the actual historical experimental model. The model is guided to continuously optimize its generation strategy until the matching degree of the generated experimental model and the historical experimental model reaches the highest. The fine-tuning process of the model will continue until the preset termination condition is met. The termination condition can be that the model loss no longer decreases significantly, reaches the predetermined number of training rounds, or the model performance on the validation set no longer improves. The setting of these conditions can prevent overfitting and ensure the generalization ability of the model on new data. The trained first target model can retrieve relevant information from the target knowledge base based on retrieval enhancement technology, combine the system architecture and historical data of the target system, and intelligently generate M types of chaos engineering experimental models. These experimental models will guide the design of fault injection strategies to ensure that the experiments can comprehensively cover potential system vulnerabilities.

[0038] The embodiment case realizes the intelligentization and automation of the chaos engineering experiment design by training the first target model in the chaos engineering experiment field, and improves the efficiency of system resilience evaluation and optimization.

[0039] After obtaining the experimental model, the faults are injected into the system based on the fault injection strategy in the experimental model. Optionally, in the fault analysis method of the system provided in the embodiment of the present application, injecting a preset fault under a target fault type into a target system based on a fault injection strategy in an experimental model comprises: determining a target fault type and a fault parameter in the fault injection strategy, wherein the target fault type comprises at least one of the following: network interruption, service downtime, data loss, and delay, and the fault parameter comprises at least one of the following: injection time, duration, frequency, and impact range on the target system; inputting the target fault type and the fault parameter into a preset chaos engineering tool, and injecting a preset fault into a test environment of the target system through the preset chaos engineering tool.

[0040] In some embodiments, the target fault type to be injected is determined according to the indication in the experimental model. The experimental model also provides detailed fault parameters, including injection time, duration, frequency, and impact range on the target system. For example, the injection time is during a business peak or trough, the duration can be a few seconds to a few minutes, the frequency determines the number of repetitions of the same fault type in the experiment, and the impact range indicates the specific service or system component affected by the fault. A preset chaos engineering tool compatible with the target system is selected. According to the target fault type and fault parameters determined in the experimental model, the preset chaos engineering tool is configured, for example, a specific fault injection script is written, the time point of fault injection is set, the duration and frequency of the fault are defined, and the component range affected by the fault is set.

[0041] Before injecting the preset fault into the target system, a test environment isolated from the production environment is required to prevent potential faults from affecting the actual business. The target fault type and fault parameters are input into the preset chaos engineering tool, and then fault injection is performed in the test environment of the target system. The tool will accurately simulate the preset faults, such as network interruption, service downtime, etc., according to the set parameters, so as to test the performance and recovery capability of the system under these abnormal conditions.

[0042] By applying the fault injection strategy in the experimental model to the target system, the stability and recovery capability of the system in the face of various fault scenarios can be systematically tested and evaluated.

[0043] After injecting faults into the target system, the fault experiment indicators of the target system injected with the preset faults are collected. Optionally, in the system fault analysis method provided in the embodiments of the present application, collecting the fault experiment indicators of the target system injected with the preset faults includes: collecting the performance indicators, log information and error rate of the target system through a preset monitoring tool, wherein the error rate is the ratio of the number of response errors to the total number of responses of the target system in the process of providing services; and performing data cleaning and normalization processing on the performance indicators, log information and error rate to obtain the fault experiment indicators.

[0044] In some embodiments, the performance indicators of the target system are collected in real time using a preset monitoring tool. These indicators include but are not limited to central processor utilization, memory usage, disk I / O, network I / O, transaction processing rate, request response time, etc., which can comprehensively reflect the system running state and efficiency. The log information generated by the target system during the fault experiment is collected through a log collection tool. During the fault experiment, the response state of the system service is monitored, the ratio of the number of response errors to the total number of responses is counted, and the error rate is obtained. The error rate provides an intuitive indicator of system stability and health, which helps to evaluate the degree of abnormal performance of the system in the experiment.

[0045] The collected raw data is cleaned to remove noise, handle missing values and correct abnormal values, ensuring the quality and consistency of the data. The cleaned data is normalized to convert indicators of different dimensions and ranges to the same scale, facilitating comparison and model training. The cleaned and normalized performance indicators are part of the fault experiment indicators. The standardized indicators can more intuitively reflect the performance of the system under fault scenarios, facilitating comparative analysis with the normal state. The log information is extracted for keywords and identified for event types, converted into structured data forms such as event frequency and key event list, serving as another part of the fault experiment indicators. Structured log information facilitates machine learning model understanding, helping to generate more accurate risk assessment reports. The calculated error rate is integrated into the fault experiment indicators, which directly reflects the availability and stability of the system service. The trend of error rate is an important basis for evaluating the system's response to fault injection.

[0046] The embodiments of the present application collect the performance indicators, log information and error rate of the target system in real time through a preset monitoring tool, and perform data cleaning and normalization processing thereon, generating fault experiment indicators that can accurately reflect the performance of the system under fault scenarios. The embodiments provide rich and accurate data support for chaos engineering experiments, and provide a solid foundation for subsequent fault analysis, system optimization and decision-making.

[0047] After obtaining the fault experiment indicators, the fault experiment indicators are analyzed through the second target model. Optionally, in the fault analysis method of the system provided in the embodiments of the present application, the fault experiment indicators are input into the second target model to obtain fault analysis results, including: processing the fault experiment indicators through a behavior identification module of the second target model to obtain abnormal behavior identification results of the target system, wherein the abnormal behavior includes at least one of the following: performance degradation, service unavailability, and resource exhaustion; identifying fault points from the fault experiment indicators through a fault point identification module of the second target model, wherein the fault points are services of the target system that have abnormal behavior in the fault scenario; processing the fault experiment indicators through a risk assessment module of the second target model to obtain risk assessment results, wherein the risk assessment results are used to indicate the stability of the target system in the fault scenario and predict the failure mode; and determining at least one of the following as the fault analysis results: the abnormal behavior identification results, the fault points, and the risk assessment results.

[0048] In some embodiments, the second target model is a deep learning-based behavior identification and risk assessment model, which consists of the following three main modules: the behavior identification module is responsible for identifying abnormal behavior of the target system from the fault experiment indicators. The abnormal behavior can include performance degradation, service unavailability, resource exhaustion, etc. The fault point identification module is used to locate specific fault points from the experimental indicators, i.e. those services or components that have abnormal behavior in the fault scenario. The risk assessment module is used to assess the stability of the target system under different types of fault scenarios, and to predict potential failure modes. Quantitative assessment of the recovery ability and potential vulnerability of the system to various faults provides a scientific basis for developing risk management and optimization strategies.

[0049] The collected fault experiment indicators of the target system are input into the second target model. The behavior identification module of the model first performs a deep analysis on the input fault experiment indicators to identify abnormal behavior in the system. For example, if the performance indicators show that the CPU utilization and memory usage have significantly increased after fault injection, while the transaction processing rate and request response time have significantly decreased, these are abnormal behaviors of performance degradation. The fault point identification module then identifies specific fault points from the fault experiment indicators, i.e. which services or components behave abnormally in the fault scenario. By analyzing the error logs in the log information and combining the changes in performance indicators, the model can locate the key fault points in the system. Finally, the risk assessment module quantitatively assesses the stability of the target system under different fault scenarios. It will generate a risk assessment report according to the changes in the experimental indicators, the identification results of abnormal behavior, and the positioning information of the fault points, predict the possible performance of the system under similar faults, and the difficulty of system recovery.

[0050] The behavior recognition result, fault point, and risk assessment result are integrated to form a comprehensive fault analysis report. The report lists the identified abnormal behaviors, fault point locations, and system risk assessment levels in detail. The fault analysis result provides specific and accurate decision support for the operation and maintenance team. For example, based on the identified fault points, targeted resource allocation or service restart can be performed immediately; and the risk assessment result is helpful for long-term planning, such as increasing redundancy, optimizing architecture, and improving system resilience. The fault analysis result should also be fed back to the target knowledge base for updating the knowledge base and optimizing the generation model.

[0051] The embodiment realizes accurate identification of abnormal behaviors of the target system in the fault scenario, rapid positioning of the fault point, and scientific quantification of the risk assessment by inputting the fault experiment index into the second target model. The efficiency and accuracy of fault analysis are improved, and the continuous optimization and resilience of the system are promoted.

[0052] If the fault point and abnormal behavior are identified, the corresponding target repair strategy is executed on the target system in time. Optionally, in the fault analysis method of the system provided in the embodiment of the application, after obtaining the fault analysis result, the method further includes: extracting the fault point and abnormal behavior in the fault analysis result, retrieving the target repair strategy for repairing the fault point and abnormal behavior from the target knowledge base based on the fault point and abnormal behavior; and executing the target repair strategy on the target system.

[0053] In some embodiments, in the output of the second target model, the fault analysis result lists the fault points and abnormal behaviors that occur in the system in detail. Based on the fault points and abnormal behaviors identified by the second target model, the system will automatically retrieve the corresponding fault repair strategies from the target knowledge base. These strategies can include service restart, resource reallocation, optimization of database queries, cache increase, etc., aiming to quickly respond to and repair the identified fault points and restore the normal operation of the system. Once the target repair strategy is generated, it can be directly executed by an automated system or script. For example, the system can automatically restart the problematic service or dynamically adjust resource allocation to address resource depletion.

[0054] The embodiment intelligently extracts the fault points and abnormal behaviors in the fault analysis result, quickly retrieves the most matched repair strategies from the target knowledge base, and automatically generates specific execution schemes in combination with the system state. The efficiency and accuracy of fault repair are improved, and the continuous optimization of the system is promoted, enhancing the recovery ability and stability of the system when facing various faults.

[0055] In order to generate a more comprehensive chaos engineering experiment, a target knowledge base needs to be constructed. Optionally, in the fault analysis method of the system provided in the embodiments of the present application, the target knowledge base is constructed in the following manner: data related to the system is acquired from a plurality of data sources, wherein the plurality of data sources include at least one of the following: system logs, historical fault data, monitoring information, operation manuals, and fault repair documents; for each piece of data, the data is converted into a preset format through a preprocessing manner to obtain preprocessed data; an index is created for each piece of preprocessed data, and all data and the index corresponding to each piece of data are stored in a preset database to obtain the target knowledge base.

[0056] In some embodiments, data related to the system is acquired from a plurality of data sources, which include but are not limited to system logs, historical fault data, monitoring information, operation manuals, fault repair documents, etc. System logs record the detailed situation of system operation, historical fault data provide direct cases of fault mode and impact range, monitoring information reflects the performance indicators of the system in various states, and operation manuals and fault repair documents contain the best practices and guidance for system operation and fault handling. For each piece of data acquired from the plurality of data sources, it is converted into a preset unified format through a preprocessing manner. The preprocessing manner can include text parsing, data conversion scripts, regular expression matching, natural language processing, etc. The most suitable preprocessing technology is selected according to the characteristics and content of the data source.

[0057] An index is created for each piece of preprocessed data, which can accelerate the data retrieval process. The index can be created based on keywords, data types, timestamps, etc. A database suitable for storing a large amount of documents and data is selected, and all preprocessed data and the index corresponding to each piece of data are stored in the preset database to construct the target knowledge base. The target knowledge base should be a dynamically updated database. As the system changes, new fault cases and repair strategies appear, the knowledge base continuously integrates new data sources, and data preprocessing and index updating are performed. To ensure the accuracy and reliability of the knowledge base, the data sources are regularly checked for quality, and outdated or inaccurate information is removed. At the same time, an expert review mechanism is introduced to ensure the professionalism and practicality of the knowledge base content.

[0058] The embodiments effectively integrate multiple data sources, perform data preprocessing, and establish data indexes to construct a target knowledge base, providing rich background data and historical cases for intelligent design and fault analysis of chaos engineering experiments. The accuracy and efficiency of fault experiments are improved, and key references are provided for the generation of fault repair strategies and system optimization.

[0059] According to another embodiment of the present application, a fault analysis system is also provided, Figure 3 is a schematic diagram of the fault analysis system provided in the embodiments of the present application, asFigure 3 As shown, the system includes: a user interface / console, an experiment execution and data collection module, an experiment design module, a data analysis and intelligent identification module, a fault repair and optimization decision module, and a report generation module.

[0060] Specifically, the user interface / console is used to display experimental results, report generation, and system control, etc.

[0061] The experiment execution and data collection module is responsible for automatically injecting pre-set faults into the target system according to the designed experimental scenario. The system simulates fault scenarios including network interruption, service crash, data loss, delay, etc. Real-time monitoring and data collection: through integrated monitoring tools, real-time collection of system performance data (such as central processor utilization, memory consumption, network delay, etc.), log information, error rate, and other key indicators. All experimental data, including monitoring data, system logs, fault injection records, etc., will be stored in the database for subsequent analysis and query.

[0062] The data analysis and intelligent identification module uses data cleaning and preprocessing techniques (such as denoising, data normalization, etc.) to organize the collected large amount of monitoring data into a structured format for analysis. The machine learning sub-module uses machine learning algorithms to identify patterns in the cleaned data, detect abnormal behavior of the system after fault injection, such as resource exhaustion, performance degradation, service unavailability, etc. Vulnerability identification: using RAG (Retrieval-Augmented Generation) technology, combined with historical fault data and system state, automatically identify potential vulnerabilities in the system. By searching external knowledge bases, provide the best repair strategies and preventive measures related to the current problem. Intelligent risk assessment: based on machine learning models and RAG technology, generate a risk assessment report for the system, assess the performance and stability of the system under different fault scenarios, predict possible failure modes, and provide optimization suggestions.

[0063] The fault repair and optimization decision module, based on RAG technology, intelligently provides decision support for system administrators in fault repair and optimization by searching external documents and historical cases. For example, the system may recommend improving the fault tolerance of a certain service, optimizing database queries, adding redundant servers, etc. Automated repair plan: combined with the results of fault experiments, the system can automatically generate repair plans and optimization strategies. These plans may include repairing code, adjusting configuration, optimizing system architecture, etc., and provide developers with detailed operation steps and reference documents. Real-time adjustment: the system can quickly identify parts that need to be optimized based on real-time experimental data, and automatically implement certain repair operations (such as increasing resources, adjusting load balancing strategies, etc.).

[0064] The report generation module is responsible for automatically generating detailed reports based on experimental results and analysis data, including fault types, specific times of occurrence, impact ranges, fault repair measures, and system optimization suggestions. The report will be presented in structured and unstructured forms, including text descriptions and chart data. Through the RAG technology, the report will not only show experimental data, but also extract relevant historical cases and best practices from the knowledge base to provide developers with more comprehensive fault repair suggestions and system optimization paths. The report also includes a feedback mechanism that allows developers to make further adjustments to the system based on the report content or provide feedback to further optimize the chaos experiment process.

[0065] The fault analysis system of the embodiment can intelligently design, execute and analyze chaos experiments by introducing RAG technology, combining artificial intelligence with chaos engineering, and automatically identifying potential vulnerabilities in the system. It improves the automation level of fault experiments, avoids the subjectivity and bias of manual judgment, and improves the efficiency and accuracy of experiments. Through intelligent analysis of experimental results, it can identify and predict system risks in advance, optimize the resilience and stability of the system, and provide scientific theoretical basis and technical support for fault tolerance and fault recovery of large-scale and complex systems. Automatically generate diverse and comprehensive fault experiment scenarios to improve experimental coverage and accuracy; reduce manual intervention through intelligent analysis of experimental data to discover system vulnerabilities and potential risks in real time; and generate intelligent optimization solutions and fault repair suggestions to improve fault response speed, optimize system stability and recovery capability, thereby significantly improving the efficiency and accuracy of chaos engineering.

[0066] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0067] Embodiment 2 The embodiments of the present application also provide a system fault analysis device. It should be noted that the system fault analysis device of the embodiments of the present application can be used to execute the system fault analysis method provided by the embodiments of the present application. The system fault analysis device provided by the embodiments of the present application is introduced below.

[0068] According to the embodiments of the present application, a device for implementing the above-mentioned system fault analysis method is also provided, Figure 4 is a schematic diagram of the system fault analysis device provided by the embodiments of the present application, as Figure 4 shown, the device comprises: The acquisition unit 401 is configured to acquire system architecture and historical system data of a target system, and detect fault data related to the target system from a target knowledge base based on the system architecture and the historical system data; The first input unit 402 is configured to input the fault data into a first target model to obtain an experiment model of a chaos engineering experiment of the target system, wherein the experiment model is used for M types of fault scenarios of an experimental target system, and M is a positive integer. The injection unit 403 is configured to inject a preset fault under a target fault type into the target system based on a fault injection strategy in the experiment model, and collect fault experiment indexes of the target system after the preset fault is injected, wherein the fault experiment indexes are used to indicate performance and a state of the target system in the fault scenario, and the fault injection strategy includes the target fault type and fault parameters to be injected. The second input unit 404 is configured to input the fault experiment indexes into a second target model to obtain fault analysis results, wherein the fault analysis results are used to indicate abnormal states and risk assessment results of the target system in the fault scenario of the preset fault.

[0069] The system fault analysis apparatus provided in the embodiment of the present application is configured to acquire system architecture and historical system data of a target system by the acquisition unit 401, detect fault data related to the target system from a target knowledge base based on the system architecture and the historical system data, input the fault data into a first target model by the first input unit 402 to obtain an experiment model of a chaos engineering experiment of the target system, wherein the experiment model is used for M types of fault scenarios of an experimental target system, and M is a positive integer, inject a preset fault under a target fault type into the target system based on a fault injection strategy in the experiment model by the injection unit 403, and collect fault experiment indexes of the target system after the preset fault is injected, wherein the fault experiment indexes are used to indicate performance and a state of the target system in the fault scenario, and the fault injection strategy includes the target fault type and fault parameters to be injected, and input the fault experiment indexes into a second target model by the second input unit 404 to obtain fault analysis results, wherein the fault analysis results are used to indicate abnormal states and risk assessment results of the target system in the fault scenario of the preset fault. By combining information retrieval, a generated model and chaos engineering, an automatic process from fault data retrieval to fault injection strategy generation and then to system performance state index analysis is realized, the purpose of more accurately identifying system weak points and improving overall stability and robustness of the system is achieved, the technical effect of improving fault analysis efficiency of a software system is realized, and the technical problem of low fault analysis efficiency of the software system caused by dependence of chaos engineering experiments on manual setting and expert experience is solved.

[0070] Optionally, in the system fault analysis apparatus provided by the embodiment of the present application, the apparatus further comprises: a sample acquisition unit, configured to acquire a training sample set in the field of chaotic engineering experiments, wherein the training sample set comprises N training samples, each training sample comprises historical fault data and a historical experiment model corresponding to the historical fault data, N is greater than M, N is a positive integer; and a training unit, configured to initialize a pre-training model, fine-tune model parameters of the pre-training model by using the training sample set, and obtain a first target model after training.

[0071] Optionally, in the system fault analysis apparatus provided by the embodiment of the present application, the injection unit 403 comprises: a first determination module, configured to determine a target fault type and a fault parameter in the fault injection strategy, wherein the target fault type comprises at least one of network interruption, service downtime, data loss and delay, and the fault parameter comprises at least one of injection time, duration, frequency and influence range on the target system; and an input module, configured to input the target fault type and the fault parameter into a preset chaotic engineering tool, and inject a preset fault in a test environment of the target system by using the preset chaotic engineering tool.

[0072] Optionally, in the system fault analysis apparatus provided by the embodiment of the present application, the injection unit 403 comprises: a collection module, configured to collect performance indicators, log information and error rates of the target system by using a preset monitoring tool, wherein the error rate is a proportion of the number of response errors to the total number of responses in the process of providing services by the target system; and a first processing module, configured to perform data cleaning and normalization processing on the performance indicators, the log information and the error rates, and obtain fault experiment indicators.

[0073] Optionally, in the system fault analysis apparatus provided by the embodiment of the present application, the second input unit 404 comprises: a second processing module, configured to process the fault experiment indicators by using a behavior recognition module of the second target model, and obtain an abnormal behavior recognition result of the target system, wherein the abnormal behavior comprises at least one of performance degradation, service unavailability and resource exhaustion; a recognition module, configured to identify a fault point from the fault experiment indicators by using a fault point identification module of the second target model, wherein the fault point is a service in which the target system has an abnormal behavior in a fault scenario; a third processing module, configured to process the fault experiment indicators by using a risk assessment module of the second target model, and obtain a risk assessment result, wherein the risk assessment result is used to indicate stability and a predicted fault mode of the target system in the fault scenario; and a second determination module, configured to determine at least one of the abnormal behavior recognition result, the fault point and the risk assessment result as a fault analysis result.

[0074] Optionally, in the system fault analysis apparatus provided by the embodiment of the present application, the apparatus further comprises an extraction unit configured to extract a fault point and an abnormal behavior in the fault analysis result, and retrieve a target repair strategy for repairing the fault point and the abnormal behavior from a target knowledge base based on the fault point and the abnormal behavior; and an execution unit configured to execute the target repair strategy on the target system.

[0075] Optionally, in the system fault analysis apparatus provided by the embodiment of the present application, the target knowledge base is constructed in the following manner: obtaining system-related data of a plurality of data sources, wherein the plurality of data sources comprise at least one of the following: system logs, historical fault data, monitoring information, operation manuals and fault repair documents; for each piece of data, converting the data into a preset format through a preprocessing manner to obtain preprocessed data; creating an index for each piece of preprocessed data, and storing all the data and the index corresponding to each piece of data to a preset database to obtain the target knowledge base.

[0076] It should be noted that the above acquisition unit 401, first input unit 402, injection unit 403 and second input unit 404 correspond to steps S201 to S204 in Embodiment 1, and the four units have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above embodiment one. It should be noted that the above modules or units can be hardware components or software components stored in the memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b, …, 102n), or can be a part of the apparatus and can run in the computer terminal 10 provided in Embodiment 1.

[0077] Embodiment 3 The embodiments of the present application can provide an electronic device, Figure 5 is a structural block diagram of an electronic device according to an embodiment of the present application. As Figure 5 shown, the electronic device can include one or more (only one is shown in the figure) processors 502, a memory 504, a storage controller, and a peripheral interface, wherein the peripheral interface is connected with a radio frequency module, an audio module and a display. Figure 5

[0078] ​The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the method and device in the embodiments of the present application. The processor executes various functions and data processing by running the software programs and modules stored in the memory, that is, implements the above method. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the terminal through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0079] The processor can call information and applications stored in the memory through the transmission device to perform the following steps: obtaining system architecture and historical system data of a target system, detecting fault data related to the target system from a target knowledge base based on the system architecture and the historical system data; inputting the fault data into a first target model to obtain an experimental model of a chaos engineering experiment of the target system, wherein the experimental model is used for M types of fault scenarios of the experimental target system, and M is a positive integer; injecting a preset fault under a target fault type into the target system based on a fault injection strategy in the experimental model, collecting fault experiment indicators of the target system after injecting the preset fault, wherein the fault experiment indicators are used to indicate the performance and state of the target system in the fault scenario, and the fault injection strategy includes the target fault type and fault parameters injected; inputting the fault experiment indicators into a second target model to obtain a fault analysis result, wherein the fault analysis result is used to indicate an abnormal state and a risk assessment result of the target system in the fault scenario of the preset fault.

[0080] The processor can also call information and applications stored in the memory through the transmission device to perform the following steps: obtaining a training sample set in the field of chaos engineering experiments, wherein the training sample set includes N training samples, each training sample includes historical fault data and a historical experimental model corresponding to the historical fault data, N is greater than M, and N is a positive integer; initializing a pre-trained model, fine-tuning model parameters of the pre-trained model through the training sample set, and obtaining a trained first target model.

[0081] The processor can also call information and applications stored in the memory through the transmission device to perform the following steps: determining the target fault type and the fault parameter in the fault injection strategy, wherein the target fault type includes at least one of the following: network interruption, service downtime, data loss, and delay, and the fault parameter includes at least one of the following: injection time, duration, frequency, and impact range on the target system; inputting the target fault type and the fault parameter into a preset chaos engineering tool, and injecting the preset fault in a test environment of the target system through the preset chaos engineering tool.

[0082] The processor can further call information and application programs stored in the memory through the transmission device to perform the following steps: collecting performance indicators, log information and error rates of the target system through a preset monitoring tool, wherein the error rate is the ratio of the number of response errors to the total number of responses of the target system in the process of providing services; performing data cleaning and normalization processing on the performance indicators, log information and error rates to obtain fault experiment indicators.

[0083] The processor can further call information and application programs stored in the memory through the transmission device to perform the following steps: processing the fault experiment indicators through a behavior identification module of the second target model to obtain an abnormal behavior identification result of the target system, wherein the abnormal behavior includes at least one of the following: performance degradation, service unavailability and resource exhaustion; identifying the fault point from the fault experiment indicators through a fault point identification module of the second target model, wherein the fault point is a service in which the target system has abnormal behavior in the fault scenario; processing the fault experiment indicators through a risk assessment module of the second target model to obtain a risk assessment result, wherein the risk assessment result is used to indicate the stability and the predicted failure mode of the target system in the fault scenario; determining at least one of the following as the fault analysis result: the abnormal behavior identification result, the fault point and the risk assessment result.

[0084] The processor can further call information and application programs stored in the memory through the transmission device to perform the following steps: extracting the fault point and the abnormal behavior in the fault analysis result, retrieving a target repair strategy for repairing the fault point and the abnormal behavior from the target knowledge base based on the fault point and the abnormal behavior; and executing the target repair strategy on the target system.

[0085] The processor can further call information and application programs stored in the memory through the transmission device to perform the following steps: obtaining system-related data of a plurality of data sources, wherein the plurality of data sources include at least one of the following: system logs, historical fault data, monitoring information, operation manuals and fault repair documents; converting each piece of data into a preset format through a preprocessing method to obtain preprocessed data; creating an index for each piece of preprocessed data, storing all data and the index corresponding to each piece of data to a preset database to obtain a target knowledge base.

[0086] By adopting the embodiment of the application, system architecture and historical system data of a target system are acquired, fault data related to the target system is detected from a target knowledge base based on the system architecture and the historical system data, the fault data is input into a first target model, an experiment model of a chaos engineering experiment of the target system is obtained, wherein the experiment model is used for M types of fault scenes of an experiment target system, and M is a positive integer, a preset fault under a target fault type is injected into the target system based on a fault injection strategy in the experiment model, fault experiment indexes of the target system after the preset fault is injected are collected, wherein the fault experiment indexes are used to indicate performance and a state of the target system in the fault scene, and the fault injection strategy includes the target fault type and fault parameters injected, and the fault experiment indexes are input into a second target model, and fault analysis results are obtained, wherein the fault analysis results are used to indicate an abnormal state of the target system in the fault scene of the preset fault and a scheme of risk assessment results. By combining information retrieval, a generation model and chaos engineering, an automatic process from fault data retrieval to fault injection strategy generation and then to system performance state index analysis is realized, the purpose of more accurately identifying system vulnerability and improving overall stability and robustness of the system is achieved, technical effects of improving fault analysis efficiency of the software system are realized, and technical problems that chaos engineering experiments depend on manual setting and expert experience and result in low fault analysis efficiency of the software system are solved.

[0087] Those skilled in the art can understand that Figure 5 The structure shown is only schematic, and the electronic device can also be a terminal device such as a smart phone, a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, and the like. Figure 5 This does not limit the structure of the electronic device. For example, the electronic device can include more or fewer components (such as a network interface, a display device, and the like) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 5 For example, the electronic device can include more or fewer components (such as a network interface, a display device, and the like) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 5 For example, the electronic device can include more or fewer components (such as a network interface, a display device, and the like) than those shown in the figure, or have a different configuration from that shown in the figure.

[0088] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by programs instructing the related hardware of the terminal device, and the programs can be stored in a computer readable storage medium, which can include a flash disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.

[0089] Embodiment 4 The embodiment of the application further provides a storage medium. Optionally, in the embodiment, the storage medium can be used to save program codes executed by the fault analysis method of the system provided in the embodiment one.

[0090] Optionally, in the embodiment, the storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0091] The application further provides a computer program product, which is suitable for executing the program of the system failure analysis method when executed on a data processing device.

[0092] The above-mentioned serial numbers of the embodiments of the application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0093] In the above-mentioned embodiments of the application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0094] In the several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the unit described as the division is only a logical division, and there can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0095] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0096] In addition, each functional unit in each embodiment of the application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0097] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk and various program code storage media.

[0098] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A system fault analysis method, characterized in that, include: Obtain the system architecture and historical system data of the target system, and detect fault data related to the target system from the target knowledge base based on the system architecture and the historical system data; The fault data is input into the first target model to obtain the experimental model of the chaos engineering experiment of the target system, wherein the experimental model is used to experiment with M types of fault scenarios of the target system, where M is a positive integer; Based on the fault injection strategy in the experimental model, a preset fault of the target fault type is injected into the target system, and the fault experimental index of the target system injected with the preset fault is collected. The fault experimental index is used to indicate the performance and status of the target system in the fault scenario. The fault injection strategy includes the injected target fault type and fault parameters. The fault experiment indicators are input into the second target model to obtain fault analysis results, wherein the fault analysis results are used to indicate the abnormal state and risk assessment results of the target system in the fault scenario of the preset fault.

2. The method according to claim 1, characterized in that, The first target model was trained in the following manner: Obtain a training sample set in the field of chaos engineering experiments, wherein the training sample set contains N training samples, each training sample includes historical fault data and the historical experimental model corresponding to the historical fault data, N is greater than M, and N is a positive integer; Initialize the pre-trained model, and fine-tune the model parameters of the pre-trained model using the training sample set to obtain the first target model after training.

3. The method according to claim 1, characterized in that, Based on the fault injection strategy in the experimental model, the target system is injected with preset faults of the target fault type, including: The target fault type and the fault parameters in the fault injection strategy are determined, wherein the target fault type includes at least one of the following: network interruption, service failure, data loss and delay, and the fault parameters include at least one of the following: injection time, duration, frequency and scope of impact on the target system; The target fault type and the fault parameters are input into a preset chaos engineering tool, and the preset fault is injected into the test environment of the target system through the preset chaos engineering tool.

4. The method according to claim 1, characterized in that, The fault test indicators of the target system injected with the preset fault include: The target system's performance metrics, log information, and error rate are collected using a preset monitoring tool. The error rate is the ratio of the number of times the target system responds to errors during the service provision process to the total number of responses. The performance metrics, log information, and error rate are cleaned and normalized to obtain the fault experiment metrics.

5. The method according to claim 1, characterized in that, The fault experiment indicators are input into the second target model to obtain the fault analysis results, including: The fault experiment indicators are processed by the behavior recognition module of the second target model to obtain the abnormal behavior recognition results of the target system, wherein the abnormal behavior includes at least one of the following: performance degradation, service unavailability and resource exhaustion; The fault point identification module of the second target model identifies fault points from the fault experiment indicators, wherein the fault point is a service of the target system that exhibits the abnormal behavior in the fault scenario; The failure experiment indicators are processed by the risk assessment module of the second target model to obtain risk assessment results, wherein the risk assessment results are used to indicate the stability of the target system in failure scenarios and predict failure modes. The fault analysis result shall be determined by at least one of the following: the abnormal behavior identification result, the fault point, and the risk assessment result.

6. The method according to claim 1, characterized in that, After obtaining the fault analysis results, the method further includes: Extract the fault points and abnormal behaviors from the fault analysis results, and retrieve the target repair strategies for repairing the fault points and abnormal behaviors from the target knowledge base based on the fault points and abnormal behaviors; The target repair strategy is executed on the target system.

7. The method according to claim 1, characterized in that, The target knowledge base is constructed in the following manner: Acquire system-related data from multiple data sources, wherein the multiple data sources include at least one of the following: system logs, historical fault data, monitoring information, operation manuals, and fault repair documents; For each piece of data, the data is converted into a preset format through preprocessing to obtain preprocessed data; An index is created for each preprocessed data entry, and all the data and the corresponding index for each data entry are stored in a preset database to obtain the target knowledge base.

8. A system fault analysis device, characterized in that, include: The acquisition unit is used to acquire the system architecture and historical system data of the target system, and detect fault data related to the target system from the target knowledge base based on the system architecture and the historical system data. The first input unit is used to input the fault data into the first target model to obtain the experimental model of the chaos engineering experiment of the target system, wherein the experimental model is used to experiment with M types of fault scenarios of the target system, where M is a positive integer; An injection unit is used to inject a preset fault of a target fault type into the target system based on the fault injection strategy in the experimental model, and to collect fault experimental indicators of the target system injected with the preset fault. The fault experimental indicators are used to indicate the performance and status of the target system in the fault scenario. The fault injection strategy includes the injected target fault type and fault parameters. The second input unit is used to input the fault experiment indicators into the second target model to obtain fault analysis results, wherein the fault analysis results are used to indicate the abnormal state and risk assessment results of the target system in the fault scenario of the preset fault.

9. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, executes the fault analysis method of the system according to any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the fault analysis method of the system according to any one of claims 1 to 7.

Images