Intelligent fault analyzing and positioning method and system

Through integrated log analysis, call link tracking, network traffic monitoring and intelligent fault analysis, automated fault positioning and intelligent alarms are realized, solving the problems of low fault positioning efficiency and poor alarm accuracy in the existing technology, and improving fault processing efficiency and operation and maintenance efficiency.

CN119512787APending Publication Date: 2025-02-25GUIZHOU POWER GRID CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411501967.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-25
Publication Date
2025-02-25

AI Technical Summary

Technical Problem

The existing intelligent fault positioning methods have problems such as low efficiency in fault positioning relying on manual analysis, lack of full-link tracking capabilities for call links between services, low alarm accuracy and long response time, and how to achieve intelligent and automated fault positioning and fault recovery.

Method used

Through integrated log analysis, call link tracking, network traffic monitoring and intelligent fault analysis, automated fault location, intelligent alarm and automatic recovery are achieved. The specific steps include: collecting and monitoring data through the first device, performing a first analysis based on the collected data, and automating alarm and recovery of the faulty part.

Benefits of technology

It realizes rapid and accurate identification of faults and takes effective recovery measures, overcomes the shortcomings of the existing technology in automation and intelligence, improves fault handling efficiency, reduces system downtime, and improves operation and maintenance efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119512787A_ABST
    Figure CN119512787A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent fault analysis and positioning method and system, and relates to the technical field of intelligent operation and maintenance and information technology operation and maintenance management, and the method comprises the steps: carrying out the data collection and monitoring through a first device; performing first analysis based on the collected data; and the fault part is automatically warned and recovered. According to the intelligent fault analysis and positioning method provided by the invention, by calling link tracking, the fault solving efficiency is improved, intelligent fault prediction and detection functions are realized through fault mode identification, the fault processing efficiency is greatly improved, and an intelligent alarm and automatic recovery mechanism ensures that the system can immediately respond when a fault occurs, so that the fault processing efficiency is greatly improved. According to the invention, highly automatic fault response and repair functions are realized, the response time of operation and maintenance personnel is reduced, normal operation of the system can be recovered within the shortest time, the influence of faults on service operation is reduced, the availability of the system is improved, and the system reliability is improved. The method has better effects in the aspects of fault positioning efficiency, accuracy and response speed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of intelligent operation and maintenance and information technology operation and maintenance management technology, and specifically to an intelligent fault analysis and positioning method and system. Background Art

[0002] With the widespread application of microservice architecture in modern information technology systems, the complexity of the system has increased significantly, and traditional single application monitoring and fault analysis methods can no longer adapt to complex distributed systems. Modern operation and maintenance requires real-time monitoring of the system and rapid fault location. Therefore, technologies such as log management, call link tracing, and network traffic monitoring have gradually become key tools for solving the operation and maintenance problems of microservice architecture. Systems based on these technologies, such as the ELK stack and eBPF technology, can provide comprehensive data collection and monitoring functions, providing operation and maintenance personnel with in-depth performance analysis and troubleshooting methods. However, as the scale of the system expands, how to intelligently analyze massive data and quickly locate faults has become one of the main challenges of current operation and maintenance technology. In recent years, the intelligent operation and maintenance platform (AIOps) combined with machine learning technology has gradually developed, significantly improving operation and maintenance efficiency through automated fault pattern recognition and intelligent fault location.

[0003] Although existing log analysis, call link tracking and network traffic monitoring technologies can provide detailed system operation data, they still have obvious deficiencies in the automation and intelligence of fault location. First, the existing technology often relies on manual analysis, resulting in low efficiency of fault location, and it is difficult to quickly find the source of the fault in a complex microservice environment. Secondly, the existing monitoring tools lack an in-depth understanding of the correlation between services and cannot achieve full-link fault tracking, especially when service calls span multiple nodes, it is easy to ignore potential bottlenecks. In addition, although some systems can generate alarm information, the accuracy and timeliness of the alarm are poor, and often cause alarm flooding due to excessive noise, thereby reducing the response efficiency of operation and maintenance personnel. In contrast, the present invention realizes automated fault location, intelligent alarm and automatic recovery by integrating log analysis, call link tracking, network traffic monitoring and intelligent fault analysis, and can quickly and accurately identify faults and take effective recovery measures, overcoming the deficiencies of the existing technology in automation and intelligence. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is: the existing intelligent fault location methods have the problems of low efficiency in relying on manual analysis for fault location, lack of full-link tracking capability of call links between services, low alarm accuracy and long response time, as well as how to achieve intelligent and automated fault location and fault recovery.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: an intelligent fault analysis and positioning method, comprising data collection and monitoring through a first device; performing a first analysis based on the collected data; and automatically alarming and restoring the faulty part.

[0007] As a preferred solution of the intelligent fault analysis and positioning method described in the present invention, wherein: the data collection and monitoring through the first device includes uniformly collecting logs through the first device, and transmitting and indexing the log data.

[0008] As a preferred solution of the intelligent fault analysis and positioning method described in the present invention, the data collection and monitoring performed by the first device includes allocating IDs, recording response information, and capturing requests.

[0009] As a preferred solution of the intelligent fault analysis and positioning method described in the present invention, the first analysis based on the collected data includes deploying a flow collector, monitoring, outputting performance indicators, and analyzing the data in real time.

[0010] As a preferred solution of the intelligent fault analysis and positioning method described in the present invention, the first analysis based on the collected data includes learning historical fault data, establishing a fault mode library, identifying fault types, identifying faults and starting the first analysis.

[0011] As a preferred solution of the intelligent fault analysis and location method of the present invention, wherein: the automatic alarm and recovery of the faulty part includes a comprehensive analysis of the call chain to identify the service nodes in the link;

[0012] Combine service call data with log information to track the root cause of service failures;

[0013] Combine network traffic monitoring with service call chain to judge service calls.

[0014] As a preferred solution of the intelligent fault analysis and location method of the present invention, wherein: the automatic alarm and recovery of the faulty part includes detecting and triggering an alarm based on the collected real-time data;

[0015] Automatically perform recovery actions based on the severity of the fault and preset policies.

[0016] Another object of the present invention is to provide an intelligent fault analysis and positioning system, which can solve the problems of low alarm accuracy and long response time in current intelligent fault positioning methods through a first analysis.

[0017] As a preferred solution of the intelligent fault analysis and positioning system described in the present invention, it includes: a data acquisition and monitoring module, a first analysis and positioning module, and an automatic alarm and recovery module; the data acquisition and monitoring module is used to acquire and monitor data in the system; the first analysis and positioning module is used to identify the fault point service in the system; the automatic alarm and recovery module is responsible for real-time alarm and recovery operations of faults.

[0018] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a step of an intelligent fault analysis and positioning method.

[0019] A computer-readable storage medium stores a computer program, which implements the steps of an intelligent fault analysis and positioning method when executed by a processor.

[0020] Beneficial effects of the present invention: The intelligent fault analysis and positioning method provided by the present invention realizes full-link call monitoring through call link tracking, quickly identifies and locates abnormal points in service calls, helps operation and maintenance personnel narrow the scope of faults, thereby reducing troubleshooting time and improving the efficiency of fault solving. Fault pattern recognition realizes intelligent fault prediction and detection functions, and can detect abnormalities before the problem is fully exposed, and intervene in time, thereby reducing system downtime and providing early warning for operation and maintenance personnel. Intelligent fault positioning realizes fault analysis across multiple dimensions, ensures the accuracy and comprehensiveness of fault positioning, quickly and accurately determines the root cause of the fault, reduces the possibility of human intervention and misjudgment, and greatly improves the efficiency of fault handling. The intelligent alarm and automatic recovery mechanism ensures that the system can respond immediately when a fault occurs. By real-time monitoring of the collected data, the system automatically triggers an alarm when an abnormality is detected, and executes preset recovery operations according to the severity of the fault, such as restarting the service or adjusting traffic distribution. It realizes highly automated fault response and repair functions, reduces the response time of operation and maintenance personnel, ensures that the system can resume normal operation in the shortest time, thereby reducing the impact of faults on business operations and improving the availability of the system. The present invention achieves better results in terms of fault location efficiency, accuracy and response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0022] Figure 1An overall flow chart of an intelligent fault analysis and locating method provided for the first embodiment of the present invention.

[0023] Figure 2 An overall flow chart of an intelligent fault analysis and positioning system provided for the third embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0025] Example 1, reference Figure 1 , is an embodiment of the present invention, and provides an intelligent fault analysis and location method, comprising:

[0026] S1: Data collection and monitoring are performed through the first device.

[0027] Furthermore, log collection is the basic data source for the entire intelligent fault analysis and location process.

[0028] Through the ELK stack (Elasticsearch, Logstash, Kibana) or similar log management systems, the log data of each microservice is collected and processed in real time. The log includes information such as service operation status, abnormal errors, and detailed processing of service requests.

[0029] The logs generated by each microservice instance are collected by Logstash. Log data is formatted and filtered to remove redundant information, and valid log data is transmitted to Elasticsearch.

[0030] Elasticsearch indexes the logs to ensure that they can be quickly queried and analyzed later.

[0031] Log information provides key evidence for system failure analysis, helping the system trace the specific operation process when a failure occurs.

[0032] It should be noted that the first device can be of multiple types, for example, using Logstash in the ELK stack (i.e. Elasticsearch, Logstash, Kibana) as the core device for data collection, Fluentd, Graylog, Filebeat, SplunkForwarder and other methods. In this embodiment, Logstash in the ELK stack (i.e. Elasticsearch, Logstash, Kibana) is used as the core device for data collection.

[0033] In another feasible embodiment, the first detection of the oscillation mode adopts Fluentd, Graylog, Filebeat, and Splunk Forwarder, including: Fluentd is an open source data collection tool with similar functions to Logstash. It implements flexible log collection, formatting, and forwarding through a plug-in system, supports collecting data from multiple sources, and sends data to multiple backends (such as Elasticsearch, MongoDB, etc.). Compared with Logstash, the advantage of Fluentd is that it has low resource usage and is relatively simple to install and configure. However, Logstash is more flexible and powerful in data processing and filtering functions. Graylog is a tool focused on log management that can collect log data from different sources and process and store it through built-in analysis functions. Graylog comes with a data collector that can perform data collection and processing similar to Logstash, but its functions may be more suitable for small and medium-sized log processing needs, and its large-scale log processing performance is slightly weaker than Logstash. Filebeat is part of the Elastic ecosystem and is compatible with Logstash. It is used to lightweightly collect log data from servers and applications and send it to Elasticsearch or Logstash. Although Filebeat's functionality focuses on lightweight log transmission, its log processing capabilities are relatively limited and it cannot perform complex filtering and conversions like Logstash, but it is a more lightweight, resource-friendly alternative suitable for resource-constrained environments. SplunkForwarder is part of the Splunk platform and is used to collect data from various sources and transmit it to the Splunk server for analysis. Although Splunk excels in performance and data processing capabilities and is very popular in large-scale enterprise environments, its high commercial costs and complex configuration requirements make it less flexible and economical than Logstash.

[0034] Furthermore, call chain tracing is used to capture the call path between services in the system, helping to understand the flow of requests in the microservice system. Based on interceptor or eBPF (Extended Berkeley Packet Filter) technology, the call process of each request between multiple microservices is recorded, and the Trace ID and Span data of the service call are captured to form a complete call chain.

[0035] Assign a unique Trace ID to each request throughout the entire request lifecycle. Through interceptor technology, capture Span data at the beginning and end of service calls, record service response time, status, caller and callee information. Track service calls and network communications at the kernel layer through eBPF, capture the flow path of requests between services and communication details.

[0036] Link tracking helps the system quickly determine the path of service calls and possible abnormal services during the call process.

[0037] S2: Perform a first analysis based on the collected data.

[0038] Furthermore, network traffic monitoring is used to monitor the quality of network communication between services to ensure that the network level of the system is stable and reliable. Real-time monitoring of network traffic between services collects key performance indicators including bandwidth utilization, latency, packet loss rate, etc. to ensure that the communication network is unimpeded.

[0039] Deploy traffic collectors in network nodes between service instances and perform lossless traffic monitoring using eBPF technology. The collectors regularly capture data packets at the network layer and calculate performance indicators such as bandwidth, latency, and packet loss rate. Perform real-time analysis on the collected data to identify abnormal traffic or network problems.

[0040] By monitoring network layer performance, the system can identify service call delays or failures caused by network communication problems.

[0041] Deploy traffic collectors in the network nodes between service instances and use eBPF technology for lossless traffic monitoring: By deploying traffic collectors between the network nodes of service instances, the system can effectively monitor the communication between different services. The innovation of this step is the use of eBPF technology for lossless monitoring, which means that the system does not need to interrupt existing network communications and does not introduce significant delays or performance losses. Therefore, this step can ensure real-time data packet collection and acquisition of key information in a complex distributed environment without negatively affecting network communications. Through this design, the system can achieve accurate monitoring of network traffic while ensuring performance.

[0042] The collector regularly captures data packets and calculates performance indicators such as bandwidth, latency, and packet loss rate: This step further expands the level of data collection. The system not only simply captures network data packets, but also comprehensively calculates performance indicators such as bandwidth utilization, latency, and packet loss rate through a regular collection mechanism. These data can promptly reflect the health of the network, especially in high-load or abnormal traffic environments. Through this regular capture and analysis method, the system can quickly locate problems in network communications and discover potential bottlenecks or failures in advance, thereby effectively preventing service interruptions or performance degradation caused by network problems.

[0043] Analyze the collected data in real time to identify abnormal traffic or network problems: By analyzing the collected traffic data in real time, the system can quickly identify abnormal traffic patterns or network problems. Compared with traditional static analysis methods, real-time analysis allows the system to quickly respond to problems in the early stages of their occurrence to prevent them from escalating. The innovation of this step is that, combined with the powerful analysis capabilities of eBPF, network monitoring is not limited to passive monitoring, but can also perform active anomaly detection and response. Through this design, the system can ensure that the network layer is always efficient and stable in a complex microservice environment.

[0044] It should be noted that fault pattern recognition is a key link in the intelligent analysis module, which automatically detects anomalies and potential faults through historical data and machine learning algorithms. Using machine learning technology and a fault pattern library, the system can detect abnormal patterns during real-time operation and identify faults based on preset fault patterns.

[0045] The system learns historical fault data, builds a fault pattern library, and identifies common fault types and abnormal patterns. It monitors various types of data (such as call chains, logs, and network traffic) in real time and matches them with known fault patterns. When a matching abnormal pattern is found, the system automatically identifies the fault and initiates further analysis. Through fault pattern recognition, the system can quickly detect potential faults and provide guidance for subsequent detailed analysis.

[0046] The system learns from historical fault data, builds a fault mode library, and identifies common fault types and abnormal modes: The core innovation of this step is to introduce machine learning into the fault identification process. By learning from historical data, the system can not only summarize common fault types, but also automatically generate corresponding fault mode libraries. Unlike traditional fault detection methods that rely on manual rule definitions, machine learning enables the fault mode library to evolve and optimize automatically. In this way, the system can cope with a wider range of fault types, reduce the need for human intervention, and thus improve the intelligence and accuracy of fault detection.

[0047] Real-time monitoring of various data (such as call chains, logs, network traffic) and matching with known failure modes: In real-time operation, the system monitors data from different sources and matches it with known patterns in the failure mode library. This real-time monitoring and matching process can quickly discover potential failures and initiate response mechanisms. The beneficial effect of this step is that it allows the system to warn of failures in the early stages of their occurrence, preventing the problem from expanding to an uncontrollable level. In addition, because the system can perform comprehensive analysis through multi-source data (such as call chains, logs, network traffic), the detection results are more comprehensive and accurate.

[0048] When a matching abnormal pattern is found, the system automatically identifies the fault and starts further analysis: This step automatically identifies the abnormal situation that matches the fault pattern and starts the subsequent analysis process. Compared with traditional methods that rely on manual detection and judgment, automated identification can not only greatly improve response speed, but also reduce the probability of misjudgment. This design enables the system to identify and analyze faults in a timely and accurate manner, greatly improving operation and maintenance efficiency and system stability.

[0049] It should be noted that the first analysis can be multiple, such as a process of conducting a deeper root cause analysis and problem location after the system detects a fault or anomaly, rule-driven analysis, manual assisted analysis, distributed tracing enhancement and other methods. In this embodiment, when the system finds a matching abnormal pattern, it automatically identifies the fault and starts a further analysis process. This analysis process refers to a process of conducting a deeper root cause analysis and problem location after the system detects a fault or anomaly. The goal of this analysis process is to accurately identify the specific cause of the anomaly and provide information support for repairing the fault. Fault root cause analysis: The system determines the root cause of the fault by analyzing data from different sources (e.g., logs, call chains, network traffic). This may involve analysis of multiple microservices, system components, or network layers. Through cross-dimensional analysis, the system can narrow the scope of the fault and determine whether the fault is caused by a logical error in the service itself, an anomaly in the dependent service, or a bottleneck in the network communication. Call link tracing and log analysis are two key analysis tools that can help quickly determine the source of the fault.

[0050] Data correlation analysis: After a failure occurs, the system analyzes the relevant data streams to confirm whether the failure is a scalability issue or is related to other services or components. Data correlation analysis combines the current failure data with the historical pattern library to identify whether there is a historical record of similar failures and use previous repair measures as a reference.

[0051] Dynamically adjust strategies: During the analysis process, the system will dynamically adjust the monitoring and repair strategies based on the current service status and failure mode. For example, if the system detects a network layer failure, it may adjust the load balancing strategy or network traffic scheduling mechanism to ensure the stable operation of other parts of the system.

[0052] Automated repair suggestions: After analyzing the root cause of the fault, the system will generate a repair suggestion report, providing specific operation suggestions including service restart, resource reallocation, network adjustment, etc. At the same time, these suggestions can be automatically executed according to the severity level of the fault. For example, a minor service overload may be solved by automatically expanding resources or traffic scheduling, while a serious service downtime will trigger an alarm and recommend manual intervention.

[0053] In another feasible embodiment, the first detection adopts rule-driven analysis, manual assisted analysis, distributed tracing enhancement and other methods, including:

[0054] 1. Rule-driven analysis

[0055] Instead of the current machine learning-based analysis process, an analysis method based on predefined rules can be adopted. This method relies on expert systems to detect specific failure modes by setting a series of rules. For example, reasonable performance thresholds or event patterns can be set for each microservice, and when the system detects behaviors that deviate from these rules, the preset analysis process is triggered. Although this method has low flexibility, it can provide relatively efficient analysis in some stable systems, especially for scenarios with clear failure modes and less changes.

[0056] 2. Human-assisted analysis

[0057] In some complex faults or systems, automated analysis may not be sufficient to accurately identify the cause of the fault. Therefore, a human-assisted fault analysis system is an alternative. This approach combines automated monitoring and alarm systems and presents relevant data to operation and maintenance engineers for further manual analysis. This approach is suitable for highly complex or highly dynamic systems, especially when the abnormal pattern is complex or the automated analysis produces uncertain results. Human intervention can effectively reduce the risk of misjudgment.

[0058] 3. Distributed tracing enhancements

[0059] Instead of traditional single data analysis, enhanced distributed tracing technology can be used. Through denser tracing granularity (such as the specific response time of each request, the status of each service instance, etc.) and combined with timing analysis technology, more detailed root cause tracing can be performed. This method can capture each step in the entire service call chain in real time, and compare it horizontally with historical performance or other service instances to quickly locate the point of anomaly. This method is usually combined with eBPF and call chain tracing technology, and is suitable for complex microservice systems.

[0060] S3: Automatically warn and restore the faulty part.

[0061] Furthermore, intelligent fault location can quickly identify the root cause of the fault by comprehensively analyzing call chains, logs, and network traffic data.

[0062] The system locates the fault point by analyzing the call chain, logs and network traffic data across multiple services and generates a detailed fault location report.

[0063] The system conducts a comprehensive analysis of the call chain and identifies service nodes that behave abnormally in the chain (such as long response time or call failure). It combines service call data with log information to track the root cause of service failures and analyze whether it is an internal error in the service or a problem with dependent services. It combines network traffic monitoring with the service call chain to determine whether service call delays or failures are caused by network problems.

[0064] Intelligent fault location comprehensively analyzes all data to ensure rapid location of the fault source and help operation and maintenance personnel take effective repair measures.

[0065] It should be noted that the intelligent alarm and automatic recovery mechanism ensures that the system can respond and perform recovery operations in a timely manner when a fault occurs, minimizing the impact of the fault. When the system detects an abnormality or fault, it automatically triggers an alarm and performs fault recovery operations according to the preset strategy, reducing the operation and maintenance response time.

[0066] Based on the real-time data collected, the system triggers an alarm when an anomaly is detected. The alarm content includes the specific location of the fault, the scope of impact, and the recommended repair measures. The system automatically performs recovery operations based on the severity of the fault and the preset strategy, such as restarting services, redistributing traffic, adjusting loads, etc. The alarm information is displayed through a visual interface, and operation and maintenance personnel can directly view and intervene in the fault recovery process. Function: Through intelligent alarms and automatic recovery, the system can respond to faults in a timely manner and restore normal services as soon as possible, reducing the impact on business operations.

[0067] Embodiment 2, an embodiment of the present invention, provides an intelligent fault analysis and positioning method. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0068] First, the experiment simulates the operating environment of multiple service instances to test the performance of the present invention in log collection, network traffic monitoring, fault pattern identification and intelligent fault handling. The specific implementation of the experimental steps is as follows:

[0069] Deploy six service instances, each of which generates different amounts of log data and runs in different network environments. Use the ELK stack (Elasticsearch, Logstash, Kibana) to build a log collection system, and use eBPF technology to monitor network traffic. Install a traffic collector on the network node of each service instance to monitor the communication between services in real time, including performance indicators such as bandwidth utilization, network latency, and packet loss rate. Use Logstash to uniformly collect the generated log data on each service instance. The log collector is configured on each service instance to capture logs including service operation status, abnormal errors, and request processing information. Format and filter the collected log data to remove redundant information and ensure that the data transmitted to Elasticsearch is structured. Elasticsearch indexes the data for subsequent rapid retrieval and analysis. Real-time traffic monitoring is performed through the deployed traffic collector. Regularly capture the data packets transmitted between each service instance. Calculate key performance indicators such as bandwidth utilization, network latency, and packet loss rate, and generate real-time traffic reports. By analyzing this data, the system identifies potential network problems, such as service call delays caused by insufficient bandwidth or excessive packet loss rate. Use machine learning technology to build a fault mode library. The system learns common fault types and abnormal modes based on historical fault data and generates a fault mode library. Monitor log data, call chain data, and network traffic in real time and match them with known fault modes. When the system finds an anomaly that matches the fault mode, it automatically identifies the fault and starts further analysis. The system quickly locates the source of the fault by comprehensively analyzing the call chain, log, and network traffic data. The system analyzes the call chain across multiple service instances and identifies service nodes with long response times or failed calls. After the fault is located, the system automatically triggers the recovery mechanism and performs repair operations (such as service restart or traffic redistribution). The Kibana visualization interface displays detailed information on faults and recovery, and operation and maintenance personnel can view and intervene in the recovery process in real time. During the experiment, the log data volume, bandwidth utilization, network latency, packet loss rate, fault detection time, and automatic recovery time of each service instance are recorded in detail and stored in a table as the basis for analyzing the experimental results.

[0070] Table 1 Experimental data table

[0071]

[0072]

[0073] In this embodiment, the experiment focuses on testing the effectiveness of log collection, call link tracking, network traffic monitoring, fault pattern identification and intelligent fault location, and especially conducts quantitative analysis on the stability of network communication, fault detection and recovery time. The experiment verifies the effectiveness of the invention by monitoring the log data, network status and fault handling time of 6 different service instances.

[0074] 1. Impact of log data volume:

[0075] From the experimental data, there are certain differences in the amount of log data generated by different service instances, ranging from 1.7GB to 2.5GB. Through Logstash, these log data are uniformly collected, filtered and transmitted, and the system can process and analyze the logs of each instance in a timely manner, and the performance will not be affected by the difference in data volume. This shows that the log collection solution of the present invention can remain stable when dealing with different loads, proving its ability to efficiently process diversified data.

[0076] 2. The correlation between bandwidth utilization and network latency:

[0077] In the comparison between bandwidth utilization and network latency, service instances with higher bandwidth utilization (such as service instance 4, 90% bandwidth utilization) have better network latency performance (95ms latency). In comparison, service instances 1 and 5, which have lower bandwidth utilization, have higher latency (120ms and 130ms respectively). By deploying traffic collectors, the system can effectively monitor bandwidth utilization and latency, detect abnormal network conditions in a timely manner, and make adjustments. This shows that network traffic monitoring plays a significant role in optimizing bandwidth allocation and reducing latency.

[0078] 3. Relationship between packet loss rate and fault detection time:

[0079] The packet loss rate directly affects the fault detection time of the system. For example, the packet loss rate of service instance 5 is 0.15%, resulting in a longer fault detection time (20 seconds), while the fault detection time of service instance 4 with a lower packet loss rate (0.02% packet loss rate) is only 10 seconds. This result proves that by monitoring the performance of the network layer in real time, the system can identify faults caused by network problems in advance, thereby shortening the fault detection time.

[0080] 4. Effectiveness of Smart Recovery:

[0081] The data of automatic recovery time shows that the system can quickly take repair measures after detecting a fault. For example, the automatic recovery time of service instance 4 is only 25 seconds, while service instance 5 with a higher packet loss rate takes 45 seconds. Although the recovery time is related to the packet loss rate and detection time, overall, the system's intelligent alarm and automatic recovery mechanism can significantly reduce the fault handling time and ensure that the service is quickly restored to normal.

[0082] From the above data analysis, it can be seen that the present invention has significant advantages in log collection, network traffic monitoring and fault recovery. Compared with the existing technology, the present invention can process the log data of different service instances in real time and efficiently, and accurately monitor the key performance indicators in network communications (such as bandwidth, latency, packet loss rate), thereby identifying potential faults in advance. In addition, through intelligent fault pattern recognition and automated recovery mechanisms, the system can quickly respond to and repair faults, reduce system downtime, and improve operation and maintenance efficiency.

[0083] Example 3, reference Figure 2 , as an embodiment of the present invention, provides an intelligent fault analysis and positioning system, including a data acquisition and monitoring module, a first analysis and positioning module, and an automatic alarm and recovery module.

[0084] The data acquisition and monitoring module is used to acquire and monitor the data in the system, the first analysis and positioning module is used to identify the fault point service in the system, and the automatic alarm and recovery module is responsible for the real-time alarm and recovery operation of the fault.

[0085] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0086] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0087] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0088] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0089] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. An intelligent fault analysis and positioning method, characterized in that: include: Performing data collection and monitoring through the first device; Performing a first analysis based on the collected data; Automatically warn and restore faulty parts.

2. The intelligent fault analysis and location method according to claim 1, characterized in that: The data collection and monitoring by the first device includes uniformly collecting logs by the first device, and transmitting and indexing the log data.

3. The intelligent fault analysis and location method according to claim 2, characterized in that: The data collection and monitoring by the first device includes allocating an ID, recording response information, and capturing requests.

4. The intelligent fault analysis and location method according to claim 3, characterized in that: The first analysis based on the collected data includes deploying a flow collector, performing monitoring, outputting performance indicators, and performing real-time analysis on the data.

5. The intelligent fault analysis and location method according to claim 4, characterized in that: The first analysis based on the collected data includes learning historical fault data, establishing a fault mode library, identifying fault types, identifying faults and starting the first analysis.

6. The intelligent fault analysis and location method according to claim 5, characterized in that: The automatic alarm and recovery of the faulty part includes a comprehensive analysis of the call chain to identify the service nodes in the link; Combine service call data with log information to track the root cause of service failures; Combine network traffic monitoring with service call chain to judge service calls.

7. The intelligent fault analysis and location method according to claim 6, characterized in that: The automatic alarm and recovery of the faulty part includes detecting and triggering an alarm based on the collected real-time data; Automatically perform recovery actions based on the severity of the failure and preset policies.

8. A system using the intelligent fault analysis and location method according to any one of claims 1 to 7, characterized in that: It includes data collection and monitoring module, first analysis and positioning module, and automatic alarm and recovery module; The data acquisition and monitoring module is used to acquire and monitor data in the system; The first analysis and location module is used to identify the fault point service in the system; The automatic alarm and recovery module is used to be responsible for real-time alarm and recovery operations of faults.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the intelligent fault analysis and locating method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the intelligent fault analysis and locating method according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Fault processing method and device, medium and program product

    CN120524182A