Service failure detection method and system

Abstract

The application provides a service fault detection method and system, and relates to the technical field of code fault detection. The method comprises the following steps: analyzing system logs to obtain log data related to error reporting; determining whether an error reporting event described by the log data is a candidate error reporting event whose risk degree meets preset requirements; in response to the error reporting event being a candidate error reporting event, obtaining system running data in a specified period; determining whether the candidate error reporting event is a target error reporting event representing a service code defect in the system based on the log data and the system running data in the specified period; in response to the error reporting event being a target error reporting event, performing preliminary fault positioning based on the log data and the system running data in the specified period to obtain a preliminary fault positioning result; and performing secondary positioning of the fault in a service code data source based on at least the preliminary fault positioning result to determine a target service code that triggers the target error reporting event.

Description

Technical Field

[0001] This invention relates to the field of code fault detection technology, and in particular to a service fault detection method and system. Background Technology

[0002] With the widespread adoption of cloud computing and microservice architectures, the scale and complexity of computer system services have increased significantly. A single service often involves multi-node deployment and multi-module interaction, and the causes of failures have shifted from single hardware or software issues to the interplay of multiple factors. This places higher demands on the accuracy of fault detection, root cause tracing capabilities, and the practicality of remediation guidance. Current service fault detection technologies have evolved from traditional rule-based detection and machine learning-assisted detection to large-model-driven detection. However, all three types of technologies have significant limitations in practical applications, particularly in multi-source data integration and code correlation analysis, which directly restricts the effectiveness of fault detection and remediation efficiency.

[0003] Traditional rule-based fault detection technology relies entirely on manually preset fault rules. However, the scope of human experience is inherently limited – operations and maintenance personnel can only set detection logic for known faults they have encountered in the past. It is difficult to identify new faults, hidden faults, or fault scenarios that are not preset. This directly results in a very narrow fault detection range and an extremely low overall detection rate. A large number of potential faults cannot be detected in time, which can easily lead to service interruption risks. Although machine learning-assisted detection technology can overcome the limitations of manual rules and achieve fault identification by training models through historical data, its ability to process natural language is generally weak due to limitations in algorithm architecture. System logs, as one of the core data for fault detection, are mostly unstructured text. Machine learning models have difficulty accurately extracting deep fault features from this type of text. Ultimately, this leads to a low fault detection rate for unstructured log data. A large number of log error faults caused by code logic errors and implicit configuration problems are missed, which cannot meet the fault detection needs of complex services.

[0004] Even with current large-scale model-driven fault detection technologies, most solutions can only partially address the aforementioned issues and still have new limitations: either they simply input log error data into the model, failing to combine it with server resource data for cross-analysis; or they simply concatenate log and resource data, lacking structured preprocessing, preventing the model from establishing the logical relationship between system logs and server resources, thus affecting fault detection accuracy. More importantly, existing technologies generally lack an effective connection between faults and their related code. Even if a fault can be detected, the root cause at the code level cannot be located, resulting in low fault location accuracy and vague, impractical repair suggestions. This not only fails to efficiently resolve faults but may also lead operations personnel to perform ineffective operations such as blind expansion and repeated restarts, increasing service interruption risks and operational costs.

[0005] In summary, current service fault detection technologies face two major challenges: First, the lack of effective integration, preprocessing, and collaborative scheduling of multi-source data such as system logs and server resources prevents large models from performing deep inference based on cross-data, resulting in insufficient fault detection accuracy. Second, the lack of a precise correlation analysis mechanism between faults and system code makes it impossible to pinpoint the code root cause of faults, leading to vague and impractical repair suggestions that fail to meet the efficiency and accuracy requirements of complex service systems for fault handling. Therefore, there is an urgent need for a fault detection method that balances multi-source data collaborative analysis with precise code root cause localization. Summary of the Invention

[0006] To address the aforementioned technical problems, embodiments of the present invention provide a service failure detection method, comprising:

[0007] Analyze the system logs to obtain log data related to the error;

[0008] Determine whether the error event described in the log data is a candidate error event whose risk level meets preset requirements;

[0009] In response to the error event being a candidate error event, system operation data within a specified time period is obtained, wherein the specified time period includes the time period during which the error event occurred;

[0010] Based on the log data and the system operation data within the specified time period, determine whether the candidate error event is a target error event that characterizes a service code defect in the system;

[0011] In response to the error event being a target error event, preliminary fault location is performed based on the log data and system operation data for a specified period to obtain preliminary fault location results. The preliminary fault location results include information related to the fault and the reasoning logic of the preliminary location.

[0012] At least based on the preliminary fault location results, a secondary fault location is performed in the service code data source to determine the target service code that caused the target error event.

[0013] In one embodiment, the method further includes:

[0014] By accessing the service code data source through a specified protocol, all submission records of the service codes of each service module in the system and descriptive information of each service module can be obtained. The service code data source includes different versions of code data submitted by each service module at different times.

[0015] The step of performing secondary fault localization in the service code data source based at least on the preliminary fault localization results to determine the target service code that triggered the target error event includes:

[0016] Based on the preliminary fault location results, the submission records of each service module, and the descriptive information of each service module, a secondary fault location is performed in the service code data source to determine the target service code that caused the target error event.

[0017] In one embodiment, the secondary fault location based on the preliminary fault location results, the submission records of each service module, and the descriptive information of each service module in the service code data source, to determine the target service code that caused the target error event, includes:

[0018] Based on the descriptive information about the fault and the error reporting time in the preliminary fault location results, the target record corresponding to the information service type and time is determined from the submission records of each service module and the descriptive information of the service module.

[0019] Based on the target record, obtain the target version code of the target service module corresponding to the target record from the service code data source;

[0020] The pre-built high-parameter large model is invoked to perform correlation reasoning on the log data, system operation data, and target version code of the target service module, so as to perform secondary fault localization in the service code data source and determine the target service code that caused the target error event.

[0021] In one embodiment, the method further includes:

[0022] Based on the preliminary fault location results, the target version code is screened for code relevance to obtain target code whose relevance to the target error event meets the relevance requirements;

[0023] The target code is input into the high-parameter large model, enabling the high-parameter large model to perform correlation reasoning on the log data, system operation data, and target code.

[0024] In one embodiment, the method further includes:

[0025] Based on the log data, fault data is determined;

[0026] The fault data and target code are encapsulated and sent to the corresponding user's maintenance terminal.

[0027] In one embodiment, the method further includes:

[0028] The target service code is analyzed by invoking a pre-built large model that outputs remediation suggestions based on code defects to determine the target remediation suggestions;

[0029] Simultaneously, the fault data, target code, and target repair suggestions are encapsulated and sent to the operation and maintenance terminal.

[0030] In one embodiment, the method further includes:

[0031] The log parsing agent accesses the system log data source through a specified protocol and collects the system logs based on a preset frequency. The system logs include error logs and middleware operation logs.

[0032] The system logs are parsed to obtain log data related to the error, including:

[0033] The log parsing agent is used to analyze and process the unstructured system logs to obtain log data containing timestamps, error levels, error descriptions, error record contexts, other log contexts associated with the same time, and line numbers. The log data is structured data.

[0034] In one embodiment, determining whether the error event described by the log data is a candidate error event whose risk level meets a threshold requirement includes:

[0035] Determine the tag type corresponding to the log data;

[0036] Determine whether historical log data with the same tag type has been processed within a preset time period from the current time. If so, discard the current log data. If not, determine whether the error event described by the log data is a candidate error event whose risk level meets the threshold requirement.

[0037] If not, the corresponding system logs are stored locally.

[0038] In one embodiment, obtaining system operation data within a specified time period includes:

[0039] Obtain system operation data within 1 minute before and after the error report time, wherein the system operation data includes at least one of the following:

[0040] CPU utilization, memory usage, and disk I / O speed.

[0041] Another embodiment of the present invention also provides a service failure detection system, comprising:

[0042] The parsing module is used to parse system logs and obtain log data related to errors;

[0043] The first judgment module is used to determine whether the error event described by the log data is a candidate error event whose risk level meets the preset requirements;

[0044] The acquisition module is used to acquire system operation data within a specified time period in response to the error event being a candidate error event, wherein the specified time period includes the time period during which the error event occurred;

[0045] The second judgment module is used to determine whether the candidate error event is a target error event that characterizes a service code defect in the system, based on the log data and the system operation data within a specified time period.

[0046] The first positioning module is used to respond to the error event as a target error event, perform preliminary fault positioning based on the log data and system operation data of a specified period, and obtain preliminary fault positioning results. The preliminary fault positioning results include information related to the fault and the reasoning logic of the preliminary positioning.

[0047] The second positioning module is used to perform secondary fault positioning in the service code data source based at least on the preliminary fault positioning results, and to determine the target service code that caused the target error event.

[0048] Based on the above-disclosed content of the embodiments of this application, it can be seen that the embodiments of this application, through the logical link of "preliminary fault location - targeted code call - root cause in-depth analysis", can effectively avoid the full code redundancy occupying the model context, enabling the large model to accurately focus on the code modules related to the fault, locate the root cause of the fault at the code level, and finally generate repair suggestions that can directly guide operation and maintenance. While improving the accuracy of fault detection, it solves the dilemma of "poor detection and difficult repair" of existing technologies in two dimensions, and provides a more reliable code-level fault detection solution for complex service systems.

[0049] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0050] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0051] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating the service failure detection method in an embodiment of the present invention.

[0053] Figure 2 This is a flowchart illustrating a service failure detection method according to another embodiment of the present invention.

[0054] Figure 3 This is a flowchart illustrating a service failure detection method in another embodiment of the present invention.

[0055] Figure 4 This is a structural block diagram of the service failure detection system in an embodiment of the present invention. Detailed Implementation

[0056] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but these are not intended to limit the scope of the invention.

[0057] It should be understood that various modifications can be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope of this disclosure will be apparent to those skilled in the art.

[0058] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present disclosure and, together with the general description of the disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the disclosure.

[0059] These and other features of the invention will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0060] It should also be understood that although the invention has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of the invention, which have the features described in the claims and are therefore all within the scope of protection defined herein.

[0061] The above and other aspects, features and advantages of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0062] Specific embodiments of the present disclosure are described thereafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure and can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the present disclosure. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but merely to serve as the basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in a variety of substantially any suitable detailed structures.

[0063] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in still another embodiment,” all of which may refer to one or more of the same or different embodiments according to this disclosure.

[0064] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0065] like Figure 1 As shown, an embodiment of the present invention provides a service failure detection method, including:

[0066] S1: Parse the system logs to obtain log data related to the error;

[0067] S2: Determine whether the error event described by the log data is a candidate error event whose risk level meets the preset requirements;

[0068] S3: In response to the error event being a candidate error event, obtain system operation data within a specified time period, wherein the specified time period includes the time period during which the error event occurred;

[0069] S4: Based on the log data and the system operation data within the specified time period, determine whether the candidate error event is a target error event that characterizes a service code defect in the system;

[0070] S5: In response to the error event being a target error event, perform preliminary fault location based on the log data and system operation data for a specified period to obtain preliminary fault location results. The preliminary fault location results include information related to the fault and the reasoning logic of the preliminary location.

[0071] S6: At least based on the preliminary fault location results, perform secondary fault location in the service code data source to determine the target service code that caused the target error event.

[0072] The solution in this embodiment can be applied to a service system. The specific service type of this service system is not fixed, and it may include multiple different intelligent agents with different hardware and software configurations to cooperate in executing the above solution. Alternatively, it can all be executed by a single intelligent agent or server, depending on the specific circumstances. This embodiment uses the service system as the executing entity for illustration.

[0073] The solution presented in this embodiment specifically addresses the core problems of existing service fault detection technologies: low fault detection rate, poor location accuracy, and difficulty in implementing repair suggestions. It meets the efficiency and accuracy requirements of complex service systems for fault handling. Addressing the shortcomings of traditional rule-based detection, which relies on human experience and has a narrow fault detection range, as well as the limitations of machine learning technology in processing unstructured logs and the inability to collaboratively analyze multi-source data, this embodiment enables structured preprocessing of system logs and dynamic scheduling of server resource data. This achieves the collaborative integration of these two types of core data, providing a comprehensive and relevant analytical data foundation for subsequent large-scale model calls, thereby improving the service fault detection rate and initial location accuracy. Meanwhile, addressing the problem that existing fault repair suggestions are vague and difficult to implement, this embodiment further establishes a logical link of "preliminary fault location - targeted code invocation - root cause in-depth analysis". This can effectively avoid the redundancy of all code occupying the model context, enabling the large model to accurately focus on the code modules related to the fault, locate the root cause of the fault at the code level, and finally generate repair suggestions that can directly guide operation and maintenance. While improving the accuracy of fault detection, it solves the dilemma of "poor detection and difficult repair" of existing technologies from two dimensions, providing a more reliable fault detection solution for complex service systems.

[0074] In practice, the technical solution of this embodiment can be summarized as a design centered around "fault detection based on multi-source data + root cause analysis of code," constructing a four-layer architecture of "data layer - collaboration layer - analysis layer - output layer." The data layer provides multi-source data, including logs and server resources. The collaboration layer integrates multi-source data to solve the problem of data fragmentation. The analysis layer enables more accurate service fault detection based on multi-source data, improving detection accuracy. The analysis layer can also associate fault codes for targeted analysis to solve the problem of inaccurate fault location. Finally, complete fault detection and repair suggestions are sent to the output layer for presentation to operations and maintenance personnel, achieving end-to-end optimization of fault "high-accuracy detection - rapid root cause analysis - implementable repair."

[0075] Specifically, in one embodiment, the method further includes:

[0076] S7: The log parsing agent accesses the system log data source through a specified protocol and collects the system logs based on a preset frequency. The system logs include error logs and middleware operation logs.

[0077] The system logs are parsed to obtain log data related to the error, including:

[0078] S101: The log parsing agent is used to analyze and process the unstructured system logs to obtain log data containing timestamps, error levels, error descriptions, error record contexts, other log contexts associated with the same time, and line numbers. The log data is structured data.

[0079] For example, the log parsing agent accesses the log data source (ELK) and collects system logs in real time at a frequency of once every 30 seconds (or other frequencies, depending on the specific frequency). The collected logs include application error logs and middleware runtime logs. Next, the log parsing agent preprocesses the collected unstructured logs and, for error log records, extracts the "timestamp, error level, error description, context surrounding the error record, related log contexts from the same time, and line number," and processes this information into structured data, forming the log data mentioned above.

[0080] Furthermore, determining whether the error event described by the log data is a candidate error event whose risk level meets the threshold requirement includes:

[0081] S201: Determine the tag type corresponding to the log data;

[0082] S202: Determine whether historical log data of the same tag type has been processed within a preset time period from the current time. If so, discard the current log data. If not, determine whether the error event described by the log data is a candidate error event whose risk level meets the threshold requirement.

[0083] S203: If not, store the corresponding system log locally.

[0084] For example, the above process can be executed by a coordinating agent in the system (service system). After receiving log data, the coordinating agent inputs it into a built-in low-parameter large model. This model will categorize the log data by tag. If a tag appears repeatedly within half an hour, it is considered a duplicate alarm, and no further action is taken. If it is not a duplicate alarm, a preset risk assessment prompt can be used to determine the risk. If the error event is a high-risk event, a concurrent data query of the resource monitoring agent is triggered; if it is a low-risk event, only the corresponding log is stored locally, and the subsequent fault location process is not triggered.

[0085] In response to a high-risk error event, the coordinating agent and the resource monitoring agent send a system operation data query request. For example, obtaining system operation data within a specified time period includes:

[0086] S301: Obtain system operation data within 1 minute before and after the error reporting time, wherein the system operation data includes at least one of the following:

[0087] CPU utilization, memory usage, and disk I / O speed.

[0088] Of course, system operation data is not limited to the above-mentioned information; it can also include real-time indicators such as network bandwidth and process liveness status. Specifically, for example, the resource monitoring agent, based on a query request, accesses the resource data source (Prometheus) via the MCP protocol to collect real-time resource indicator data within a specified time range, such as data within ±1 minute of the log alarm time. This data includes resource indicators matching the required resource data and their corresponding timestamps. The resource monitoring agent then returns the collected "timestamp-resource indicator data" key-value pairs to the coordinating agent, ensuring a perfect match between the data and the log alarm time. After obtaining the key-value pairs, the coordinating agent, without performing additional data processing such as sorting / alignment, can directly integrate the log data and resource data in a "one-to-one timestamp correspondence" manner to form a structured data packet, and push the integrated "log + resource" data packet to the specific execution party in subsequent processes.

[0089] In one embodiment, the method further includes:

[0090] S8: Access the service code data source through a specified protocol to obtain all submission records of the service codes of each service module in the system and descriptive information of each service module. The service code data source includes different versions of code data submitted by each service module at different times.

[0091] The step of performing secondary fault location in the service code data source based at least on the preliminary fault location results and the descriptive information of each of the service modules to determine the target service code that triggered the target error event includes:

[0092] S10: Based on the preliminary fault location results, the submission records of each service module, and the descriptive information of each service module, perform secondary fault location in the service code data source to determine the target service code that caused the target error event.

[0093] The step of performing secondary fault location based on the preliminary fault location results, the submission records of each service module, and the descriptive information of each service module in the service code data source to determine the target service code that triggered the target error event includes:

[0094] S1001: Based on the descriptive information about the fault and the error reporting time in the preliminary fault location results, determine the target record that matches the information service type and time in the submission records of each service module and the descriptive information of the service module.

[0095] S1002: Based on the target record, obtain the target version code of the target service module corresponding to the target record from the service code data source;

[0096] S1003: Call the pre-built high-parameter large model to perform correlation reasoning on the log data, system operation data, and target version code of the target service module, so as to perform secondary fault localization in the service code data source and determine the target service code that caused the target error event.

[0097] As an optional embodiment, such as Figure 2 As shown, the method further includes:

[0098] S11: Based on the preliminary fault location results, the target version code is screened for code relevance to obtain target code whose relevance to the target error event meets the relevance requirements;

[0099] S12: Input the target code into the high-parameter large model, so that the high-parameter large model can perform correlation reasoning on the log data, system operation data and target code.

[0100] Specifically, in the analysis architecture, the large analysis model first performs inference based on integrated data to initially locate the direction of the fault, i.e., to perform preliminary fault localization, determine the possible causes of the fault, and generate descriptive information about the fault accordingly. Then, it sends a code query request to the code agent in the system, containing a description of the fault direction and a specified error time. Upon receiving the query request, the code agent accesses the Git code data source via the MCP protocol (the specific protocol is not fixed) and requests a "list of code for all service code modules and Git commit records". Next, the code agent uses a built-in low-parameter large model to match the fault description and error time in the query request with the description in the code list to determine the target record. Then, it uses the MCP protocol to retrieve the target version service code of the matching target service module from the Git code data source. Finally, the analysis layer calls the low-parameter large model to filter the target version service code based on the fault description, removing irrelevant code and retaining only the core code relevant to the fault. Finally, the analysis layer inputs the retained code (target code), log data, and system operation data into a high-parameter large model. By leveraging the model's powerful semantic understanding and logical reasoning capabilities, it performs correlation and reasoning on the data from the above three dimensions to accurately locate the root cause of the fault at the code level and avoid misjudgments caused by single data.

[0101] Furthermore, such as Figure 3 As shown, the method further includes:

[0102] S13: Determine fault data based on the log data;

[0103] S14: Encapsulate the fault data and target code, and send them to the corresponding user's operation and maintenance terminal.

[0104] as well as

[0105] S15: Invoke a pre-built neural network model for outputting repair suggestions based on code defects to analyze the target service code and determine the target repair suggestions;

[0106] S16: Simultaneously encapsulate the fault data, target code, and target repair suggestions, and send them to the operation and maintenance terminal.

[0107] For example, in this embodiment, after obtaining the target code, i.e., the defective service code, log data is encapsulated with the target code, or fault data that allows users to quickly obtain fault information is determined based on the log data and encapsulated with the target code. The encapsulated data is then transmitted to the user's maintenance terminal for viewing. Alternatively, to simultaneously provide users with repair suggestions and assist them in quickly determining repair strategies, the analysis layer in this embodiment also has a pre-built high-parameter large model used to output repair suggestions based on code defects. These repair suggestions can be encapsulated together and then sent to the user. The maintenance terminal can be, but is not limited to, a web management platform / mobile app. After receiving the data packet, the maintenance terminal displays the fault results and repair suggestions in a visual interface, allowing maintenance personnel to intuitively obtain fault information, quickly perform repair operations, and reduce manual troubleshooting costs.

[0108] As can be seen from the content disclosed in the above embodiments, the overall architecture of this application includes a four-layer structure, specifically:

[0109] Data Layer: Provides foundational data support for fault detection, comprising two core data sources. First, a system log data source, connecting to the service log storage system (ELK), covering application runtime logs (including error descriptions, line numbers, and process IDs) and middleware logs (including database and message queue connection / read / write logs). Second, a server resource data source, connecting to server monitoring tools (Prometheus), covering real-time metrics such as CPU utilization, memory usage, disk I / O throughput, network bandwidth, and process liveness status. Third, a code data source, connecting to code repositories (Git), storing the associated source code (categorized by module, including version history and commit records).

[0110] The collaboration layer, serving as the core for data integration and scheduling, deploys four types of agents: log parsing agents, resource monitoring agents, coordination agents, and code agents. The log parsing agent is responsible for structured preprocessing of logs; the resource monitoring agent is responsible for resource data collection; the coordination agent has a built-in low-parameter, large-scale model for data filtering and cross-agent scheduling; and the code agent operates independently and has a built-in large-scale model for targeted acquisition and initial organization of code modules. All agents communicate bidirectionally with the data layer via the MCP protocol.

[0111] Analysis Layer: The core is the large language model module, which includes a "fault detection submodule" and a "code root cause analysis submodule". The former detects whether a fault has occurred based on multi-source data integrated by the collaboration layer through semantic reasoning; the latter, based on the detection results of the former, calls the target module in the code data source and uses the powerful semantic understanding and logical reasoning capabilities of the large language model to perform logical analysis on the code and determine the code-level root cause of the fault.

[0112] Output Layer: Responsible for result visualization and feedback, including the operations and maintenance terminal (operations and maintenance management platform, mobile APP) and the repair suggestion generation module. Based on the root cause conclusions of the analysis layer, the repair suggestion generation module generates repair suggestions in the form of "problem description - root cause code location - operation steps - verification method", which are simultaneously pushed to the operations and maintenance terminal for operations and maintenance personnel to execute directly.

[0113] Based on the above, this application, by constructing the aforementioned four-layer architecture, can dynamically schedule log parsing agents and resource monitoring agents to obtain concurrent related data through the low-parameter large model built into the collaboration layer combined with the MCP protocol, and accurately detect service faults based on multi-source data. Simultaneously, relying on independent code agents, it uses low-parameter large model-based targeted matching and simplifies fault-related code, then uses a layered large model (low-parameter scheduling, high-parameter analysis) to complete the root cause localization of faults. Based on this, it not only solves the problem of low fault detection rate in traditional technologies, but also deeply analyzes the root cause of faults through code association, effectively improving the initial fault detection accuracy and the depth of code root cause localization, shortening fault repair time, and reducing the manpower cost for operations and maintenance personnel to investigate data and code.

[0114] like Figure 4 As shown, another embodiment of the present invention also provides a service failure detection system, including:

[0115] The parsing module is used to parse system logs and obtain log data related to errors;

[0116] The first judgment module is used to determine whether the error event described by the log data is a candidate error event whose risk level meets the threshold requirement;

[0117] The acquisition module is used to acquire system operation data within a specified time period in response to the error event being a candidate error event, wherein the specified time period includes the time period during which the error event occurred;

[0118] The second judgment module is used to determine whether the candidate error event is a target error event that characterizes a service code defect in the system, based on the log data and the system operation data within a specified time period.

[0119] The first positioning module is used to respond to the error event as a target error event, perform preliminary fault positioning based on the log data and system operation data of a specified period, and obtain preliminary fault positioning results. The preliminary fault positioning results include information related to the fault and the reasoning logic of the preliminary positioning.

[0120] The second positioning module is used to perform secondary fault positioning in the service code data source based at least on the preliminary fault positioning results, and to determine the target service code that caused the target error event.

[0121] In one embodiment, the device further includes:

[0122] The acquisition module is used to access the service code data source through a specified protocol to obtain all submission records of the service codes of each service module in the system. The service code data source includes different versions of code data submitted by each service module at different times.

[0123] The step of performing secondary fault localization in the service code data source based at least on the preliminary fault localization results to determine the target service code that triggered the target error event includes:

[0124] Based on the preliminary fault location results and the submission records of each service module, a secondary fault location is performed in the service code data source to determine the target service code that caused the target error event.

[0125] In one embodiment, the secondary fault location based on the preliminary fault location results and the submission records of each service module in the service code data source, to determine the target service code that triggered the target error event, includes:

[0126] Based on the descriptive information about the fault and the error reporting time in the preliminary fault location results, the target record corresponding to the information service type and time is determined in the submission records of each of the service modules.

[0127] Based on the target record, obtain the target version code of the target service module corresponding to the target record from the service code data source;

[0128] The pre-built high-parameter large model is invoked to perform correlation reasoning on the log data, system operation data, and target version code of the target service module, so as to perform secondary fault localization in the service code data source and determine the target service code that caused the target error event.

[0129] In one embodiment, the device further includes:

[0130] The filtering module is used to filter the target version code based on the preliminary fault location results to obtain target code whose relevance to the target error event meets the relevance threshold.

[0131] The input module is used to input the target code into the high-parameter large model, so that the high-parameter large model can perform correlation reasoning on the log data, system operation data, and target code.

[0132] In one embodiment, the device further includes:

[0133] The determination module is used to determine fault data based on the log data;

[0134] The encapsulation module is used to encapsulate the fault data and target code and send them to the corresponding user's operation and maintenance terminal.

[0135] In one embodiment, the device further includes:

[0136] The calling module is used to call a pre-built neural network model that outputs repair suggestions based on code defects to analyze the target service code and determine the target repair suggestions;

[0137] Simultaneously, the fault data, target code, and target repair suggestions are encapsulated and sent to the operation and maintenance terminal.

[0138] In one embodiment, the parsing module is further configured to:

[0139] The log parsing agent accesses the system log data source through a specified protocol and collects the system logs based on a preset frequency. The system logs include error logs and middleware operation logs.

[0140] The system logs are parsed to obtain log data related to the error, including:

[0141] The log parsing agent is used to analyze and process the unstructured system logs to obtain log data containing timestamps, error levels, error descriptions, error record contexts, other log contexts associated with the same time, and line numbers. The log data is structured data.

[0142] In one embodiment, determining whether the error event described by the log data is a candidate error event whose risk level meets a threshold requirement includes:

[0143] Determine the tag type corresponding to the log data;

[0144] Determine whether historical log data with the same tag type has been processed within a preset time period from the current time. If so, discard the current log data. If not, determine whether the error event described by the log data is a candidate error event whose risk level meets the threshold requirement.

[0145] If not, the corresponding system logs are stored locally.

[0146] In one embodiment, obtaining system operation data within a specified time period includes:

[0147] Obtain system operation data within 1 minute before and after the error report time, wherein the system operation data includes at least one of the following:

[0148] CPU utilization, memory usage, and disk I / O speed.

[0149] Another embodiment of the present invention also provides an electronic device, comprising:

[0150] One or more processors;

[0151] Memory, configured to store one or more programs;

[0152] When the one or more programs are executed by the one or more processors, the one or more processors implement the service failure detection method as described in any one of the above descriptions.

[0153] Furthermore, one embodiment of the present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the service failure detection method described above. It should be understood that the various solutions in this embodiment have the corresponding technical effects in the above method embodiments, and will not be repeated here.

[0154] Furthermore, embodiments of the present invention also provide a computer program product, which is tangibly stored on a computer-readable medium and includes computer-readable instructions that, when executed, cause at least one processor to perform a service failure detection method such as those described in the embodiments above.

[0155] It should be noted that the computer storage medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program configured for use by or in connection with an instruction execution system, system, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, antenna, optical fiber, RF, etc., or any suitable combination thereof.

[0156] Furthermore, those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0157] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0158] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0159] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.

Claims

1. A service failure detection method, characterized in that, include: Analyze the system logs to obtain log data related to the error; Determine whether the error event described in the log data is a candidate error event whose risk level meets preset requirements; In response to the error event being a candidate error event, system operation data within a specified time period is obtained, wherein the specified time period includes the time period during which the error event occurred; Based on the log data and the system operation data within the specified time period, determine whether the candidate error event is a target error event that characterizes a service code defect in the system; In response to the error event being a target error event, preliminary fault location is performed based on the log data and system operation data for a specified period to obtain preliminary fault location results. The preliminary fault location results include information related to the fault and the reasoning logic of the preliminary location. At least based on the preliminary fault location results, a secondary fault location is performed in the service code data source to determine the target service code that caused the target error event; The method further includes: By accessing the service code data source through a specified protocol, all submission records of the service codes of each service module in the system and descriptive information of each service module are obtained. The service code data source includes different versions of code data submitted by each service module at different times. The step of performing secondary fault localization in the service code data source based at least on the preliminary fault localization results to determine the target service code that triggered the target error event includes: Based on the preliminary fault location results, the submission records of each service module, and the descriptive information of each service module, a secondary fault location is performed in the service code data source to determine the target service code that caused the target error event.

2. The service failure detection method according to claim 1, characterized in that, The secondary fault location based on the preliminary fault location results, the submission records of each service module, and the descriptive information of each service module, in the service code data source, to determine the target service code that triggered the target error event, includes: Based on the descriptive information about the fault and the error time in the preliminary fault location results, the target record corresponding to the service type and time is determined from the submission records and descriptive information of each service module. Based on the target record, obtain the target version code of the target service module corresponding to the target record from the service code data source; The pre-built high-parameter large model is invoked to perform correlation reasoning on the log data, system operation data, and target version code of the target service module, so as to perform secondary fault localization in the service code data source and determine the target service code that caused the target error event.

3. The service failure detection method according to claim 2, characterized in that, The method further includes: Based on the preliminary fault location results, the target version code is screened for code relevance to obtain target code whose relevance to the target error event meets the relevance requirements; The target code is input into the high-parameter large model, enabling the high-parameter large model to perform correlation reasoning on the log data, system operation data, and target code.

4. The service failure detection method according to claim 1, characterized in that, The method further includes: Based on the log data, fault data is determined; The fault data and target code are encapsulated and sent to the corresponding user's maintenance terminal.

5. The service failure detection method according to claim 4, characterized in that, The method further includes: The target service code is analyzed by invoking a pre-built large model that outputs remediation suggestions based on code defects to determine the target remediation suggestions; Simultaneously, the fault data, target code, and target repair suggestions are encapsulated and sent to the operation and maintenance terminal.

6. The service failure detection method according to claim 1, characterized in that, The method further includes: The log parsing agent accesses the system log data source through a specified protocol and collects the system logs based on a preset frequency. The system logs include error logs and middleware operation logs. The system logs are parsed to obtain log data related to the error, including: The log parsing agent is used to analyze and process the unstructured system logs to obtain log data containing timestamps, error levels, error descriptions, error record contexts, other log contexts associated with the same time, and line numbers. The log data is structured data.

7. The service failure detection method according to claim 1, characterized in that, Determining whether the error event described by the log data is a candidate error event whose risk level meets the threshold requirement includes: Determine the tag type corresponding to the log data; Determine whether historical log data with the same tag type has been processed within a preset time period from the current time. If so, discard the current log data. If not, determine whether the error event described by the log data is a candidate error event whose risk level meets the threshold requirement. If not, the corresponding system logs are stored locally.

8. The service failure detection method according to claim 1, characterized in that, The acquisition of system operation data within a specified time period includes: Obtain system operation data within 1 minute before and after the error report time, wherein the system operation data includes at least one of the following: CPU utilization, memory usage, and disk I / O speed.

9. A service failure detection system, characterized in that, include: The parsing module is used to parse system logs and obtain log data related to errors; The first judgment module is used to determine whether the error event described by the log data is a candidate error event whose risk level meets the preset requirements; The acquisition module is used to acquire system operation data within a specified time period in response to the error event being a candidate error event, wherein the specified time period includes the time period during which the error event occurred; The second judgment module is used to determine whether the candidate error event is a target error event that characterizes a service code defect in the system, based on the log data and the system operation data within a specified time period. The first positioning module is used to respond to the error event as a target error event, perform preliminary fault positioning based on the log data and system operation data of a specified period, and obtain preliminary fault positioning results. The preliminary fault positioning results include information related to the fault and the reasoning logic of the preliminary positioning. The second positioning module is used to perform secondary fault positioning in the service code data source based at least on the preliminary fault positioning results, and to determine the target service code that caused the target error event. The system also includes: The acquisition module is used to access the service code data source through a specified protocol to obtain all the submission records of the service codes of each service module in the system. The service code data source includes different versions of code data submitted by each service module at different times. The step of performing secondary fault localization in the service code data source based at least on the preliminary fault localization results to determine the target service code that triggered the target error event includes: Based on the preliminary fault location results and the submission records of each service module, a secondary fault location is performed in the service code data source to determine the target service code that caused the target error event.

Images