Problem positioning method, device, equipment, medium and product based on log analysis

Abstract

The application provides a problem positioning method and device based on log analysis, equipment, medium and product, relates to the technical field of big data, and specifically relates to the technical field of artificial intelligence and big data. The method comprises the following steps: in response to detecting an abnormal event, acquiring log data; obtaining a plurality of log records with the same call chain identifier in the log data; the call chain identifier is used for associating log records generated when the same business request flows between different service groups; generating a root cause hypothesis based on the time sequence and semantic relationship between the plurality of log records through a target model; the root cause hypothesis represents the problem position and reason causing the abnormal event.

Description

Technical Field

[0001] This application relates to the field of big data technology, specifically to the field of artificial intelligence and big data technology, and more specifically to a problem localization method, device, electronic device, storage medium and program product based on log analysis. Background Technology

[0002] In banking systems, with the widespread application of microservice architecture and distributed systems, a single customer transaction request often needs to go through the collaborative processing of multiple service components such as account service, payment service, and risk control service. When the system encounters an anomaly, the relevant log records are scattered across different service nodes. Log analysis technologies are usually based on keyword matching or predefined rules for anomaly detection. However, when faced with complex call scenarios across service components, it is often difficult to accurately locate the root cause of the problem, affecting the rapid recovery of banking operations. Summary of the Invention

[0003] In view of the above problems, embodiments of this application provide a problem localization method, apparatus, electronic device, storage medium, and program product based on log analysis.

[0004] According to a first aspect of this application, a problem localization method based on log analysis is provided, comprising: in response to the detection of an abnormal event, acquiring log data; obtaining multiple log records with the same call chain identifier in the log data; generating a root cause hypothesis based on the temporal order and semantic relationship between the multiple log records through a target model; the root cause hypothesis characterizing the problem location and cause of the abnormal event; and the call chain identifier being used to associate log records generated when the same business request flows between different service groups.

[0005] According to an embodiment of this application, obtaining multiple log records with the same call chain identifier from log data includes: extracting the target call chain identifier associated with the abnormal event from the log data; retrieving all log records in the log data whose call chain identifier field value matches the target call chain identifier based on the target call chain identifier; sorting all log records in chronological order according to their respective timestamps, and selecting a subset of log records whose time range covers the time of the abnormal event as multiple log records; wherein, the subset of log records includes log records within a preset time window before the time of the abnormal event and log records within a preset time window after the time of the abnormal event; the timestamp represents the time point when the log record was generated.

[0006] According to embodiments of this application, a root cause hypothesis is generated based on the temporal order and semantic relationships between multiple log records using a target model. This includes: performing semantic parsing on each log record using the target model to identify at least one semantic element among error type, error location, affected object, and severity; constructing an event timeline chain based on the temporal order of the multiple log records, whereby the event timeline chain represents the evolutionary sequence formed by the events corresponding to the multiple log records in chronological order; performing causal reasoning based on the correlation between the semantic elements corresponding to each event in the event timeline chain using the target model to determine the problem location of the abnormal event and the cause of the abnormal event; the correlation includes at least one of the triggering relationship between the error type in a preceding event and the error type in a subsequent event, and the propagation relationship between the affected object in a preceding event and the error location in a subsequent event; and generating a root cause hypothesis based on the problem location and cause.

[0007] According to embodiments of this application, the method further includes: acquiring system monitoring indicator data associated with the abnormal event, wherein the system monitoring indicator data includes time-series values ​​of at least one performance indicator among processor utilization, memory utilization, network latency, and request throughput within a preset time window before and after the occurrence of the abnormal event; performing abnormal pattern recognition on the system monitoring indicator data through a target model to determine the target performance indicator experiencing abnormal fluctuations and the corresponding time of abnormal fluctuations; determining whether the time of abnormal fluctuations is earlier than the timestamp of the first log record in the event time-series chain; if the time of abnormal fluctuations is earlier than the timestamp of the first log record, determining that the system resource corresponding to the target performance indicator is the problem location of the abnormal event, and using the abnormal state of the target performance indicator as a pre-triggering factor in causal reasoning.

[0008] According to embodiments of this application, the above method further includes: obtaining the confidence level corresponding to the root cause hypothesis generated by the target model; if the confidence level does not meet the preset conditions, obtaining verification data; and updating the root cause hypothesis and / or confidence level by the target model based on the verification data so that the confidence level meets the preset conditions.

[0009] According to an embodiment of this application, obtaining verification data includes: generating a verification plan based on root cause hypotheses using a target model; the verification plan includes one or more verification instructions for verifying or disproving root cause hypotheses, each verification instruction being associated with a target information type to be acquired and a corresponding information source location; acquiring data corresponding to the target information type from at least one of a monitoring indicator database, a log storage system, and a configuration management database, based on the information source location corresponding to each verification instruction in the verification plan, as verification data; the monitoring indicator database stores system performance indicators and service health status data, and the configuration management database stores the status information of system configuration items.

[0010] According to embodiments of this application, the preset conditions include at least one of the following: the confidence level reaches a preset threshold; the verification data contains confirmatory evidence that matches the problem location and cause in the root cause hypothesis, and the confirmatory evidence includes error log records consistent with the error type pointed to by the root cause hypothesis, monitoring indicator data consistent with the resource exhaustion state pointed to by the root cause hypothesis, or configuration item status information consistent with the configuration error pointed to by the root cause hypothesis; and the number of iterative verifications for the root cause hypothesis reaches a preset upper limit.

[0011] According to a second aspect of this application, a problem localization device based on log analysis is provided, comprising: a log acquisition module for acquiring log data in response to detecting an abnormal event; a log association module for acquiring multiple log records with the same call chain identifier in the log data; the call chain identifier is used to associate log records generated when the same business request flows between different service groups; and a root cause reasoning module for generating root cause hypotheses based on the temporal order and semantic relationship between multiple log records through a target model; the root cause hypothesis characterizes the problem location and cause of the abnormal event.

[0012] According to a third aspect of this application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0013] According to a fourth aspect of this application, a computer-readable storage medium is also provided, on which a computer program or instructions are stored, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0014] According to a fifth aspect of this application, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0015] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1 The illustration shows an application scenario diagram of a log analysis-based problem localization method, apparatus, device, medium, and program product according to embodiments of this application;

[0017] Figure 2 This illustration shows one of the flowcharts of a log analysis-based problem localization method according to an embodiment of this application;

[0018] Figure 3A second flowchart illustrating a problem localization method based on log analysis according to an embodiment of this application is shown schematically.

[0019] Figure 4 A flowchart of a problem localization method based on log analysis according to an embodiment of this application is illustrated in part three.

[0020] Figure 5 A flowchart of a problem localization method based on log analysis according to an embodiment of this application is illustrated in the schematic diagram.

[0021] Figure 6 This schematic diagram illustrates the structural block diagram of a log analysis-based problem location device according to an embodiment of this application;

[0022] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a log analysis-based problem localization method according to an embodiment of this application. Detailed Implementation

[0023] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0025] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0026] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0027] Figure 1The illustration schematically depicts application scenarios of a log-based problem localization method, apparatus, electronic device, storage medium, and program product according to embodiments of this application. For example... Figure 1 As shown, application scenario 100 according to an embodiment of this application may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables. For example, bank maintenance personnel can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive abnormal event alarms, query root cause analysis results, or send fault handling instructions, etc.

[0028] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be electronic devices such as smartphones, wearable devices, personal computers, tablets, and workstations. Clients for log analysis and operation and maintenance management applications can be installed on these terminal devices. These applications may include, but are not limited to, application performance monitoring platform clients, distributed tracing system clients, intelligent operation and maintenance management tools, log analysis platform clients, etc. (these are just examples). Furthermore, this application embodiment does not limit the form of the application, and may include, but is not limited to, applications, mini-programs, etc., installed on the terminal devices, or even web pages.

[0029] Server 105 can be a server providing log analysis-based problem localization services, such as a backend server providing intelligent fault diagnosis support for a bank's microservice architecture business system (this is just an example). Server 105 can receive log data from distributed service nodes and abnormal event detection results from a monitoring system, and perform correlation analysis on multiple log records with the same call chain identifier in the log data. Based on the temporal order and semantic relationships between these multiple log records, a root cause hypothesis is generated using a target model, and then the root cause hypothesis (e.g., problem location, cause of the anomaly, scope of impact, call chain details, etc.) is fed back to the terminal device. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, content delivery networks, and basic cloud computing services such as big data and artificial intelligence. The server can be a backend server of a bank's intelligent operation and maintenance platform, used to provide backend services for clients of log analysis and operation and maintenance management applications.

[0030] It should be noted that the log analysis-based problem localization method provided in this application embodiment can generally be executed by server 105 and / or terminal devices 101-103. Correspondingly, the log analysis-based problem localization device provided in this application embodiment can generally be set in server 105 and / or terminal devices 101-103. In some embodiments, server 105 may be a server deployed with log collection components and target models, directly performing log correlation analysis and root cause reasoning localization locally; in other embodiments, server 105 may be an independent log analysis server, collecting log data from various distributed service nodes and performing root cause localization analysis.

[0031] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0032] Figure 2 A flowchart illustrating one of the problem localization methods based on log analysis according to an embodiment of this application is shown. Figure 2 As shown, the problem localization method based on log analysis according to the embodiments of this application may include steps S210 to S230.

[0033] Step S210: In response to the detection of an abnormal event, log data is obtained.

[0034] Step S220: Obtain multiple log records with the same call chain identifier from the log data; the call chain identifier is used to associate log records generated when the same business request flows between different service groups.

[0035] Step S230: Generate root cause hypotheses based on the temporal order and semantic relationships between multiple log records using the target model; root cause hypotheses characterize the location and cause of the problem leading to the abnormal event.

[0036] In step S210, an abnormal event refers to a situation that deviates from the normal state during system operation, and is used to identify faults or performance degradation phenomena that require root cause analysis.

[0037] For example, abnormal events may include, but are not limited to: service response timeout, interface call failure, system resource exhaustion alarm, sudden increase in user request error rate, database connection abnormality, application process crash, and business transaction failure.

[0038] Similarly, log data refers to the collection of status and event information recorded by various components of the system during operation.

[0039] In one feasible implementation, an abnormal event can be detected in real time by a monitoring system. When the monitoring system finds that the service response time exceeds a preset threshold or the error rate exceeds the normal range, it triggers an abnormal event notification. After responding to the notification, the system retrieves log data related to the time period of the abnormal event from the log storage system.

[0040] In another feasible implementation, the abnormal event can be manually triggered by the user. The user enters a description of the fault or selects a specific abnormal time point through the operation interface. The system determines the time range and impact range of the abnormal event based on the information entered by the user, and extracts the log data generated by the relevant service components within the corresponding time period from the log storage system.

[0041] It should be noted that the log data obtained can come from multiple service components in the distributed system, including the application service layer, middleware layer, database layer, and infrastructure layer. These log data may adopt different formats and structures, but they all contain basic information such as timestamps, service identifiers, log levels, and message content.

[0042] Optionally, when obtaining log data, the scope of log data acquisition can be determined based on the type or scope of impact of the abnormal event to avoid acquiring irrelevant log data, thereby improving the efficiency of subsequent processing.

[0043] In step S220, a log record refers to a single record item in the log data, which can be understood as a specific event or status information recorded by the system at a certain moment.

[0044] For example, log recording may include, but is not limited to: records of service receiving requests, records of calling external interfaces, records of database query execution, records of errors and exceptions occurring, and records of resource allocation and release.

[0045] The call chain identifier is used to associate log records generated when the same business request flows between different service groups. Specifically, when a user initiates a business request, the request may pass through multiple service components such as the gateway service, application service, and database service in sequence. Each service component generates corresponding log records when processing the request. By embedding the same call chain identifier in these log records, the log records scattered across different service components can be linked together, thereby reconstructing the complete processing path of the business request.

[0046] In one feasible implementation, the call chain identifier field of each log record can be parsed from the log data. Then, the target call chain identifier can be determined based on the business request associated with the abnormal event. Finally, all log records whose call chain identifier field value matches the target call chain identifier can be filtered out as multiple log records with the same call chain identifier.

[0047] In another feasible implementation, the call chain tracing system can be queried based on the request identification information contained in the abnormal event, the call chain identifier corresponding to the request identifier can be obtained from the call chain tracing system, and then the call chain identifier can be used to retrieve multiple log records with the same call chain identifier from the log data.

[0048] Optionally, when there are multiple different call chain identifiers in the log data, the log record corresponding to the call chain identifier directly associated with the abnormal event can be selected first, or the log record corresponding to the call chain identifier that is closest in time to the time when the abnormal event occurred can be selected.

[0049] In step S230, the target model refers to a large language model with natural language understanding and reasoning capabilities. It can be understood as a model that can analyze text semantics and make logical inferences, used to extract information from log records and deduce the root cause of abnormal events.

[0050] Root cause hypothesis refers to inferential conclusions about the cause of an abnormal event. It can be understood as a judgment about the source and triggering conditions of the fault derived through analysis and reasoning, which is used to guide subsequent problem verification and repair operations.

[0051] For example, the root cause hypothesis could be "Service A caused a request processing timeout due to the database connection pool being exhausted", where the problem location is the database connection pool of Service A, and the cause is the exhaustion of connection pool resources.

[0052] In one feasible implementation, multiple log records can be arranged in chronological order according to timestamps and then input into the target model. The target model identifies error messages, warning messages, and status change information by analyzing the text content of each log record. Based on the temporal order and semantic correlation of these messages, it infers the propagation path and triggering conditions of the abnormal event, thereby generating a root cause hypothesis that includes the location and cause of the problem.

[0053] In another feasible implementation, a structured representation of multiple log records can be constructed first. This structured representation includes the timestamp, service identifier, log level, and key information of each log record. Then, this structured representation is input into the target model. The target model analyzes the event evolution process based on the time sequence between log records and identifies causal relationships based on the semantic relationships between log records, thereby generating root cause hypotheses.

[0054] It should be noted that the chronological order reflects the order in which the events recorded in the logs occur, while the semantic relationship reflects the logical connection between the contents of the log records. By combining these two aspects of information, the target model can more accurately locate the root cause of abnormal events.

[0055] Optionally, when generating root cause hypotheses, the target model can incorporate a pre-built knowledge base that stores historical failure cases, common error modes, and system architecture information to improve the accuracy and interpretability of root cause hypotheses.

[0056] By adopting the technical solution of this application, log data is acquired in response to the detection of abnormal events, and multiple log records with the same call chain identifier are obtained from them. This enables the association of log records that were originally scattered across different service nodes, realizing a unified log view across service components. On this basis, the target model generates root cause hypotheses based on the temporal order and semantic relationship between the multiple log records, enabling the system to understand the sequential relationship and inherent meaning of log records in the business request flow process. This breaks through the limitations of traditional keyword matching or predefined rules, which can only perform surface text retrieval, and can deeply analyze the propagation chain of abnormal events among multiple service components. The root cause hypothesis represents the problem location and cause of the abnormal event, directly outputting the root source of the problem rather than just staying at the level of abnormal phenomena. Compared with related technologies, this application uses call chain identifiers to achieve cross-service log association, combined with the comprehensive analysis capabilities of the target model for temporal order and semantic relationship, to accurately locate the root cause of the problem in complex call scenarios, significantly improving the accuracy of locating abnormal problems and the efficiency of recovery in banking business systems.

[0057] Based on the above embodiments, as an optional embodiment, in order to more accurately locate log records related to abnormal events and reduce interference from irrelevant data.

[0058] Figure 3 A second flowchart illustrating a problem localization method based on log analysis according to an embodiment of this application is shown schematically. Figure 3 As shown, step S220 may further include the following steps.

[0059] Step S310: Extract the target call chain identifier associated with the abnormal event from the log data.

[0060] Step S320: Retrieve all log records in the log data whose call chain identifier field value matches the target call chain identifier based on the target call chain identifier.

[0061] Step S330: Sort all log records in chronological order according to their respective timestamps, and select a subset of log records whose time range covers the time when the abnormal event occurred, as multiple log records.

[0062] Optionally, the log record subset includes log records within a preset time window before the occurrence of the abnormal event and log records within a preset time window after the occurrence of the abnormal event; the timestamp represents the time point when the log record was generated.

[0063] In step S310, the target call chain identifier refers to the call chain identifier corresponding to the business request directly related to the abnormal event, which is used to uniquely identify the complete processing flow of the business request in the system.

[0064] In one feasible implementation, the target call chain identifier can be directly obtained from the alarm information or monitoring data of the abnormal event. When the monitoring system triggers an abnormal alarm, the alarm information usually contains the request identifier or call chain identifier that caused the abnormality. The system can directly parse the identifier from the structured fields of the alarm information as the target call chain identifier.

[0065] In another feasible implementation, the target call chain identifier can be inferred from the log data based on the time of occurrence of the abnormal event and the service components affected. Specifically, the time range is first determined based on the time of occurrence of the abnormal event, and log records containing errors or abnormal states within that time range are filtered out from the log data. Then, the call chain identifier field value is extracted from these log records, and the log records are sorted according to their error level and correlation with the abnormal event. The call chain identifier with the highest correlation is selected as the target call chain identifier.

[0066] Optionally, when an abnormal event involves multiple concurrent business requests, multiple target call chain identifiers can be extracted, each corresponding to a different business request process, thereby supporting parallel analysis of multiple call chains.

[0067] In step S320, the call chain identifier field refers to a data field in the log record specifically used to store the call chain identifier. This field maintains a consistent naming convention and data format in the log records of different service components, thereby ensuring the consistency and traceability of the call chain identifier.

[0068] In one feasible implementation, an index can be created on the call chain identifier field in the log storage system. The index can be used to perform an exact match query on the target call chain identifier. The log storage system can quickly locate all log records containing the target call chain identifier based on the index and return these log records as search results.

[0069] In step S330, the timestamp represents the time point when the log record was generated. It usually uses a standard time format to record the precise occurrence time of the log event, which is used to reconstruct the time sequence of events and analyze the temporal relationship between events.

[0070] In one feasible implementation, the timestamp field values ​​of all log records can be extracted and sorted in ascending order according to the size of the timestamp values ​​to obtain a log record sequence arranged in chronological order. Then, the timestamp value corresponding to the time when the abnormal event occurs can be determined, and a time range can be set with this timestamp value as the center. Log records whose timestamp values ​​fall within this time range can be filtered out as a subset of log records.

[0071] Optionally, the log record subset includes log records within a preset time window before the occurrence of the abnormal event and log records within a preset time window after the occurrence of the abnormal event. The log records before the occurrence of the abnormal event are used to analyze the pre-triggered conditions and evolution process of the abnormal event, while the log records after the occurrence of the abnormal event are used to analyze the subsequent impact and propagation path of the abnormal event.

[0072] It should be noted that the size of the preset time window can be configured according to the system's response characteristics and the average processing time of business requests. For systems with fast response speeds, the preset time window can be set to a smaller value, while for business requests with more complex processing flows, the preset time window can be set to a larger value, thereby ensuring that the subset of log records contains sufficient contextual information while avoiding the introduction of too much irrelevant data.

[0073] Optionally, when filtering a subset of log records, priority can be given to sorting log records according to their level, with priority given to retaining error-level and warning-level log records. For information-level and debug-level log records, they can be selectively retained based on their correlation with abnormal events, thereby further reducing the amount of data while ensuring the integrity of the analysis.

[0074] By adopting the embodiments of this application, a subset of log records highly related to abnormal events can be accurately extracted from massive log data. Compared with all log records with the same call chain identifier obtained directly, this subset of log records is more focused on the key time period when the abnormal event occurred in the time dimension, effectively filtering out log records that are far away from the abnormal event in time, thereby providing more refined and targeted data input for subsequent root cause analysis, and improving the efficiency and accuracy of problem localization.

[0075] Based on the above embodiments, as an optional embodiment, in order to achieve deep semantic understanding of multiple log records and cross-service causal relationship mining, thereby accurately generating hypotheses pointing to the root cause of the failure.

[0076] Figure 4 A flowchart illustrating the problem localization method based on log analysis according to an embodiment of this application is shown in Figure 3. Figure 4 As shown, step S230 may further include the following steps.

[0077] Step S410: Semantic parsing is performed on each log record in the multiple log records using the target model to identify at least one semantic element among error type, error location, affected object, and severity.

[0078] Step S420: Construct an event time sequence chain based on the time order of multiple log records. The event time sequence chain represents the evolution sequence formed by the events corresponding to multiple log records in chronological order.

[0079] Step S430: The target model performs causal reasoning based on the correlation between semantic elements corresponding to each event in the event timeline to determine the problem location of the abnormal event and the cause of the abnormal event; the correlation includes at least one of the following: the triggering relationship between the error type in the preceding event and the error type in the subsequent event, and the propagation relationship between the affected object in the preceding event and the error location in the subsequent event.

[0080] In step S410, semantic elements refer to key information items in the log record that characterize a fault or abnormal state, used to describe the specific characteristics and scope of impact of the abnormal event.

[0081] For example, error types may include, but are not limited to: connection timeout, resource exhaustion, null pointer exception, permission denied, data format error, and service unreachable; error locations may include, but are not limited to: specific service names, function names, line numbers, configuration item names, and database table names; affected objects may include, but are not limited to: user session identifiers, order numbers, data record primary keys, network connection identifiers, and thread identifiers; severity may include, but is not limited to: fatal errors, critical warnings, general errors, and error messages.

[0082] In one feasible implementation, the text content of each log record can be provided as input to the target model. The target model performs semantic analysis on the log text to identify keywords and phrases that indicate error status, extract the error name or error code corresponding to the error type, locate the name of the service component, function or resource that caused the error in the log text as the error location, identify the business object identifier mentioned in the log text as the affected object, determine the severity based on the level field of the log record or the tone of the error description, and output the above identification results as the semantic elements of the log record.

[0083] In another feasible implementation, the log records can be preprocessed first to extract structured fields such as timestamps, service identifiers, and log levels from the log text. Then, the remaining message text is input into the target model for semantic parsing. The target model combines the extracted structured field information and the semantic content of the message text to comprehensively determine the error type, error location, affected objects, and severity, thereby improving the accuracy of semantic element recognition.

[0084] Optionally, during the semantic parsing process, the target model can utilize a pre-built domain knowledge base, which contains definitions of common error types, descriptions of typical error patterns, dependencies between system components, and other information. By retrieving relevant entries from the knowledge base, the target model can more accurately understand the meaning of technical terms and specific error patterns in the log text, thereby improving the accuracy of semantic element recognition.

[0085] In step S420, the event sequence chain refers to a sequence structure formed by connecting multiple events in chronological order of their occurrence, which is used to show the occurrence and evolution of abnormal events.

[0086] In one feasible implementation, multiple log records can be arranged in order of their timestamps, each log record can be regarded as an event node, and these event nodes can be connected in chronological order to form an event sequence on a timeline. This sequence is the event timeline chain. Each node in the event timeline chain contains the timestamp of the corresponding log record, the service identifier, and the semantic elements identified in step S410.

[0087] In another feasible implementation, multiple log records can be divided into different service processing stages based on the call chain identifier and service identifier. Within each service processing stage, the events corresponding to the log records are arranged in chronological order. Then, the different service processing stages are connected according to the flow order of business requests to form a cross-service event sequence chain. This event sequence chain not only reflects the time order of events within a single service, but also reflects the time order relationship of business requests flowing between multiple services.

[0088] In step S430, the association relationship refers to the logical connection between the semantic elements of different events in the event time sequence chain, which is used to characterize the possible influence and dependence between events.

[0089] Causal reasoning refers to the logical analysis process that infers the cause of an abnormal event and the path of its propagation based on the correlation and time sequence between events, in order to locate the root cause of a fault.

[0090] Optionally, the association relationship includes at least one of the following: the triggering relationship between the error type in the preceding event and the error type in the subsequent event, and the propagation relationship between the affected object in the preceding event and the error location in the subsequent event.

[0091] In this context, the triggering relationship refers to the causal link between an error type occurring in a preceding event and a specific error type occurring in a subsequent event. For example, a database connection timeout error in a preceding event may trigger a transaction rollback error in a subsequent event.

[0092] Similarly, propagation relationship refers to the fact that an object affected in a preceding event becomes the location or object involved in an error in a subsequent event, indicating that the impact of an anomaly is passed along the business process or data flow. For example, if a cached key-value object becomes invalid in a preceding event, an error will occur when accessing that cached key in a subsequent event.

[0093] In one feasible implementation, an event sequence chain can be input into the target model. The target model traverses adjacent or close event pairs in the event sequence chain, analyzes the semantic elements of the preceding and subsequent events in each event pair, determines whether there is a known triggering relationship between the error type of the preceding event and the error type of the subsequent event, and determines whether the affected object of the preceding event matches the error location or affected object of the subsequent event. When a triggering relationship or propagation relationship exists, the target model establishes a causal relationship between the two events and records the type and strength of the causal relationship. Through the analysis of all event pairs in the event sequence chain, the target model constructs a complete causal relationship network, identifies the earliest event that occurs and triggers a series of subsequent errors from this network, determines the error location corresponding to the event as the problem location of the abnormal event, and determines the error type corresponding to the event as the cause of the abnormal event.

[0094] In another feasible implementation, the reasoning ability of the target model can be utilized, combined with domain knowledge and historical failure cases, to comprehensively analyze the semantic elements in the event sequence chain. The target model not only considers the direct correlation between adjacent events, but also the indirect correlation across multiple events. Through logical reasoning, it identifies hidden causal chains. For example, if the preceding event A leads to the intermediate event B, and the intermediate event B in turn leads to the subsequent event C, the target model can infer that event A is the root cause of event C, thereby accurately locating the problem location and cause.

[0095] Optionally, during the causal reasoning process, the target model can assign a confidence score to each identified causal association. This score reflects the credibility of the causal association. When there are multiple possible causal links, the target model can select the link with the highest confidence score as the final root cause path based on the confidence scores of each causal link, thereby determining the location and cause of the problem.

[0096] By employing the embodiments of this application, the target model performs semantic parsing on each log record and identifies semantic elements such as error type, error location, affected objects, and severity. This overcomes the limitations of traditional methods that can only perform text matching and achieves a deep semantic understanding of log content. Based on this, an event timeline chain is constructed based on the chronological order of multiple log records, transforming scattered log records into a structured event evolution sequence, providing a clear timeline for subsequent causal analysis. Furthermore, the target model performs causal reasoning based on the triggering and propagation relationships between the semantic elements corresponding to each event in the event timeline chain. This enables the identification of the propagation path and causal chain of abnormal events across multiple service components, accurately determining the problem location and cause of the abnormal event.

[0097] Based on the above embodiments, as an optional embodiment, in order to identify underlying resource anomalies that have not yet been captured by log records, thereby discovering system-level fault precursors that have occurred before the application layer logs are generated, the above log analysis-based problem localization method may further include the following steps.

[0098] Obtain system monitoring metrics data associated with the abnormal event. The system monitoring metrics data includes the time-series values ​​of at least one of the following performance indicators: processor utilization, memory utilization, network latency, and request throughput, within a preset time window before and after the occurrence of the abnormal event.

[0099] By using a target model to identify abnormal patterns in system monitoring data, the target performance indicators that experience abnormal fluctuations and the corresponding times of these abnormal fluctuations can be determined.

[0100] Determine whether the time of the abnormal fluctuation is earlier than the timestamp of the first log record in the event timeline.

[0101] If the time of the abnormal fluctuation is earlier than the timestamp of the first log record, the system resource corresponding to the target performance indicator is determined to be the problem location of the abnormal event, and the abnormal state of the target performance indicator is used as the pre-trigger factor in causal reasoning.

[0102] Among them, system monitoring index data refers to the numerical records of quantitative indicators that reflect the system's operating status collected by the monitoring system, which are used to characterize the system's resource usage and performance.

[0103] For example, processor utilization represents the proportion of processor resources used, memory utilization represents the proportion of memory resources used, network latency represents the time overhead of network communication, and request throughput represents the number of requests processed by the system per unit time.

[0104] Similarly, time-series values ​​refer to a series of numerical points recorded in chronological order for a performance indicator. Each numerical point corresponds to the measured value of the performance indicator at a specific moment, and is used to reflect the trend of the performance indicator over time.

[0105] In one feasible implementation, the time range for data acquisition can be determined based on the time of occurrence of the abnormal event. This time range covers a preset time window before and after the time of occurrence of the abnormal event. The sampling data of each performance indicator within this time range is queried from the monitoring system, and the queried time series values ​​are used as system monitoring indicator data. The size of the preset time window can be set according to the system's monitoring sampling frequency and the type of abnormal event, thereby ensuring that the acquired time series values ​​can fully reflect the changes in system state before and after the abnormal event.

[0106] Among them, abnormal pattern recognition refers to the process of detecting abnormal change patterns that deviate from the normal fluctuation range from time series values, which is used to discover abnormal states of performance indicators.

[0107] Similarly, target performance metrics refer to performance metrics that have been identified as exhibiting abnormal fluctuations, used to identify the types of system resources that have experienced anomalies.

[0108] Similarly, the abnormal fluctuation moment refers to the point in time when the time series value of the target performance indicator begins to change abnormally, and is used to determine the time location of the anomaly.

[0109] In one feasible implementation, the time-series values ​​of each performance indicator in the system monitoring index data can be input into the target model. The target model identifies time periods that deviate significantly from the normal pattern by analyzing the changing trend, fluctuation amplitude and statistical characteristics of the time-series values. For the identified abnormal time periods, the target model determines the performance indicator corresponding to the time period as the target performance indicator and takes the starting time of the abnormal time period as the abnormal fluctuation time.

[0110] Among them, the preceding triggering factor refers to the abnormal state that exists before the first log record in the event sequence chain is generated. This abnormal state may trigger or cause subsequent application layer errors and abnormal events.

[0111] In one feasible implementation, the corresponding system resources can be determined according to the type of the target performance indicator, the system resource can be identified as the problem location of the abnormal event, and the abnormal state of the target performance indicator at the time of abnormal fluctuation, such as "processor utilization remains high" or "memory utilization rises sharply to the critical value", can be added as a pre-trigger factor to the input of causal inference. When the target model performs causal inference, it takes the pre-trigger factor as the starting point or important precondition of the event sequence chain and analyzes how it leads to the occurrence of subsequent events in the event sequence chain, thereby forming a complete causal link that includes the underlying resource abnormality and the upper-layer application error.

[0112] Optionally, when multiple target performance indicators are identified to have abnormal fluctuations and all of them are earlier than the timestamp of the first log record, the system resource corresponding to the earliest and most affected performance indicator can be identified as the primary problem location based on the chronological order and severity of the abnormal fluctuations of each target performance indicator. The abnormal states of other performance indicators can be identified as secondary pre-triggering factors, thereby establishing a multi-level causal analysis view.

[0113] By adopting the embodiments of this application, system monitoring indicator data associated with abnormal events is obtained, and the status of underlying system resources is included in the analysis scope, thus overcoming the limitations of relying solely on log data. By identifying abnormal patterns in the system monitoring indicator data through the target model, abnormal fluctuation patterns can be automatically discovered from the time-series values. By determining whether the time of abnormal fluctuation is earlier than the timestamp of the first log record in the event time-series chain, the causal relationship between system resource abnormalities and application layer errors can be distinguished. When the time of abnormal fluctuation is earlier than the timestamp of the first log record, the system resource corresponding to the target performance indicator is identified as the problem location and used as a pre-triggering factor, thus realizing the construction of a complete causal link from underlying resource abnormalities to upper-layer application errors.

[0114] Based on the above embodiments, as an optional embodiment, in order to avoid misjudgment caused by directly outputting low-confidence root cause hypotheses, supplementary verification data is introduced for iterative analysis to improve the reliability of root cause hypotheses.

[0115] Figure 5 A flowchart illustrating the problem localization method based on log analysis according to an embodiment of this application is shown in Figure 4. Figure 5 As shown, the above-mentioned log analysis-based problem localization method may also include the following steps.

[0116] Step S510: Obtain the confidence level of the root cause hypothesis generated by the target model.

[0117] Step S520: If the confidence level does not meet the preset conditions, then obtain the verification data.

[0118] Step S530: Update the root cause hypothesis and / or confidence level of the target model based on the validation data so that the confidence level meets the preset conditions.

[0119] In step S510, confidence level refers to the quantitative assessment of the credibility of the root cause hypothesis generated by the target model, which is used to characterize the probability or certainty level that the root cause hypothesis accurately points to the actual cause of failure.

[0120] In one feasible implementation, the target model can calculate a numerical confidence score based on factors such as the strength of causal relationships identified during causal reasoning, the accuracy of semantic element matching, and the completeness of the event timeline during the generation of root cause hypotheses.

[0121] In step S520, the preset condition refers to the standard used to determine whether the root cause hypothesis is reliable enough, and to decide whether supplementary data needs to be obtained for further verification.

[0122] Validation data refers to additional data used to validate or supplement root cause hypotheses, providing more evidence to support or revise root cause hypotheses.

[0123] The type and source of validation data can be determined based on the content of the root cause hypothesis and the inadequacy of currently available data. For example, validation data may include, but is not limited to: log data with an extended time range, log data from other relevant service components, more granular system monitoring metrics data, application configuration file content, database query execution plans, and detailed network traffic records.

[0124] In one feasible implementation, the target model can analyze the weaknesses of the current root cause hypothesis and determine what types of verification data need to be supplemented. For example, when the root cause hypothesis points to a certain service component but the component has few log records, the log collection scope can be expanded to obtain more log records of the component as verification data. When the root cause hypothesis involves network communication problems but lacks network-level evidence, network traffic data can be obtained as verification data.

[0125] In step S530, the verification data, along with the log data and system monitoring index data obtained in the preceding steps, can be input into the target model. The target model performs a comprehensive analysis of all the data, re-executes steps such as semantic parsing, event time-series chain construction, and causal reasoning, generates an updated root cause hypothesis based on the analysis results, and calculates the updated confidence level.

[0126] In another feasible implementation, validation data can be input into the target model as supplementary evidence. The target model determines whether the validation data supports the current root cause hypothesis. If the validation data is consistent with the current root cause hypothesis, the content of the root cause hypothesis remains unchanged but the confidence level is increased. If the validation data contradicts the current root cause hypothesis or reveals a new causal relationship, the content of the root cause hypothesis is modified and the confidence level is recalculated.

[0127] Optionally, steps S520 and S530 can be iterated multiple times. That is, if the confidence level still does not meet the preset conditions after the first update, new verification data can be obtained again and updated again until the confidence level meets the preset conditions or the preset number of iterations is reached.

[0128] By employing the embodiments of this application, the reliability of the root cause hypothesis can be quantitatively assessed by obtaining the confidence level corresponding to the root cause hypothesis generated by the target model. When the confidence level does not meet the preset conditions, verification data is obtained, avoiding the risk of misjudgment caused by directly outputting low-confidence hypotheses. By updating the root cause hypothesis and confidence level based on the verification data, the target model achieves iterative optimization of root cause analysis. Compared with the method of outputting analysis results all at once in related technologies, this application, through the closed-loop processing of confidence level assessment and verification data supplementation, can proactively obtain supplementary evidence for correction when the initial analysis results are uncertain, thereby improving the accuracy and reliability of the root cause hypothesis.

[0129] Based on the above embodiments, as an optional embodiment, the preset conditions may include at least one of the following: the confidence level reaches a preset threshold; the verification data contains confirmatory evidence that matches the problem location and cause in the root cause hypothesis, and the confirmatory evidence includes error log records consistent with the error type pointed to by the root cause hypothesis, monitoring indicator data consistent with the resource exhaustion state pointed to by the root cause hypothesis, or configuration item status information consistent with the configuration error pointed to by the root cause hypothesis; the number of iterative verifications for the root cause hypothesis reaches a preset upper limit.

[0130] Confirmatory evidence refers to data records in the verification data that can directly support the root cause hypothesis, and is used to verify the correctness of the root cause hypothesis from actual data.

[0131] In one feasible implementation, when the confidence level reaches a preset threshold, it can be determined that the preset condition has been met. At this point, there is no need to continue acquiring verification data, and the current root cause hypothesis can be directly output as the final result. This method is based on the quantitative evaluation of confidence level; when the confidence level is high enough, it indicates that the root cause hypothesis has sufficient credibility.

[0132] In another feasible implementation, when confirmatory evidence exists in the verification data, the preset conditions can be determined to be met. Specifically, the target model can analyze the content of the verification data to determine whether the verification data contains records that match the problem location and cause in the root cause hypothesis. For example, when the root cause hypothesis points to a database connection timeout error for a certain service, if the verification data contains an error log record of "database connection timeout" recorded by that service at the corresponding time point, then the error log record constitutes confirmatory evidence; when the root cause hypothesis points to a memory exhaustion problem for a certain server, if the verification data shows that the server's memory usage rate reaches or approaches 100% at the corresponding time point, then the monitoring indicator data constitutes confirmatory evidence; when the root cause hypothesis points to a configuration item setting error, if the status information of the configuration item in the verification data shows that its value is inconsistent with expectations or was incorrectly modified before the failure occurred, then the status information of the configuration item constitutes confirmatory evidence. When confirmatory evidence exists, it indicates that the root cause hypothesis is supported by actual data, and even if the confidence value does not reach the preset threshold, the root cause hypothesis can be considered to have been sufficiently verified.

[0133] In another feasible implementation, when the number of iterative verifications of the root cause hypothesis reaches a preset upper limit, the preset condition can be determined to be met. This method is used to avoid infinite iterative verification. When multiple verifications and updates have been performed but the confidence level has not yet reached the preset threshold and no confirmatory evidence has been obtained, to prevent resource consumption and time delays caused by over-verification, the iteration process can be terminated when the preset upper limit is reached. The current root cause hypothesis is then output as the final result, and the confidence level of the hypothesis can be explained to the user, allowing the user to decide whether to adopt it based on the actual situation.

[0134] It should be noted that the above three criteria can be used individually or in combination. When using a combination, as long as any one of the criteria is met, the preset conditions can be determined to be met.

[0135] By adopting the embodiments of this application, a judgment standard based on quantitative evaluation is provided by setting a confidence level reaching a preset threshold as a preset condition; a judgment standard based on actual evidence is provided by setting the existence of confirmatory evidence in the verification data as a preset condition, so that the root cause hypothesis can be confirmed based on conclusive evidence even if the confidence level value does not reach the threshold; and by setting the number of iterations to reach a preset upper limit as a preset condition, the waste of resources caused by infinite iteration is avoided, and the problem localization process is ensured to be completed within a reasonable time.

[0136] Based on the above embodiments, as an optional embodiment, in order to automate and target the verification data acquisition process and avoid blindly collecting a large amount of irrelevant data, the above step S520 may further include the following steps.

[0137] A verification plan is generated based on the root cause hypothesis using the target model. The verification plan includes one or more verification instructions for confirming or disproving the root cause hypothesis. Each verification instruction is associated with the type of target information to be acquired and the corresponding information source location.

[0138] Based on the information source location corresponding to each verification instruction in the verification plan, data corresponding to the target information type is obtained from at least one of the monitoring indicator database, log storage system, and configuration management database as verification data; the monitoring indicator database stores system performance indicators and service health status data, and the configuration management database stores the status information of system configuration items.

[0139] Among them, the verification plan refers to the data acquisition scheme formulated for the root cause hypothesis; the verification instruction refers to the specific operation item in the verification plan; the target information type refers to the category or content characteristics of the data required by the verification instruction; and the information source location refers to the storage location or acquisition path of the data corresponding to the target information type, which is used to locate the system or database where the data is located.

[0140] In one feasible implementation, the target model can analyze the problem location and cause involved in the root cause hypothesis, and determine what types of supplementary evidence are needed to verify the hypothesis. For example, when the root cause hypothesis points to a memory leak problem in a certain service, the target model can generate a verification plan containing the following verification instructions: The first verification instruction requests the acquisition of the service's historical memory usage curve data, with the target information type being "time-series data of memory usage for a specific service" and the information source location being "the memory metric table of service A in the monitoring metric database"; the second verification instruction requests the acquisition of the service's garbage collection logs, with the target information type being "garbage collection event records" and the information source location being "the runtime log directory of service A in the log storage system".

[0141] Optionally, multiple verification instructions in the verification plan can be sorted by priority. The data corresponding to the verification instructions with higher priority are more critical to verifying the root cause hypothesis. The system can execute the high-priority verification instructions first, thereby improving verification efficiency.

[0142] Among them, the monitoring index database refers to the database system that stores system performance indicators and service health status data, and is used to provide quantitative data on the system's operating status; the log storage system refers to the system that stores various log records, and is used to provide event records during the system's operation; the configuration management database refers to the database that stores the status information of system configuration items, and is used to provide the current value and change history of the system configuration.

[0143] In one feasible implementation, each verification instruction in the verification plan can be traversed, the information source location in each verification instruction can be parsed, and a corresponding data query request can be constructed based on the database type and specific location information of the information source location identifier. The query request can be sent to the monitoring indicator database, log storage system or configuration management database to obtain the data corresponding to the target information type, and the obtained data can be summarized as verification data.

[0144] By adopting the embodiments of this application, a verification plan is generated based on the root cause hypothesis through the target model, which can specifically determine the data types and sources required for verification, avoiding the inefficiency of blindly collecting data. The verification plan clearly associates the target information type and information source location through verification instructions, transforming the abstract verification requirements into executable data acquisition operations. Verification data is obtained from the monitoring indicator database, log storage system and configuration management database according to the verification instructions, realizing automated data collection across multiple data sources.

[0145] Based on the above-described log analysis-based problem localization method, embodiments of this application also provide a log analysis-based problem localization device. The following will combine... Figure 6 The device is described in detail.

[0146] Figure 6 A schematic block diagram of a log analysis-based problem location device according to an embodiment of this application is shown.

[0147] like Figure 6 As shown, the log analysis-based problem location device 600 of this embodiment includes a log acquisition module 610, a log association module 620, and a root cause reasoning module 630.

[0148] The log acquisition module 610 is used to acquire log data in response to the detection of an abnormal event; the log association module 620 is used to obtain multiple log records with the same call chain identifier in the log data; the call chain identifier is used to associate log records generated when the same business request flows between different service groups; the root cause reasoning module 630 is used to generate root cause hypotheses based on the time sequence and semantic relationship between multiple log records through the target model; the root cause hypothesis represents the problem location and cause of the abnormal event.

[0149] According to an embodiment of this application, the log association module 620 is further configured to extract the target call chain identifier associated with the abnormal event from the log data; retrieve all log records in the log data whose call chain identifier field value matches the target call chain identifier based on the target call chain identifier; sort all log records in chronological order according to their respective timestamps, and filter out a subset of log records whose time range covers the time of the abnormal event occurrence, as multiple log records; wherein, the subset of log records includes log records within a preset time window before the time of the abnormal event occurrence and log records within a preset time window after the time of the abnormal event occurrence; the timestamp represents the time point when the log record was generated.

[0150] According to an embodiment of this application, the root cause reasoning module 630 is further configured to perform semantic parsing on each log record among multiple log records using a target model, identifying at least one semantic element among error type, error location, affected object, and severity; construct an event timeline chain based on the chronological order of multiple log records, the event timeline chain representing the evolutionary sequence formed by the events corresponding to multiple log records in chronological order; perform causal reasoning based on the correlation between the semantic elements corresponding to each event in the event timeline chain using the target model, to determine the problem location of the abnormal event and the cause of the abnormal event; the correlation includes at least one of the triggering relationship between the error type in the preceding event and the error type in the subsequent event, and the propagation relationship between the affected object in the preceding event and the error location in the subsequent event; and generate a root cause hypothesis based on the problem location and the cause.

[0151] According to an embodiment of this application, the root cause reasoning module 630 is further configured to acquire system monitoring indicator data associated with the abnormal event. The system monitoring indicator data includes the time-series values ​​of at least one performance indicator among processor utilization, memory utilization, network latency, and request throughput within a preset time window before and after the occurrence of the abnormal event. The module performs abnormal pattern recognition on the system monitoring indicator data through a target model to determine the target performance indicator that experienced the abnormal fluctuation and the corresponding time of the abnormal fluctuation. The module determines whether the time of the abnormal fluctuation is earlier than the timestamp of the first log record in the event time-series chain. If the time of the abnormal fluctuation is earlier than the timestamp of the first log record, the module determines that the system resource corresponding to the target performance indicator is the problem location of the abnormal event and uses the abnormal state of the target performance indicator as a pre-triggering factor in causal reasoning.

[0152] According to an embodiment of this application, the root cause reasoning module 630 is further configured to obtain the confidence level corresponding to the root cause hypothesis generated by the target model; if the confidence level does not meet the preset conditions, then obtain the verification data; and update the root cause hypothesis and / or confidence level by the target model based on the verification data so that the confidence level meets the preset conditions.

[0153] According to an embodiment of this application, the root cause reasoning module 630 is further configured to generate a verification plan based on the root cause hypothesis through the target model. The verification plan includes one or more verification instructions for verifying or disproving the root cause hypothesis. Each verification instruction is associated with the type of target information to be acquired and the corresponding information source location. Based on the information source location corresponding to each verification instruction in the verification plan, data corresponding to the target information type is acquired from at least one of the monitoring indicator database, the log storage system, and the configuration management database as verification data. The monitoring indicator database stores system performance indicators and service health status data, and the configuration management database stores the status information of system configuration items.

[0154] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a log analysis-based problem localization method according to an embodiment of this application.

[0155] like Figure 7 As shown, an electronic device 700 according to an embodiment of this application includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory 702 or a program loaded from a storage portion 708 into a random access memory 703. The processor 701 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0156] Random access memory 703 stores various programs and data required for the operation of electronic device 700. Processor 701, read-only memory 702, and random access memory 703 are interconnected via bus 704. Processor 701 executes various steps of the method flow according to embodiments of this application by executing programs in read-only memory 702 and / or random access memory 703. It should be noted that the programs may also be stored in one or more memories other than read-only memory 702 and random access memory 703. Processor 701 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0157] According to embodiments of this application, the electronic device 700 may further include an input / output interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the input / output interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card, such as a local area network card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0158] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0159] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 702, and / or random access memory 703, and / or one or more memories other than read-only memory 702 and random access memory 703 described above.

[0160] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0161] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0162] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by processor 701, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0163] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0164] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0165] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A problem localization method based on log analysis, characterized in that, include: In response to the detection of an abnormal event, log data is retrieved; Obtain multiple log records with the same call chain identifier from the log data; The call chain identifier is used to associate log records generated when the same business request flows between different service groups; The target model generates root cause hypotheses based on the temporal order and semantic relationships between the multiple log records; the root cause hypotheses characterize the problem location and cause of the abnormal event.

2. The method according to claim 1, characterized in that, The process of obtaining multiple log records with the same call chain identifier from the log data includes: Extract the target call chain identifier associated with the abnormal event from the log data; Based on the target call chain identifier, retrieve all log records in the log data whose call chain identifier field value matches the target call chain identifier; Sort all log records in chronological order according to their respective timestamps, and select a subset of log records whose time range covers the time when the abnormal event occurred, as the multiple log records; The log record subset includes log records within a preset time window before the occurrence of the abnormal event and log records within a preset time window after the occurrence of the abnormal event; the timestamp represents the time point when the log record was generated.

3. The method according to claim 1, characterized in that, The process of generating root cause hypotheses based on the temporal order and semantic relationships between the multiple log records using the target model includes: The target model is used to perform semantic parsing on each of the multiple log records to identify at least one semantic element among error type, error location, affected object, and severity. An event time sequence chain is constructed based on the time order of the multiple log records. The event time sequence chain represents the evolution sequence formed by the events corresponding to the multiple log records in chronological order. The target model performs causal reasoning based on the correlation between semantic elements corresponding to each event in the event timeline to determine the problem location and cause of the abnormal event; the correlation includes at least one of the following: the triggering relationship between error types in preceding events and error types in subsequent events, and the propagation relationship between affected objects in preceding events and error locations in subsequent events. The root cause hypothesis is generated based on the location of the problem and the cause.

4. The method according to claim 3, characterized in that, The method further includes: Acquire system monitoring metrics data associated with the abnormal event. The system monitoring metrics data includes time-series values ​​of at least one performance indicator among processor utilization, memory utilization, network latency, and request throughput within a preset time window before and after the occurrence of the abnormal event. The target model is used to identify abnormal patterns in the system monitoring index data to determine the target performance index that experiences abnormal fluctuations and the corresponding time of abnormal fluctuations. Determine whether the time of the abnormal fluctuation is earlier than the timestamp of the first log record in the event timeline; If the time of the abnormal fluctuation is earlier than the timestamp of the first log record, then the system resource corresponding to the target performance indicator is determined to be the problem location of the abnormal event, and the abnormal state of the target performance indicator is used as the pre-triggering factor in the causal reasoning.

5. The method according to claim 1, characterized in that, Also includes: Obtain the confidence level of the root cause hypothesis generated by the target model; If the confidence level does not meet the preset conditions, then verification data is obtained; The target model updates the root cause hypothesis and / or the confidence level based on the validation data, so that the confidence level meets the preset condition.

6. The method according to claim 5, characterized in that, The obtained verification data includes: The target model generates a verification plan based on the root cause hypothesis. The verification plan includes one or more verification instructions for confirming or disproving the root cause hypothesis. Each verification instruction is associated with the type of target information to be acquired and the corresponding information source location. Based on the information source location corresponding to each verification instruction in the verification plan, data corresponding to the target information type is obtained from at least one of the monitoring indicator database, log storage system, and configuration management database as the verification data; the monitoring indicator database stores system performance indicators and service health status data, and the configuration management database stores the status information of system configuration items.

7. The method according to claim 5, characterized in that, The preset conditions include at least one of the following: The confidence level reaches a preset threshold; The verification data contains confirmatory evidence that matches the problem location and cause in the root cause hypothesis. The confirmatory evidence includes error log records that are consistent with the error type pointed to by the root cause hypothesis, monitoring indicator data that are consistent with the resource exhaustion state pointed to by the root cause hypothesis, or configuration item status information that are consistent with the configuration error pointed to by the root cause hypothesis. The number of iterative verifications for the root cause hypothesis has reached a preset upper limit.

8. A problem location device based on log analysis, characterized in that, include: The log acquisition module is used to acquire log data in response to the detection of abnormal events; The log association module is used to obtain multiple log records with the same call chain identifier in the log data; the call chain identifier is used to associate log records generated when the same business request flows between different service groups. The root cause reasoning module is used to generate root cause hypotheses based on the temporal order and semantic relationship between the multiple log records through the target model; the root cause hypotheses represent the problem location and cause of the abnormal event.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images