Intelligent diagnosis method, system, electronic device and program product for distributed system

Abstract

The application provides an intelligent diagnosis method for a distributed system, which comprises the following steps: determining a sequence of weighted log template entropy of each service instance in the distributed system based on logs of each service instance in the distributed system, taking a first preset time window as a sliding window; generating an entropy variation abnormal event and publishing when an entropy value of any one of the sequences exceeds a confidence interval of an adaptive dynamic baseline of the sequence; wherein the confidence interval of the adaptive dynamic baseline is determined by a time series prediction model, and the entropy variation abnormal event at least comprises an abnormal service instance identifier and an abnormal timestamp; when a number of the entropy variation abnormal events detected within a second preset time window is greater than an event threshold, performing root cause positioning to form a list of abnormal root source suspects; forming a diagnosis prompt word for each suspect service in the list of abnormal root source suspects, and inputting the formed prompt word into a large language model; and analyzing and forming an intelligent diagnosis result by the large language model. The application also provides a corresponding system, an electronic device and a program product, etc.

Description

Technical Field

[0001] This application relates to distributed systems, and more specifically, to the diagnosis of anomalies in distributed systems. Background Technology

[0002] In the field of log anomaly detection and fault handling, existing technologies are mainly divided into two categories: deep learning-based intelligent anomaly detection and rule-based automated fault response.

[0003] Patent application CN120653776A discloses a deep learning-based log anomaly detection method. This method utilizes a pre-trained SBERT model to vectorize logs, extract log parameters and template features, and performs sequence-level and individual log-level anomaly detection in parallel through a classification sub-model. By fusing detection results at different granularities, this method aims to improve the model's detection accuracy and generalization ability for complex log sequences.

[0004] Patent application CN120508436A discloses a post-fault handling process. The solution obtains and parses explicit "error" logs, automatically locates the target line of code that caused the error, and retrieves the code committer's information. Subsequently, the system automatically adds a comment containing the error information to the code line and notifies the relevant developers.

[0005] However, these solutions still have room for improvement when dealing with the operational challenges of modern complex distributed systems. Summary of the Invention

[0006] According to one aspect of this application, an intelligent diagnostic method for distributed systems is provided to address at least one of the problems mentioned herein.

[0007] An intelligent diagnostic method for a distributed system, according to an embodiment of one aspect of this application, may include determining a sequence of weighted log template entropy for each service instance based on logs of each service instance in the distributed system, using a first preset time window as a sliding window; generating and publishing an entropy change anomaly event when the entropy value of any sequence exceeds the confidence interval of the adaptive dynamic baseline of that sequence; wherein the confidence interval of the adaptive dynamic baseline is determined by a time series prediction model, and the entropy change anomaly event includes at least an abnormal service instance identifier and an abnormal timestamp; when the number of entropy change anomaly events detected within a second preset time window is greater than an event threshold, root cause localization is performed to form a list of suspected root cause services; for each suspected service in the list of suspected root cause services, diagnostic prompt words are generated and input into a large language model; and the large language model is used for analysis to generate an intelligent diagnostic result.

[0008] According to another aspect of this application, an intelligent diagnostic system for a distributed system is provided. The diagnostic system includes an anomaly detection module, a localization module, and a diagnostic module. The anomaly detection module uses logs of each service instance in the distributed system to determine a sequence of weighted log template entropy for each service instance using a first preset time window as a sliding window. When the entropy value in any of the sequences exceeds the confidence interval of the adaptive dynamic baseline for that sequence, an entropy-change anomaly event is generated and published. The confidence interval of the adaptive dynamic baseline is determined by a time series prediction model. The entropy-change anomaly event includes at least an anomaly service instance identifier (ID) and an anomaly timestamp. The localization module can be used to perform root cause localization when the number of entropy-change anomaly events detected within a second preset time window exceeds an event threshold, thereby forming a list of suspected root cause services. The diagnostic module can be used to generate diagnostic prompts for each suspected service in the list of suspected root cause services, and input the generated diagnostic prompts into a large language model for analysis and intelligent diagnostic results.

[0009] This application also provides a program product comprising instructions that, when executed, implement any of the methods described herein. The program product can be downloaded via a network or stored in a storage medium for use.

[0010] This application also provides a non-volatile storage medium on which instructions are stored, which, when executed, can implement any of the method embodiments described herein.

[0011] The distributed system intelligent diagnostic scheme provided by the embodiments of this application can proactively discover unknown silent faults through semantic weighted entropy and adaptive dynamic baseline, effectively filter out interference from normal business fluctuations, and significantly reduce the false alarm rate; it can also quickly and accurately locate the root cause, improve the automation and intelligence level of distributed system operation and maintenance, reduce operation and maintenance costs, and enhance system operation stability. Attached Figure Description

[0012] This application will be more fully understood by referring to the following detailed description of specific embodiments in conjunction with the accompanying drawings, wherein the same reference numerals in the drawings refer to the same elements, wherein: Figure 1 This is a flowchart of an intelligent diagnostic method for a distributed system according to an embodiment of this application; Figure 2 This is a schematic diagram illustrating the semantic weighting process according to some embodiments of this application; Figure 3 This is a flowchart illustrating the online calculation of the entropy of each weighted log template within the first preset time window, based on the log template sequence of each service instance according to some embodiments of this application. Figure 4 This is a flowchart of root cause localization based on some examples from this application; Figure 5 This is a flowchart of an embodiment for generating diagnostic prompt words; Figure 6 This is a schematic diagram of the structure of an intelligent diagnostic system for a distributed system according to an embodiment of this application; Figure 7 This is a structural diagram of an electronic device according to another embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the described implementation methods are only a part of the implementation methods of the technical solutions of this application, and not all of them.

[0014] Figure 1 This is a flowchart of an intelligent diagnostic method for a distributed system according to an embodiment of this application. In the example of this application, the geographically dispersed nodes of the distributed system are connected through a network to collectively complete tasks that a single node cannot efficiently accomplish. Distributed systems include, for example, cloud computing platforms, big data processing systems, distributed databases, blockchain systems, and Internet of Things (IoT) systems.

[0015] In step S10, based on the logs of each service instance in the distributed system, a time series of weighted log template entropy is determined using a first preset time window as a sliding window. For example, if the first preset time window is 1 minute, the weighted log template entropy of each service instance within 1 minute is determined, and then the weighted log template entropy of each service instance within the next 1 minute is determined, thus forming a sequence of weighted log template entropy for each service instance. It should be noted that the time series here is used to indicate that the formed entropy sequence is a time-based sequence, that is, a sequence obtained by sliding the time window. In step S12, when the entropy value of any of the sequences exceeds the confidence interval of the adaptive dynamic baseline of the sequence, an entropy change anomaly event is generated. The entropy change anomaly event includes at least the abnormal service instance identifier (ID) and the abnormal timestamp. In some examples, it may also include the actual entropy value, the baseline prediction value, and the deviation magnitude. In the process of forming each entropy sequence, a confidence interval of the adaptive dynamic baseline can be generated for the entropy sequence through a time series prediction model. As an example, the time series prediction model is, for example, the Prophet model. The deviation magnitude is the degree of deviation between the actual weighted log template entropy value (i.e., the actual entropy value) of the service instance and the corresponding adaptive dynamic baseline prediction value when generating an entropy change anomaly events. It can be used to quantify the severity of the anomaly. In step S14, when the number of entropy change anomaly events detected within the second preset time window exceeds the event threshold, root cause localization is performed to form a list of suspected anomaly root cause services. For example, the second preset time window is 5 minutes. If the number of entropy change anomaly events detected within 5 minutes exceeds the event threshold, it indicates that the anomaly event needs to be located and its cause investigated. The event threshold can be pre-set, and "greater than" in this document can include the case of equal to, which is determined according to the actual application. In step S16, for each suspected service in the list of suspected anomaly root cause services, diagnostic prompt words are generated and input into the large language model. In step S18, the large language model is used for analysis to generate intelligent diagnostic results.

[0016] The weighted log template entropy sequence, exhibiting periodicity and trends, can be used with time series forecasting models such as Prophet to generate confidence intervals for an adaptive dynamic baseline. For example, by inputting the weighted log template entropy time series of a single service instance into the time series forecasting model, the model automatically learns the trend and periodicity of the sequence, outputting a predicted baseline including confidence intervals—the adaptive dynamic baseline. As new entropy data is generated within a sliding time window, the model can incrementally train and dynamically adjust baseline parameters to ensure the baseline always adapts to the latest operational status of the service instance. For instance, the baseline generated by the Prophet model is in the form of confidence intervals, such as a 95% confidence interval. An entropy value exceeding this interval is considered an anomaly, which is more suitable for actual operational scenarios compared to static thresholds. As an example, the time series forecasting model can support configuring special events, such as system upgrades, allowing manual annotation of these time periods and adjustment of the baseline to further reduce the false positive rate.

[0017] Furthermore, the deviation magnitude is calculated based on the weighted log template entropy sequence generated in step S10 and the baseline predicted value output by a time series model such as Prophet in step S12. Specifically, when the actual entropy value exceeds the baseline confidence interval, the deviation magnitude can be obtained by subtracting the baseline predicted value from the actual entropy value to obtain a difference that intuitively reflects the magnitude of the deviation. Alternatively, the deviation magnitude can also be obtained by "(actual entropy value - baseline predicted value) / baseline predicted value × 100%", which is a ratio representing the proportion of quantified deviation. This deviation magnitude can be stored in the event set as an attribute of the entropy change anomaly event, as will be described below. Figure 4 The process shown is employed.

[0018] By generating a weighted log template entropy time series through a sliding time window and matching it with an adaptive dynamic baseline, instead of using a static threshold, it can adapt to the log fluctuation patterns of different runtime stages of the distributed system, significantly reducing the false positive and false negative rates. The warning judgment mechanism of the second preset time window and event threshold used in step S14 avoids isolated warnings of single service anomalies, which helps to accurately identify system-level and cascading failures. The root cause localization process makes the diagnostic process much more efficient than traditional manual anomaly checking. Based on the independent calculation of entropy sequences for single service instances, it takes into account service isolation and global fault awareness, avoiding the problem that traditional log analysis is difficult to adapt to multi-service and complex dependency scenarios in distributed systems.

[0019] For step S10, for example, it can be implemented through the following process: First, the log stream of each service instance log is parsed online into a structured log template sequence with corresponding semantic weights to obtain the log template sequence of each service instance. For example, the online log parsing algorithm Drain is used to convert the log stream into a structured log template sequence. In the example of this application, each log template is assigned a semantic weight, which can be obtained by querying the semantic weight corresponding to the log template in a mapping table that maps templates to weights. If a new template is obtained after parsing (e.g., the template is not found to exist in the mapping table during parsing), the semantic weight calibration process is initiated to determine the semantic weight for the new template. Subsequently, based on the log template sequence of each service instance, the entropy of each weighted log template within the first preset time window is calculated online to obtain the time series of the entropy of each weighted log template as the first preset time window slides, and the adaptive dynamic baseline is established for the time series of the entropy of each weighted log template. As an example, a semantic weight w_i is defined for each independent log template i, and the semantic weight w_i ranges from 0 to 1. The closer w_i is to 1, the stronger the semantic indication that the log template indicates the existence or impending failure of the system; the closer it is to 0, the more likely it is a regular informational log.

[0020] Figure 2 This is a schematic illustration of the semantic weight calibration process according to some embodiments of this application. When the semantic weight calibration process is triggered, step S20 is initiated, invoking a dedicated large language model. When the log parsing module identifies a new, unseen log template, a one-time weight calibration process is triggered. In some specific examples of this application, the dedicated large language model is a large language model trained or adjusted using specialized knowledge in the field of distributed system operation and maintenance, including, for example, massive amounts of fault work orders, abnormal log samples, and technical documents. In step S22, the semantic weight of the new template is obtained from the dedicated large language model through weight acquisition prompts, the weight acquisition prompts including the text content of the new template. Specifically, the text content of the new log template is encapsulated in a structured prompt, thereby forming a weight acquisition prompt, which is then input to the dedicated large language model. For example, the weight acquisition prompt is: "Please evaluate the probability that the following log template indicates a system failure, and give a score between 0.0 and 1.0. 1.0 represents a serious error, and 0.0 represents normal information. Log template: [Template content]". The dedicated large language model, upon receiving the weighted prompt, returns a floating-point tree, whose value represents the semantic weight of the new template. This semantic weight is stored in a mapping table for later use. Semantic weight calibration is performed once for each new log template to ensure high performance in online processing.

[0021] Figure 3This is a flowchart illustrating the online calculation of the weighted log template entropy within a first preset time window based on the log template sequence for each service instance, according to some embodiments of this application. For each service instance's log template sequence, the following can be executed: Figure 3 The process shown determines the weighted log template entropy of each service instance, and as the first preset time window slides, a weighted log template entropy sequence for that service instance is formed. The weighted log template sequences of multiple service instances are respectively imaged. Figure 3 The process, along with the sliding of the first preset time window, can form a weighted log template entropy sequence for multiple service instances. In step S30, the number of times the log template of each service instance appears within the first preset time window is counted. In step S32, the semantic weight of the log template of each service instance is obtained from the mapping table. In step S34, based on the counted frequency and the semantic weight, the weighted frequency of the log template of each service instance is calculated. In step S36, the total weighted frequency of the log templates of all service instances within the first preset time window is calculated. In step S38, based on the calculated weighted frequency of the log template of each service instance and the calculated total weighted frequency, the weighted probability of the log template of each service instance is determined. In step S39, the weighted log template entropy of each service instance is determined based on the weighted probability. For example, within 1 minute, the number of times log template i of a service instance appears, n_i; the semantic weight w_i of log template i is obtained from the mapping table. Calculate the weighted frequency n'_i using formula (1). Further, calculate the total weighted frequency N' of all log templates within 1 minute, which is the sum of each weighted frequency n'_i (i.e., N' = Σ(n'_i)). Calculate the weighted probability p'_i of each template within 1 minute using formula (2). Then calculate the weighted log template entropy H_sw using formula (3). Formula (1) is as follows: n'_i = n_i * (1 +α* w_i) (1) Here, α is an adjustable "weight amplification factor" (e.g., α=1) used to control the degree of influence of semantic weights on the original frequency. This formula means that logs with high weights (strong fault predictability) have a significantly amplified "presence" in entropy calculation.

[0022] Formula (2) is as follows: p'_i = n'_i / N' (2) Formula (3) is as follows: H_sw = - Σ (p'_i * log(p'_i)) (3) Thus, when a small number of high-weight error logs appear in a distributed system, H_sw will generate much more dramatic fluctuations than traditional entropy, allowing it to be captured by downstream time series prediction models earlier and more accurately.

[0023] For multiple weighted log template entropy sequences corresponding to multiple service instances, if the entropy value in any one of the sequences exceeds the confidence interval of the adaptive dynamic baseline for that sequence, an entropy change anomaly event is generated and published, for example, to an internal message queue. This event is structured data and may include: the identifier (ID) of the abnormal service instance, the anomaly timestamp, the actual entropy value (i.e., the calculated weighted log template entropy of that service instance), the baseline prediction value, the deviation magnitude, and other information.

[0024] According to the method in this application, entropy change anomaly events are continuously detected. If the number of entropy change anomaly events detected within a second preset time window (e.g., 5 minutes) is greater than the event threshold, root cause localization is performed to form a list of suspected anomaly root causes. Figure 4 This is a flowchart of root cause localization based on some examples of this application. In step S40, the set of entropy change anomaly events and service call graph required for root cause localization are obtained. Entropy change anomaly events can be published to an internal queue, and the set of all entropy change anomaly events within a second preset time window can be obtained from the queue. Specifically, the detected entropy change anomaly events within the second preset time window are obtained to form a set E, where the elements in the set are single entropy change anomaly events (e_i). A single entropy change anomaly event includes a service instance identifier (ID), an anomaly timestamp, an actual entropy value, a baseline prediction value, and a deviation magnitude. The set of entropy change anomaly events can be expressed as Equation (4): E = { e_1, e_2, …, e_k} (4) The service call graph illustrates the topological location and upstream / downstream information of each service instance. The service call graph is represented by a directed graph G = (V, E), where node vi represents a service instance, and edge e_ij represents a call from service i to service j. For each entropy-change anomaly event, there is an anomalous service instance (vi). In short, the service call graph can capture the call relationships between services using, for example, distributed tracing probes and link tracing systems, thereby generating a directed graph. This graph is typically cached in real-time or stored in memory for deep topology analysis, etc.

[0025] In step S42, for each abnormal service instance in the entropy change anomaly event set, the root cause suspicion score of each abnormal service instance is determined based on its abnormal timestamp, the information associated with the abnormal service instance in the service call graph, and the deviation magnitude. Specifically, the root cause suspicion score (Si) of each abnormal service instance can be calculated according to formula (5): Si=ωT×Tscore(vi)+ωP×Pscore(vi)+ωM×Mscore(vi) (5) Wherein, ωT, ωP, and ωM correspond to the weighting coefficients of time, topology, and drastic change, respectively. Tscore(vi) represents the time priority score of the service instance (vi); Pscore(vi) represents the topological position score of the service instance (vi); and Mscore(vi) represents the drastic change score of the service instance (vi). In the example of this application, based on the event timestamp, the earlier the service experiences an anomaly, the higher its time priority score; analyzing the position of the service on the service call graph, the more upstream the node (larger out-degree, smaller in-degree), the higher its topological position score; based on the deviation magnitude in the event, the more drastic the change in entropy value, the higher the drastic change score.

[0026] In step S44, after the calculation is completed, the suspicion scores of the abnormal service instances corresponding to all entropy change anomaly events are sorted. In step S45, the top K service instances with the highest scores in the sorting results are selected to generate a potential anomaly root cause suspected service list, wherein the anomaly root cause suspected service list includes service instance identifiers and their related anomaly information; wherein K is an integer and less than or equal to the number of entropy change anomaly events in the entropy change anomaly event set.

[0027] For each suspected service instance in the list of suspected root causes of anomalies, a diagnostic prompt word is generated, and the generated prompt word is input into the large language model. Figure 5 This is a flowchart of one embodiment for generating diagnostic prompts. In step S50, for each suspected service in the list of suspected root cause services, evidence related to the anomaly is obtained from the log database. Specifically, for each suspected service, the log database is queried back to extract key original log samples that caused the drastic change in entropy value before and after the anomaly point. In step S52, the service profile of the suspected service is retrieved from the configuration management database. Specifically, the configuration management database (CMDB) is accessed to obtain metadata such as the service's functional description, responsible person, and dependencies. In step S54, based on the anomaly log keywords, similar cases and solutions are queried from a preset knowledge base using retrieval enhancement generation technology. Specifically, using the anomaly log and the suspected service name as keywords, similar cases and solutions are retrieved from an internal knowledge base (such as historical fault tickets and maintenance documents) that serves as the preset knowledge base. In step S56, based on the evidence, the service profile, and the searched similar cases and solutions, diagnostic prompts are generated and input into the large language model. Specifically, the evidence, service profiles, similar cases, and solutions obtained in steps S50, S52, and S54 are integrated into a structured diagnostic prompt.

[0028] The process may further include step S57, where the large language model analyzes the fault based on this rich contextual information, generates intelligent diagnostic results, and outputs them. The intelligent diagnostic results may include a problem summary, possible cause deductions, and specific troubleshooting steps or repair suggestions. Further, in step S58, these intelligent diagnostic results are pushed to designated parties, such as the service manager's electronic terminal, via alert channels (e.g., DingTalk, WeChat Work).

[0029] The intelligent diagnostic method for distributed systems implemented in this application can proactively detect unknown anomalies, transforming the fault handling mode from passive response to proactive early warning. For example, in Figure 1 In step S10, by constructing a weighted log template entropy time series and combining it with the adaptive dynamic baseline anomaly detection in step S12, an unsupervised entropy change detection mechanism is formed. Unlike traditional rule-based monitoring systems that can only identify preset fault types, the solution in this application does not require pre-defined fault rules. It can capture silent faults that traditional monitoring cannot detect through abnormal fluctuations in entropy values, such as potential service performance degradation and implicit dependency anomalies, thereby proactively discovering risks before the impact of faults escalates, significantly improving the foresight of system operation and maintenance. The intelligent diagnostic method for distributed systems in this application embodiment can also significantly improve the accuracy and efficiency of root cause localization, quickly overcoming the alarm storm dilemma. Addressing the problem of low efficiency in manual troubleshooting under alarm storms in traditional operation and maintenance, this application employs a multi-dimensional scoring model (e.g., integrating time, topology, and the intensity of change) during the root cause localization process. Figure 4 Step S42). By obtaining the set of abnormal events and service call graph within the second preset time window, the root cause suspicion score of each abnormal service instance is calculated and sorted using a formula (e.g., Figure 4 Steps S44-S45 can accurately pinpoint the top K core root cause suspect services from massive anomaly events in seconds to minutes, narrowing the scope of investigation by more than 90%, changing the inefficient mode of traditional manual anomaly investigation one by one, and greatly improving the fault response speed.

[0030] Furthermore, the intelligent diagnostic method for distributed systems in this application embodiment can also realize the intelligentization and automation of the diagnostic process, significantly shortening the average repair time. For example, in Figure 1 Steps S16-S18 shown, and Figure 5The corresponding diagnostic prompt word construction process introduces a large language model to achieve automated diagnosis: Step S50 extracts evidence from abnormal logs, Step S52 calls the service profile, and Step S54 retrieves similar cases and solutions based on Retrieval Enhancement Generation (RAG) technology. This multi-dimensional information is integrated into structured prompt words and input into the large language model, ultimately directly outputting intelligent diagnostic results that include a problem summary, root cause inference, and troubleshooting and repair steps. This significantly reduces reliance on the experience of operations and maintenance personnel and drastically shortens the average fault repair time.

[0031] Furthermore, the intelligent diagnostic method for distributed systems in this application also improves the alarm signal-to-noise ratio and the robustness of the diagnostic mechanism, effectively filtering out interference from normal business fluctuations. Compared with traditional log anomaly detection methods (such as standard information entropy), which cannot distinguish the semantic importance of log templates and are easily interfered with by harmless log fluctuations, the method in this application effectively filters out interference from normal business fluctuations, significantly reducing the false alarm rate. Specifically, Figure 1 In the log parsing process of step S10, a "semantic weighted entropy" mechanism is introduced: through steps S20-S22 (see... Figure 2 The semantic weighting process utilizes a large language model trained with domain-specific knowledge of operations and maintenance to assign quantified semantic weights to different log templates (higher weights for stronger fault predictability). This is then followed by steps S30-S39 (see...). Figure 3 The weighted log template entropy is obtained by calculating the weighted frequency and weighted probability of the fault logs. This significantly amplifies the presence of high-weight fault logs in the entropy calculation. Combined with the Prophet model adaptive dynamic baseline in step S12, this effectively filters out interference from normal business fluctuations, significantly reduces the false alarm rate, and greatly improves the effectiveness of alarms and the robustness of system diagnosis.

[0032] Figure 6 This is a schematic diagram of the structure of an intelligent diagnostic system for a distributed system according to an embodiment of this application. Figure 6As shown, the system includes an anomaly detection module 60, a localization module 62, and a diagnosis module 64. The anomaly detection module 60 analyzes the logs of each service instance in the distributed system, using a first preset time window as a sliding window to determine the time series of the weighted log template entropy of each service instance. When the entropy value in any of the sequences exceeds the confidence interval of the adaptive dynamic baseline for that sequence, an entropy change anomaly event is generated and published, for example, to a message queue. This entropy change anomaly event includes at least an anomaly service instance identifier and an anomaly timestamp, and may also include the actual entropy value, the baseline prediction value, and the deviation magnitude. As an example, the confidence interval of the adaptive dynamic baseline is determined by, for example, a time series prediction model like Prophet. The localization module 62 performs root cause localization when the number of entropy change anomaly events detected within a second preset time window exceeds an event threshold, to form a list of suspected root cause services. The diagnosis module 64 generates diagnostic prompts for each suspected service in the list of suspected root cause services and inputs these prompts into a large language model for analysis and intelligent diagnostic results.

[0033] In a further example, the anomaly detection module 60 includes a parsing unit 601 and an entropy calculation unit 602. The parsing unit 601 is used to parse the log stream of each service instance's logs online into a structured log template sequence with corresponding semantic weights, thereby obtaining a log template sequence for each service instance. The entropy calculation unit 602, based on the log template sequence of each service instance, calculates the entropy of each weighted log template within the first preset time window online, so as to obtain a sequence of the entropy of each weighted log template as the first preset time window slides, and establishes the adaptive dynamic baseline for each sequence of weighted log template entropy. In a further example, the entropy calculation unit 602 is configured to perform... Figure 3 The process involves calculating the weighted log template entropy for each service instance.

[0034] Furthermore, the diagnostic system also includes a semantic weighting module 66. When triggered, the semantic weighting module 66 determines semantic weights for the new template and stores the determined semantic weights in a mapping table; wherein the mapping table is a mapping table between corresponding templates and weights. The parsing unit 601 is also configured to, during the online parsing of the log stream of each service instance log into a sequence of structured log templates, if the log template is a new template, trigger the semantic weighting module to calibrate the new template. In a more specific example, the semantic weighting module 66 is configured to, for example... Figure 2 The process shown is used to calibrate the new template.

[0035] For example, the localization module 62 is configured to acquire the set of entropy-changing abnormal events and the service call graph required for root cause localization; for each abnormal service instance in the set of entropy-changing abnormal events, the localization module 62 determines the root cause suspicion score of each abnormal service instance based on its abnormal timestamp and the information associated with the abnormal service instance in the service call graph. Further, the localization module 62 sorts the root cause suspicion scores of all abnormal service instances; selects the top K service instances with the highest scores in the sorting result to generate a potential root cause suspicion service list, wherein the list contains service instance IDs and corresponding abnormal information; wherein K is an integer less than or equal to the number of entropy-changing abnormal events in the set of entropy-changing abnormal events. The localization module 62 can perform, for example, referencing... Figure 4 The process described.

[0036] The diagnostic module 64 may include a prompt word information acquisition unit 640 and a prompt word generation unit 642. For each suspected service in the list of suspected root cause services, the prompt word information acquisition unit 640 acquires evidence related to the anomaly from the log database; retrieves the service profile of the suspected service from the configuration management database; and, based on anomaly log keywords, queries similar cases and solutions from a preset knowledge base using retrieval enhancement generation technology. The prompt word generation unit 642, based on the evidence, the service profile, and the searched similar cases and solutions, generates diagnostic prompt words and inputs them into the large language model. As an example, the diagnostic module 64 may perform a combination of... Figure 5 The process described includes, for example, the prompt word information acquisition unit 640 executing steps S50, S52 and S54; and the prompt word generation unit 642 executing step 56.

[0037] According to embodiments of this application, a large language model analyzes the fault based on this rich contextual information, forms an intelligent diagnostic result, and outputs it as feedback to diagnostic prompts. Figure 6 The system shown can also be configured to execute Figure 5 Steps S57 and S58, as shown, involve pushing these intelligent diagnostic results to designated parties, such as the electronic terminal of the service manager, through alert channels (such as DingTalk, WeChat Work, etc.).

[0038] all in all, Figure 6 The intelligent diagnostic system for distributed systems shown can be used to perform the above-mentioned combined... Figures 1 to 5 Any of the methods described in the embodiments. As an example, the intelligent diagnostic system can be implemented through a combination of software and hardware, such as implementing the functions of each module or unit in a programming language and loading these programming languages ​​into, for example, a processor or hardware including a processor.

[0039] Figure 7This is a structural diagram of an electronic device according to another embodiment of the present application. The electronic device includes a memory 70 and a processor 72. The memory stores instructions, and the processor 72, when executing these instructions, can realize the above-described combination. Figures 1 to 5 Any one of the described method embodiments.

[0040] This application also provides a program product, which includes instructions that, when executed, enable the above-mentioned combination. Figures 1 to 5 Any of the methods described in the embodiments. The program product can be downloaded via a network or stored on a storage medium for use.

[0041] This application also provides a non-volatile storage medium on which instructions are stored, which, when executed, can achieve the above-mentioned combination. Figures 1 to 5 Any one of the described method embodiments.

[0042] The distributed system intelligent diagnostic schemes provided in the embodiments of this application can proactively discover unknown silent faults through semantic weighted entropy and adaptive dynamic baseline design, effectively filter out interference from normal business fluctuations, and significantly reduce the false alarm rate. With the help of multi-dimensional scoring models, the root cause can be quickly and accurately located from alarm storms. Combined with large language models and RAG technology, automated intelligent diagnosis and solution generation can be achieved, significantly shortening the average fault repair time. At the same time, the automated accumulation of operation and maintenance knowledge and the evolution of system diagnostic capabilities are realized, comprehensively improving the automation and intelligence level of distributed system operation and maintenance, reducing operation and maintenance costs, and enhancing system operation stability.

[0043] In the method flow described in conjunction with the accompanying drawings, the labels for each step are merely for distinguishing different steps, and not for limiting the data to be performed on each step. For example... Figure 5 Steps S50, S52, and S54 in the process do not have a specific execution order; they can be executed in parallel or in any order.

[0044] The technical features in the various embodiments of this application can be combined with each other to form new implementation methods without departing from the spirit of this application and without conflicting with each other. Although specific embodiments of this application have been shown and described in detail to illustrate the principles of this application, it should be understood that this application can be implemented in other ways without departing from such principles.

Claims

1. An intelligent diagnostic method for distributed systems, characterized in that, The intelligent diagnostic method includes: Based on the logs of each service instance in the distributed system, the sequence of weighted log template entropy for each service instance is determined using a first preset time window as a sliding window; When the entropy value of any of the sequences exceeds the confidence interval of the adaptive dynamic baseline of that sequence, an entropy change anomaly event is generated and published; wherein, the confidence interval of the adaptive dynamic baseline is determined by the time series prediction model, and the entropy change anomaly event includes at least an abnormal service instance identifier and an abnormal timestamp; When the number of entropy change anomalies detected within the second preset time window exceeds the event threshold, root cause localization is performed to form a list of suspected anomaly root causes. For each suspected service in the list of suspected anomaly root causes, a diagnostic prompt word is generated, and the generated prompt word is input into the large language model; The large language model is used for analysis to generate intelligent diagnostic results.

2. The method according to claim 1, characterized in that, Based on the logs of each service instance in the distributed system, a sequence of weighted log template entropy is determined using a first preset time window as a sliding window, including: The log stream of each service instance is parsed online into a sequence of structured log templates with corresponding semantic weights to obtain the log template sequence for each service instance. Based on the log template sequence of each service instance, the entropy of each weighted log template within the first preset time window is calculated online, so as to obtain the sequence of each weighted log template entropy as the first preset time window slides, and the adaptive dynamic baseline is established for the sequence of each weighted log template entropy.

3. The method according to claim 2, characterized in that, The log stream of each service instance is parsed online into a sequence of structured log templates with corresponding semantic weights to obtain the log template sequence for each service instance, including: The log stream of each service instance is parsed online into a sequence of structured log templates; If the log template is determined to be a new template during the parsing process, a semantic weight calibration process is triggered to determine the semantic weight for the new template and to store the determined semantic weight in a mapping table; wherein, the mapping table is a mapping table for corresponding templates and weights.

4. The method according to claim 3, characterized in that, The semantic weighting process includes: Invoke the dedicated large language model; The semantic weights of the new template are obtained from the dedicated large language model by obtaining prompt words based on weights, wherein the prompt words for obtaining weights include the text content of the new template; The semantic weight of the new template is then stored in the mapping table. The dedicated large language model is pre-trained based on a dedicated knowledge base of the distributed system.

5. The method according to claim 2, characterized in that, Based on the log template sequence of each service instance, the weighted log template entropy of each service instance within the first preset time window is calculated online, including: Count the number of times the log template of each service instance appears within the first preset time window; Obtain the semantic weight of the log template for each service instance from the mapping table; The weighted frequency of the log template for each service instance is calculated based on the count and the semantic weight. Calculate the weighted total frequency of log templates for all service instances within the first preset time window; The weighted probability of the log template of each service instance is determined based on the weighted frequency of the log template of each service instance and the calculated total weighted frequency. The weighted log template entropy of each service instance is determined based on the weighted probability.

6. The method according to claim 1, characterized in that, The root cause localization includes: Obtain the set of entropy-variable abnormal events and the service call graph required for root cause localization; For each abnormal service instance in the entropy change abnormal event set, the root cause suspicion score of each abnormal service instance is determined based on its abnormal timestamp and the information associated with the abnormal service instance in the service call graph.

7. The method according to claim 6, characterized in that, A list of suspected root causes of the anomalies was created, including: Sort the root cause suspicion scores of all abnormal service instances; Select the top K service instances with the highest scores from the sorting results to generate a list of suspected potential anomaly root causes. The list includes the service instance ID and the corresponding anomaly information. K is an integer less than or equal to the number of entropy change anomaly events in the entropy change anomaly event set.

8. The method according to claim 1, characterized in that, For each suspected service in the list of suspected root causes of anomalies, diagnostic prompts are generated, and these prompts are input into the large language model, including: For each suspected service in the list of suspected root causes of anomalies, obtain evidence related to the anomaly from the log database; retrieve the service profile of the suspected service from the configuration management database; and query similar cases and solutions from a preset knowledge base based on the keywords in the anomaly logs using retrieval enhancement generation technology. Based on the evidence, the service profile, and the searched similar cases and solutions, diagnostic prompts are generated and input into the large language model; The service profile includes at least a description of the suspected service's functions and its upstream and downstream dependencies.

9. An intelligent diagnostic system for distributed systems, characterized in that, The intelligent diagnostic system includes: An anomaly detection module is used to determine the sequence of weighted log template entropy for each service instance in the distributed system using a first preset time window as a sliding window. When the entropy value in any of the sequences exceeds the confidence interval of the adaptive dynamic baseline of that sequence, an entropy change anomaly event is generated and published. The confidence interval of the adaptive dynamic baseline is determined by a time series prediction model. The entropy change anomaly event includes at least an abnormal service instance identifier and an abnormal timestamp. The positioning module is used to perform root cause localization when the number of entropy change anomaly events detected within the second preset time window exceeds the event threshold, so as to form a suspected service list of anomaly root causes. The diagnostic module is used to generate diagnostic prompts for each suspected service in the list of suspected root causes of anomalies, and input the generated diagnostic prompts into a large language model for analysis and generation of intelligent diagnostic results.

10. The system according to claim 9, characterized in that, The anomaly detection module includes: The parsing unit is used to parse the log stream of each service instance into a structured log template sequence with corresponding semantic weights online, so as to obtain the log template sequence of each service instance. The entropy calculation unit calculates the entropy of each weighted log template within the first preset time window based on the log template sequence of each service instance, so as to obtain the sequence of the entropy of each weighted log template as the first preset time window slides, and establishes the adaptive dynamic baseline for the sequence of the entropy of each weighted log template.

11. The system according to claim 10, characterized in that, The system also includes a semantic weighting module, and The parsing unit is also configured to trigger the semantic weight labeling module if the log template is a new template during the process of parsing the log stream of each service instance into a structured log template sequence online. The semantic weight labeling module determines the semantic weight for the new template when it is triggered, and stores the determined semantic weight in a mapping table. The mapping table is used to map templates to weights.

12. The system according to claim 11, characterized in that, The semantic weighting module is configured as follows: Invoke the dedicated large language model; The semantic weights of the new template are obtained from the dedicated large language model by obtaining prompt words based on weights, wherein the prompt words for obtaining weights include the text content of the new template; The dedicated large language model is pre-trained based on a dedicated knowledge base of the distributed system.

13. The system according to claim 10, characterized in that, The entropy calculation unit is also configured to calculate the weighted log template entropy for each service instance through the following process: Count the number of times the log template of each service instance appears within the first preset time window; Obtain the semantic weight of the log template for each service instance from the mapping table; The weighted frequency of the log template for each service instance is calculated based on the count and the semantic weight. Calculate the weighted total frequency of log templates for all service instances within the first preset time window; The weighted probability of the log template of each service instance is determined based on the weighted frequency of the log template of each service instance and the calculated total weighted frequency. The weighted log template entropy of each service instance is determined based on the weighted probability.

14. The system according to claim 9, characterized in that, The positioning module is configured to: Obtain the set of entropy-variable abnormal events and the service call graph required for root cause localization; For each abnormal service instance in the entropy change abnormal event set, the root cause suspicion score of each abnormal service instance is determined based on its abnormal timestamp and the information associated with the abnormal service instance in the service call graph.

15. The system according to claim 14, characterized in that, The location module is configured to generate a list of suspected anomaly root causes services through the following process: Sort the root cause suspicion scores of all abnormal service instances; Select the top K service instances with the highest scores from the sorting results to generate a list of suspected potential anomaly root causes. The list includes the service instance ID and the corresponding anomaly information. K is an integer less than or equal to the number of entropy change anomaly events in the entropy change anomaly event set.

16. The system according to claim 9, characterized in that, The diagnostic module includes: The prompt word information acquisition unit is used to obtain evidence related to the anomaly from the log database for each suspected service in the list of suspected services for the root cause of the anomaly; call the service profile of the suspected service from the configuration management database; and query similar cases and solutions from a preset knowledge base based on the keywords in the anomaly logs through retrieval enhancement generation technology. The prompt word generation unit is used to generate diagnostic prompt words based on the evidence, the service profile, and the searched similar cases and solutions, and input them into the large language model; The service profile includes at least a description of the suspected service's functions and its upstream and downstream dependencies.

17. An electronic device, characterized in that, The electronic device includes: Memory, used to store instructions; A processor configured to implement the method according to any one of claims 1 to 8 when executing the instructions.

18. A program product, characterized in that, The program product includes instructions that, when executed, enable the implementation of the method according to any one of claims 1 to 8.

19. A non-volatile storage medium, characterized in that, Instructions are stored on the non-volatile storage medium, and when executed, the instructions are capable of implementing the method according to any one of claims 1 to 8.

Images