Distributed system fault validation method, apparatus, device, and product

Abstract

The application discloses a distributed system fault confirmation method, device and equipment and product, and relates to the technical field of fault diagnosis. The method comprises the following steps: intercepting original logs of a distributed system through an interception component to obtain interception logs, and degrading the interception logs according to an invalid error feature library to obtain structured logs; classifying the structured logs through an expert rule engine to obtain classified alarm results; supplementarily classifying the classified alarm results through a pre-constructed knowledge distillation classification model to obtain refined classification results, and confirming a final fault result according to the classified alarm results and the refined classification results. Thus, the problems of excessive log classification alarm, rigid rules, incomplete coverage and difficult real-time processing of model reasoning are solved, and the efficiency of distributed system fault confirmation is improved.

Description

Technical Field

[0001] This application relates to the field of fault diagnosis technology, and in particular to a method, apparatus, equipment and product for confirming faults in a distributed system. Background Technology

[0002] Currently, the operation and maintenance monitoring of large-scale distributed software systems generally adopts a technical solution that combines threshold rule alarms, keyword matching, and manual troubleshooting. The system pre-sets thresholds or abnormal keyword rules for various indicators. When a matching item appears in the log, an alarm is triggered, and operation and maintenance personnel can locate the fault accordingly. In order to improve the accuracy of anomaly detection, the industry has tried to introduce pre-trained language models such as BERT for classification. However, due to the limited number of model parameters and inference efficiency, engineering implementation faces challenges.

[0003] However, existing technologies still have significant shortcomings. First, the abuse of log levels leads to a proliferation of alarms, with many expected or negligible anomalies being marked as ERROR level, while truly fatal anomalies are drowned out by noise, causing alarm fatigue. Second, the hierarchical methods based on keywords or thresholds are rigid and cannot semantically distinguish the severity of anomalies, resulting in frequent false alarms and missed alarms, and they are difficult to cover the constantly evolving business anomaly patterns. Moreover, expert rules are limited by human experience and cannot exhaust all anomaly types, resulting in coverage blind spots when facing new or combined anomalies. In addition, deep learning models have high inference latency and cannot meet the real-time streaming processing requirements of massive logs in a general-purpose CPU environment.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a method, apparatus, device and product for fault confirmation in a distributed system, which aims to solve the technical problems of rampant log hierarchical alarms, rigid and incomplete rules, and difficulty in real-time processing of model inference.

[0006] To achieve the above objectives, this application proposes a method for acknowledging faults in a distributed system, the method comprising: The raw logs of the distributed system are intercepted by the interception component to obtain intercepted logs, and the intercepted logs are downgraded according to the invalid error feature library to obtain structured logs. The structured logs are classified using an expert rule engine to obtain classified alert results. The graded alarm results are supplemented by a pre-built knowledge distillation classification model to obtain refined graded results, and the final fault result is confirmed based on the graded alarm results and refined graded results.

[0007] In one embodiment, the step of intercepting the raw logs of the distributed system using an interception component to obtain intercepted logs includes: The raw log data is obtained by intercepting the raw logs output by the business nodes of the distributed system using an interception component. The original log data is filtered out to obtain candidate log data, which has a log level of not less than the error level. Extract the timestamp, service name, interface name, and log level of the candidate log data, and obtain the context information corresponding to the timestamp; Interception logs are generated based on the context information, timestamp, service name, interface name, and log level.

[0008] In one embodiment, the step of classifying the structured logs using an expert rule engine to obtain classified alert results includes: Based on the exception type, keyword features, and stack pattern in the structured log, the structured log is classified into levels by the expert rule engine to obtain hierarchical logs; Based on the service name and interface name in the structured log, window aggregation statistics are performed on the hierarchical log to obtain window statistics results; Alarm detection is performed on the window statistics results using dynamic thresholds to obtain tiered alarm results.

[0009] In one embodiment, before the step of supplementing the classification of the graded alarm results using a pre-built knowledge distillation classification model to obtain refined classification results, the method further includes: A lightweight code model is trained by distillation using a pre-trained code model, and independent classification exits are set after several Transformer encoding layers of the lightweight code model to obtain a pre-trained distillation model. The pre-trained distillation model is trained using log training samples to obtain the initial distillation model; The initial distillation model is weighted and quantized to obtain a knowledge distillation classification model.

[0010] In one embodiment, the step of performing weight quantization on the initial distillation model to obtain a knowledge distillation classification model includes: The linear layer weights in the initial distillation model are quantized to obtain the quantized linear layer weights. The quantization scaling factor is determined based on the numerical range of the linear layer weights; The knowledge distillation classification model is obtained by calibrating the quantized linear layer weights using the quantization scaling factor.

[0011] In one embodiment, the step of supplementing the classification of the graded alarm results using a pre-built knowledge distillation classification model to obtain refined classification results includes: The knowledge distillation classification model is used to reason layer by layer about the graded alarm results to obtain the initial graded results and the confidence level of the results output by the classification exit. Extract the target classification results from the initial classification results whose confidence level exceeds the confidence threshold; Extract the set of unlabeled samples from the initial classification results whose confidence level is lower than the confidence threshold; A refined classification result is generated based on the target classification result.

[0012] In one embodiment, the step of confirming the final fault result based on the graded alarm result and the refined graded result includes: The hierarchical alarm results and the refined hierarchical results are merged and aligned according to a preset aggregation dimension to obtain a fused hierarchical result; The knowledge distillation classification model is trained using the unlabeled sample set to obtain an updated knowledge distillation classification model; The updated knowledge distillation classification model is used to reclassify the graded alarm results to obtain updated refined graded results. The final fault result is generated based on the fusion classification result and the updated refined classification result.

[0013] Furthermore, to achieve the above objectives, this application also proposes a distributed system fault confirmation device, which includes: The interception module is used to intercept the raw logs of the distributed system through the interception component to obtain intercepted logs, and to downgrade the intercepted logs according to the invalid error feature library to obtain structured logs; The grading module is used to grade the structured logs through an expert rule engine to obtain grading alarm results; The confirmation module is used to supplement the classification of the graded alarm results by a pre-built knowledge distillation classification model to obtain refined classification results, and to confirm the final fault result based on the graded alarm results and the refined classification results.

[0014] In addition, to achieve the above objectives, this application also proposes a distributed system fault confirmation device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the distributed system fault confirmation method as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the distributed system fault confirmation method described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the distributed system fault confirmation method described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: This application proposes a method, apparatus, device, and product for confirming faults in a distributed system. It intercepts raw logs from a distributed system using an interception component to obtain intercepted logs. These intercepted logs are then downgraded according to an invalid error feature library to obtain structured logs. An expert rule engine then classifies these structured logs to obtain graded alarm results. A pre-built knowledge distillation classification model further classifies these graded alarm results to obtain refined classification results. Finally, the fault result is confirmed based on these graded alarm results and refined classification results. Thus, by first downgrading the intercepted logs according to the invalid error feature library to filter out expected error noise at the source and suppress alarm proliferation, and then using the expert rule engine to classify the structured logs, the severity is differentiated by rules to alleviate rule rigidity. For coverage blind spots where rules are insufficient, a lightweight knowledge distillation model is used to supplement the classification of rule results while ensuring real-time CPU inference, resulting in refined classification results. Finally, the rule and model results are merged to confirm the fault. This solves the problems of rampant log-based graded alarms, rigid and incomplete rule coverage, and difficulty in real-time model inference processing in existing systems, thereby improving the efficiency of fault confirmation in distributed systems. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the distributed system fault confirmation method of this application. Figure 2This is a schematic diagram illustrating the teacher-student model training involved in the distributed system fault confirmation method of this application; Figure 3 This is a flowchart illustrating Embodiment 2 of the distributed system fault confirmation method of this application; Figure 4 This is a simplified flowchart of the distributed system fault confirmation method provided in Embodiment 2 of this application; Figure 5 This is a schematic diagram of the module structure of the distributed system fault confirmation device according to an embodiment of this application; Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the distributed system fault confirmation method in this application embodiment.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution of this application embodiment is as follows: The raw logs output by the business nodes of the distributed system are intercepted by an interception component to obtain raw log data; log data with a log level not lower than the error level is selected from the raw log data to obtain candidate log data; the timestamp, service name, interface name, and log level of the candidate log data are extracted, and the context information corresponding to the timestamp is obtained; interception logs are generated based on the context information, timestamp, service name, interface name, and log level. Based on the exception type, keyword features, and stack trace in the structured logs, the structured logs are classified into levels using the expert rule engine to obtain hierarchical logs; window aggregation statistics are performed on the hierarchical logs based on the service name and interface name in the structured logs to obtain window statistical results; alarm detection is performed on the window statistical results using dynamic thresholds to obtain hierarchical alarm results. A lightweight code model is distilled using a pre-trained code model, and independent classification exits are set after several Transformer encoding layers of the lightweight code model to obtain a pre-trained distillation model. The pre-trained distillation model is then trained using log training samples to obtain an initial distillation model. The initial distillation model undergoes weight quantization to obtain a knowledge distillation classification model. The linear layer weights in the initial distillation model are quantized to obtain quantized linear layer weights. A quantization scaling factor is determined based on the numerical range of the linear layer weights. The quantized linear layer weights are calibrated using the quantization scaling factor to obtain the knowledge distillation classification model. The knowledge distillation classification model is used to perform layer-by-layer reasoning on the graded alarm results to obtain the initial graded results and result confidence levels output by the classification exits. Target graded results with confidence levels exceeding a confidence threshold are extracted from the initial graded results. A set of unlabeled samples with confidence levels below the confidence threshold are extracted from the initial graded results. A refined graded result is generated based on the target graded results. The hierarchical alarm results and refined hierarchical results are merged and aligned according to a preset aggregation dimension to obtain a fused hierarchical result. The knowledge distillation classification model is trained using the unlabeled sample set to obtain an updated knowledge distillation classification model. The hierarchical alarm results are then reclassified using the updated knowledge distillation classification model to obtain an updated refined hierarchical result. Finally, a fault result is generated based on the fused hierarchical result and the updated refined hierarchical result. This solves the problems of rampant log hierarchical alarms, rigid and incomplete rules, and difficulty in real-time model inference processing in existing systems, enabling fault confirmation in distributed systems and improving the efficiency of fault confirmation.Based on the solution of this invention, this paper addresses the problems of log level abuse leading to alarm overload, where a large number of expected or negligible anomalies are marked as ERROR level, and real fatal anomalies are drowned out by noise, causing alarm fatigue. Secondly, the rigid hierarchical methods based on keywords or thresholds cannot semantically distinguish the severity of anomalies, resulting in frequent false positives and false negatives, and are difficult to cover constantly evolving business anomaly patterns, thus leading to low efficiency. Therefore, this paper designs a distributed system fault confirmation method and verifies the effectiveness of the method when confirming faults in a distributed system. Finally, the efficiency of distributed system fault confirmation using the method of this invention is significantly improved.

[0025] In this embodiment, for ease of description, the following description uses a distributed system fault confirmation device as the execution subject.

[0026] Due to the limitations of existing log monitoring and intelligent operation and maintenance technologies, the efficiency of distributed system fault confirmation needs improvement. One issue is the proliferation of alarms. Many expected or negligible anomalies in distributed systems are marked as error levels, while truly fatal anomalies are drowned out by noise, leading to alarm fatigue. If effective fault signals cannot be distinguished from massive amounts of redundant alarms in a timely manner, this will reduce the efficiency of distributed system fault confirmation. Another issue is rigid hierarchical classification. When keyword- or threshold-based classification methods cannot semantically distinguish the severity of anomalies, false positives and false negatives occur frequently if the semantic features of anomalies are not considered, significantly reducing the efficiency of distributed system fault confirmation. Furthermore, there is the problem of incomplete rule coverage. Expert rules are limited by human experience and cannot exhaustively cover all anomaly types. However, when facing new or combined anomalies, rules have coverage blind spots, which also affects the efficiency of distributed system fault confirmation. Therefore, real-time processing also faces challenges in current distributed system operation and maintenance. Due to the large number of parameters and high inference latency of deep learning models, it is difficult to meet the real-time streaming processing requirements of massive logs in a general-purpose CPU environment. If the fault confirmation model is not lightweighted and inference accelerated, its efficiency will also decrease.

[0027] This application provides a solution that first downgrades intercepted logs according to an invalid error feature library to filter out expected error noise at the source and suppress alarm proliferation. Then, an expert rule engine grades the structured logs, using rules to differentiate severity and alleviate rule rigidity. For coverage blind spots where rules are difficult to exhaustively cover, a lightweight knowledge distillation model is used to take over the teacher's capabilities. While ensuring real-time CPU inference, the rule results are supplemented with classification to obtain refined grading results. Finally, the rule and model results are integrated to confirm the fault. This solves the problems of rampant alarms in existing log grading systems, rigid and incomplete rule coverage, and difficulty in real-time processing of model inference, thus improving the efficiency of fault confirmation in distributed systems.

[0028] Based on this, embodiments of this application provide a method for fault confirmation in a distributed system, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the distributed system fault confirmation method of this application.

[0029] In this embodiment, the distributed system fault confirmation method includes steps S01 to S03: Step S01: The original logs of the distributed system are intercepted by the interception component to obtain intercepted logs, and the intercepted logs are downgraded according to the invalid error feature library to obtain structured logs; Before the implementation of this embodiment, it should be clear that the abuse of log levels in the prior art has led to a flood of alarms. A large number of expected or negligible anomalies are marked as ERROR level, and real fatal anomalies are drowned out by noise, causing alarm fatigue. Secondly, the hierarchical method based on keywords or thresholds is rigid and cannot semantically distinguish the severity of anomalies, resulting in frequent false alarms and missed alarms. Moreover, it is difficult to cover the constantly evolving business anomaly patterns. In addition, expert rules are limited by human experience and cannot exhaust all anomaly types, resulting in coverage blind spots when facing new or combined anomalies.

[0030] Therefore, in order to solve the above problems, this embodiment deploys an independent and reusable log interception basic component in the log printing link of each business service in the distributed system, and adopts the AOP aspect-oriented programming concept to achieve non-intrusive interception of business without modifying the business code and log printing logic.

[0031] The system pre-maintains a dynamically configurable, hot-update invalid error feature library. The feature library includes regular expressions and keyword features for various expected and negligible ERROR level logs, such as parameter validation failures, legitimate business rejections, proactive downgrade fallbacks, and harmless anomalies that can be captured.

[0032] Before the logs are output to the ground, the aspect dynamically intercepts the raw logs generated by all business nodes, performs automatic downgrading processing on ERROR-level logs that hit the invalid error feature library, and downgrades their log level from ERROR to WARN, filtering out massive invalid alarm noise from the source, while fully preserving the original log level metadata, stack information and business context for subsequent audit backtracking and sample collection.

[0033] After noise reduction and cleaning, the logs are uniformly organized into a standardized JSON structured format that includes timestamps, service names, interface names, log levels, exception types, and stack summaries. The logs are then uniformly reported via a message queue, providing a unified data source with high signal-to-noise ratio and high standardization for downstream streaming hierarchical statistics and intelligent model classification. This avoids the problems of alarm overload and real faults being drowned out by noise from the data base level.

[0034] Step S02: The structured logs are classified using an expert rule engine to obtain classified alarm results; Based on high-quality structured log data obtained through real-time consumption of message queues, the system is equipped with a configurable expert rule engine that supports hot updates and canary releases. Relying on the Flink streaming computing framework, it achieves low-latency, high-throughput real-time log parsing and anomaly classification.

[0035] Based on the exception types, unique keyword features, program stack error structure and error path of the structured logs, the rule engine achieves fine-grained exception level determination and completes three levels of interpretable classification: FATAL severe exception, ERROR general exception, and WARN warning exception.

[0036] To address the issue of frequent alarm triggering from single instantaneous abnormal logs, causing operational alarm jitter and invalid pushes, the system opens a sliding time window with "service name" and "interface name" as the sole aggregation dimensions. It aggregates and statistically analyzes the occurrence frequency, incremental changes, year-on-year and month-on-month fluctuations, and abnormal percentage changes of various abnormal logs within the window, generating multi-dimensional window statistical results.

[0037] Meanwhile, an adaptive dynamic threshold mechanism is introduced, which can automatically adjust the alarm trigger threshold according to the normal business traffic and error baseline, avoiding false alarms and missed alarms under the fixed threshold in the business peak and trough scenario. Finally, it outputs stable, accurate, interpretable and low jitter standardized hierarchical alarm results, and achieves near real-time fallback coverage of normal known business anomalies.

[0038] Step S03: The graded alarm results are supplemented and classified using a pre-built knowledge distillation classification model to obtain refined graded results. The final fault result is then confirmed based on the graded alarm results and the refined graded results.

[0039] To address the limitations of expert manual rules, which are subject to experience constraints and cannot exhaust all anomaly patterns, thus failing to cover novel unknown anomalies, combined superposition anomalies, and boundary scene anomalies, this embodiment introduces a lightweight knowledge distillation intelligent classification model to perform secondary supplementary classification on the rule classification results, constructing a collaborative classification system of "rule as a safety net and model to fill in the gaps".

[0040] During the model training phase, a high-quality three-level labeled dataset is constructed using a large language model-assisted semi-automatic annotation. Data augmentation and imbalanced sampling strategies are used to optimize the sample distribution. The model structure adopts a teacher-student knowledge distillation and multi-exit dynamic inference architecture, combined with a refined INT8 weight quantization strategy. While ensuring classification accuracy close to that of large models, the number of model parameters and CPU inference latency are significantly reduced, making it suitable for real-time streaming processing scenarios with massive logs.

[0041] After the model is launched, it performs layer-by-layer semantic reasoning and refined classification on the hierarchical alarm results output by expert rules to make up for the blind spots of manual rules.

[0042] Meanwhile, based on the high and low confidence levels of inference, reliable samples and samples to be optimized are distinguished. Low-confidence, high-value samples are used to continuously drive the model to self-iterate and update, constantly adapting to new abnormal scenarios generated by business iterations. Finally, the interpretable rule classification results and the highly covered model fine classification results are integrated to output the final fault judgment result that takes into account accuracy, real-time performance, and completeness.

[0043] Specifically, step S01 above, which involves intercepting the raw logs of the distributed system using an interception component to obtain the intercepted logs, includes the following steps: Step S011: The raw logs output by the business nodes of the distributed system are intercepted by the interception component to obtain the raw log data; Step S012: Filter out log data with a log level not lower than the error level from the original log data to obtain candidate log data; Step S013: Extract the timestamp, service name, interface name, and log level of the candidate log data, and obtain the context information corresponding to the timestamp; Step S014: Generate an interception log based on the context information, timestamp, service name, interface name, and log level.

[0044] During the normal operation of a distributed system microservice cluster, all business applications uniformly introduce a lightweight log interception basic component as a basic dependency. The component relies on the native printing pointcut of the log framework to achieve non-intrusive interception and collection of all raw logs. It does not interfere with business logic, does not affect interface performance or business link stability, and can comprehensively collect all raw log data generated during the operation of each business node, service interface, and business link.

[0045] To reduce the computational overhead of backend streaming computing and intelligent classification, and to achieve precise focus on abnormal log processing, the system performs initial screening and filtering on all raw log data, retaining only abnormal log data with a log level greater than or equal to ERROR, and removing normal logs such as INFO, DEBUG, and regular WARN that have no fault risk, thus obtaining candidate abnormal log data to be processed.

[0046] The system performs in-depth analysis and field extraction on candidate logs, accurately capturing core structured fields such as the precise timestamp, the name of the service to which it belongs, the name of the requested interface, and the original log level for each abnormal log. At the same time, it captures complete context information such as the corresponding business call chain, input and output parameters, and upstream and downstream service call records based on the interval before and after the timestamp, thus fully restoring the business scenario and operating status at the time of the abnormality.

[0047] The system integrates all structured fields and contextual scenario data, completes data regularization, deduplication, and field completion, and generates standardized interception logs with complete fields, traceable scenarios, and unified formats. It completes the accurate collection, filtering, and preliminary structured preprocessing of distributed abnormal logs, laying the data foundation for subsequent noise reduction, degradation, and intelligent classification.

[0048] More specifically, step S02 above, which involves classifying the structured logs using an expert rule engine to obtain graded alarm results, includes: Step S021: Based on the exception type, keyword features, and stack pattern in the structured log, the structured log is classified into levels using the expert rule engine to obtain hierarchical logs; Step S022: Based on the service name and interface name in the structured log, perform window aggregation statistics on the hierarchical log to obtain the window statistics results; Step S023: Alarm detection is performed on the window statistics results using dynamic thresholds to obtain graded alarm results.

[0049] The preprocessed structured logs are uniformly pushed to the streaming computing cluster. The system uses a built-in hot-updateable expert rule engine to accurately classify log anomaly levels. The rule engine has a complete set of anomaly judgment logic, which can perform fine-grained level identification of individual structured logs based on multi-dimensional features such as anomaly type, error keyword combination, program stack error level and error characteristics, and anomaly throwing location obtained from parsing the structured logs, strictly distinguishing three anomaly levels: FATAL critical exceptions are fatal exceptions that affect core business processes and can lead to business interruption, data corruption, or service crashes. They include high-risk issues such as SQL structured query exceptions, null pointer exceptions, memory overflow (OOM), database / connection pool exhaustion, and distributed consistency exceptions.

[0050] ERROR is a general logical exception that does not affect the overall chain and only causes local business failure. It includes non-fatal interface call failure, business data verification failure, and unexpected single business error.

[0051] WARN warnings and exceptions are self-healing and recoverable normative prompts and minor degradation exceptions, including non-core function degradation logs, retryable short timeouts, code style verification prompts, etc. Finally, the full log is processed in a hierarchical manner to obtain a standardized hierarchical log dataset.

[0052] To avoid invalid alarms triggered by single instantaneous anomalies and causing operational jitter, the system uses service name + interface name as the aggregation dimension and opens a sliding time window with customizable duration. It performs comprehensive aggregation statistics on the number of occurrences, instantaneous increments, year-on-year and month-on-month fluctuations, and sudden changes in the proportion of anomalies at each level within the window, generating refined window statistical results.

[0053] The system combines a business baseline adaptive dynamic threshold algorithm to dynamically adjust the alarm trigger threshold based on historical normal error data and traffic fluctuation patterns. It performs abnormal outbreak detection on window statistics results, accurately identifies fault scenarios such as short-term abnormal surges, high-frequency continuous errors, and progressively deteriorating faults, filters out occasional harmless single anomalies, and finally outputs stable, accurate, and low-false-reporting graded alarm results.

[0054] Furthermore, prior to step S03 above, which involves supplementing the classification of the graded alarm results using a pre-built knowledge distillation classification model to obtain refined graded results, the method further includes: Step S0301: Distill the lightweight code model using the pre-trained code model, and set independent classification exits after several Transformer encoding layers of the lightweight code model to obtain the pre-trained distillation model. Step S0302: Train the pre-trained distillation model using log training samples to obtain the initial distillation model; Step S0303: Perform weight quantization processing on the initial distillation model to obtain the knowledge distillation classification model.

[0055] The model training in this embodiment uses the following... Figure 2 The teacher-student model architecture shown here, before online intelligent inference, involves the system completing the entire process of building and optimizing a lightweight knowledge distillation model adapted to distributed log anomaly classification scenarios.

[0056] During the cold start phase of the model, a massive amount of historical real business anomaly logs are first collected to complete structured parsing and anomaly feature extraction. Based on general large language models such as Tongyi Qianwen, exclusive prompt words are designed for semi-automatic pre-classification, automatically extracting the root cause of log errors, anomaly type and severity level. Then, a small amount of manual review is used to correct classification biases and quickly build a three-level labeled dataset of FATAL / ERROR / WARN, greatly reducing the cost of pure manual labeling.

[0057] To address the issue of imbalanced log samples, data augmentation methods such as synonym replacement and back-translation rewriting are used to expand the minority anomalous samples. At the same time, the SMOTE oversampling algorithm is used to balance and optimize the FATAL high-risk minority class samples to ensure a balanced distribution of the dataset.

[0058] The model training adopts the classic teacher-student distillation architecture. The general CodeBERT-Base code pre-trained model is selected as the teacher model, and its powerful code log semantic understanding ability is used to build an upper bound on classification accuracy. The lightweight CodeBERTa-small model with fewer parameters and higher inference efficiency is selected as the student model, which inherits the feature learning and classification capabilities of the teacher model.

[0059] A multi-exit dynamic reasoning mechanism is introduced, with independent classification exits added to the back end of multiple Transformer encoding layers in the student model. This allows the model to exit early once the shallow reasoning reaches the target, significantly reducing the amount of forward computation.

[0060] The training loss function adopts a multi-dimensional weighted composite loss scheme, which integrates adaptive cross-entropy loss with class weights, temperature softening KL divergence distillation loss, multi-layer outlet differential loss, hidden layer feature mean square error matching loss and attention matrix matching loss, to comprehensively constrain the student model to fit the feature distribution of the teacher model.

[0061] After completing the distillation pre-training, the model is fine-tuned using the constructed log annotation dataset to fit the abnormal distribution of real business logs, resulting in an initial distillation model with complete classification capabilities. Subsequently, the model is finally optimized through fine-grained quantization and compression to obtain a knowledge distillation classification model that can be deployed in real time.

[0062] Furthermore, step S0303 above, which involves weight quantization of the initial distillation model to obtain the knowledge distillation classification model, includes the following steps: Step S03031: Quantize the linear layer weights in the initial distillation model to obtain the quantized linear layer weights; Step S03032: Determine the quantization scaling factor based on the numerical range of the linear layer weights; Step S03033: The quantized linear layer weights are calibrated using the quantization scaling factor to obtain the knowledge distillation classification model.

[0063] To address the engineering challenges of large initial distillation model parameters, high inference latency in general CPU environments, and inability to adapt to real-time streaming processing of massive logs, this embodiment adopts a refined and differentiated INT8 weight quantization strategy to achieve inference acceleration and model lightweighting while preserving the model's classification accuracy to the maximum extent.

[0064] The quantization process employs a selective quantization mechanism, which excludes network layers such as embedding layers and normalization layers that are highly sensitive to feature accuracy from the quantization exclusion list. This avoids semantic feature shifts and a decrease in classification accuracy caused by quantization. Weight quantization compression is performed only on linear layers that bear the core matrix operations and consume the most computing power.

[0065] The floating-point weight parameters of all linear layers in the initial distillation model are uniformly compressed to reduce their precision. The high-precision floating-point parameters are quantized into low-precision integer parameters, resulting in quantized linear layer weight data, which significantly reduces the model's storage volume and computational overhead.

[0066] The system comprehensively calculates the numerical distribution range, maximum and minimum values ​​of all quantized weight parameters. Combined with the accuracy tolerance threshold of the log classification task, it adaptively solves the quantization scaling factor to suit the current model, which is used to compensate for accuracy loss and parameter offset errors during the quantization process.

[0067] The quantization scaling factor obtained by the solution is used to perform point-by-point calibration, correction and restoration optimization of all quantized linear layer weight parameters to eliminate quantization distortion and ensure that the model's semantic feature extraction capability and classification accuracy are basically unaffected.

[0068] After quantization, combined with adaptive parallel inference configuration based on the number of CPU cores, full graph optimization is enabled and cached model files are generated, ultimately resulting in a lightweight knowledge distillation classification model that is highly accurate, fast inference, and adaptable to streaming deployment on general-purpose CPUs.

[0069] This embodiment, through the aforementioned scheme, specifically intercepts the raw logs of the distributed system using an interception component to obtain intercepted logs. These intercepted logs are then downgraded according to an invalid error feature library to obtain structured logs. An expert rule engine then classifies these structured logs to obtain graded alarm results. A pre-built knowledge distillation classification model further classifies these graded alarm results to obtain refined classification results. Finally, the final fault result is confirmed based on these graded alarm results and refined classification results. Thus, by first downgrading the intercepted logs according to the invalid error feature library to filter out expected error noise at the source and suppress alarm proliferation, and then using the expert rule engine to classify the structured logs, the severity is differentiated by rules to alleviate rule rigidity. For coverage blind spots that are difficult to exhaustively cover with rules, a lightweight knowledge distillation model is used to provide supplementary classification of rule results while ensuring real-time CPU inference, resulting in refined classification results. Finally, the rule and model results are merged to confirm the fault. This solves the problems of rampant log-based graded alarms, rigid and incomplete rule coverage, and difficulty in real-time model inference processing in existing systems, thereby improving the efficiency of fault confirmation in distributed systems.

[0070] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 In step S03, where the graded alarm results are supplemented and classified using a pre-built knowledge distillation classification model to obtain refined graded results, the distributed system fault confirmation method further includes steps S031-S034: Step S031: The hierarchical alarm results are reasoned layer by layer through the knowledge distillation classification model to obtain the initial hierarchical results and result confidence of the classification output. Step S032: Extract the target classification results whose confidence level exceeds the confidence threshold from the initial classification results; Step S033: Extract the set of samples to be labeled from the initial classification results whose confidence level is lower than the confidence threshold; Step S034: Generate refined classification results based on the target classification results.

[0071] The optimized and deployed lightweight multi-exit knowledge distillation classification model is connected to the online real-time inference link to perform full secondary fine-grained supplementary classification on the hierarchical alarm results output by the expert rule engine.

[0072] The model relies on multiple independent classification exits to achieve progressive inference. After the shallow Transformer encoding layer completes its inference, it immediately outputs the initial classification result for the corresponding exit, along with the corresponding confidence score, to characterize the reliability of the current classification result. Deeper exits can perform secondary corrections and reinforcements on the shallow results, achieving multi-level accuracy verification. The system presets a fixed confidence threshold and performs tiered screening and identification on the initial classification results output from all exits. Classification results with confidence scores higher than the threshold are judged as valid and reliable results and collected into the target classification result set, serving as the final intelligent classification basis for rule blind spots and effectively compensating for novel, combined, and boundary anomalies that cannot be identified by manual rules.

[0073] At the same time, the system automatically collects all low-confidence inference samples with confidence levels below the threshold. Through a dual screening strategy of uncertainty sampling and cluster diversity sampling, it selects high-value log samples with large feature differences, strong representativeness, and iterative value, and summarizes them to build a sample set to be labeled. These samples are mostly new anomalies generated by business iteration and niche anomalies not covered by rules, which are the core data source for the continuous evolution of the model.

[0074] Finally, based on the high-confidence target classification results after screening, the fine classification of all blind zone anomalies is completed, forming a model fine classification result with high accuracy and high coverage, which complements the expert rule classification result.

[0075] Specifically, step S03 above, which involves confirming the final fault result based on the graded alarm results and the refined graded results, includes: Step S035: Merge and align the graded alarm results and the refined graded results according to the preset aggregation dimension to obtain the fused graded results; Step S036: Train the knowledge distillation classification model using the sample set to be labeled to obtain an updated knowledge distillation classification model; Step S037: The graded alarm results are reclassified using the updated knowledge distillation classification model to obtain updated refined graded results; Step S038: Generate the final fault result based on the fusion classification result and the updated refined classification result.

[0076] To integrate the strong interpretability of expert rules with the high coverage of intelligent models and construct a continuously iterative fault identification closed loop, the system performs multi-dimensional fusion and iterative optimization of rule-based hierarchical alarm results and model-based refined hierarchical results.

[0077] Based on preset aggregation dimensions such as service name, interface name, time of occurrence of an anomaly, anomaly type, and error chain, the two types of classification results are precisely aligned, data is deduplicated, and results are complementary and integrated. This unifies the anomaly level judgment criteria, corrects individual scenarios of rule misjudgment and model misjudgment, and obtains a fused classification result that takes into account both human experience and intelligent semantic analysis, ensuring the interpretability and accuracy of the classification results of common known anomalies.

[0078] The system uniformly sorts out the low-confidence, high-value unlabeled sample set collected during the reasoning process. After a small amount of manual label correction, it is used as a brand-new training sample and fed into the knowledge distillation classification model to carry out incremental iterative training. The model weight parameters and feature extraction logic are continuously updated so that the model can gradually adapt to new abnormal scenarios and make up for the shortcomings in recognition, resulting in a brand-new knowledge distillation classification model after iterative updates.

[0079] The original graded alarm results are reclassified and inferred again using the updated and evolved model, and the updated refined graded results are output, which are adapted to the latest business anomaly scenarios and have further improved accuracy.

[0080] Finally, the initial fusion and classification results are integrated with the refined classification results after iterative optimization. Multi-dimensional cross-validation, fault level review, and impact scope determination are performed to output accurate, complete, and traceable final fault results of the distributed system, which can be directly used for operation and maintenance fault location, root cause analysis, and alarm push.

[0081] This embodiment, through the above-described scheme, specifically uses the knowledge distillation classification model to perform layer-by-layer reasoning on the graded alarm results, obtaining the initial graded results and result confidence levels of the classification output; extracting target graded results from the initial graded results whose result confidence levels exceed the confidence threshold; extracting a set of unlabeled samples from the initial graded results whose result confidence levels are lower than the confidence threshold; and generating refined graded results based on the target graded results. Thus, firstly, the intercepted logs are downgraded according to the invalid error feature library, filtering out expected error noise from the source to suppress alarm proliferation. Then, the structured logs are graded by an expert rule engine, using rules to differentiate severity and alleviate rule rigidity. For coverage blind spots that are difficult to exhaustively cover with rules, a lightweight knowledge distillation model is used to take over the teacher's capabilities, supplementing the classification of rule results while ensuring real-time CPU reasoning, obtaining refined graded results. Finally, the rule and model results are integrated to confirm the fault, solving the problems of rampant log graded alarms, rigid and incomplete rule coverage, and difficulty in real-time processing of model reasoning, thereby improving the efficiency of fault confirmation in distributed systems.

[0082] For example, to help understand the implementation flow of the distributed system fault confirmation method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 4 , Figure 4 A simplified flowchart of a method for fault confirmation in a distributed system is provided, specifically: This embodiment constructs an end-to-end, self-iterable, and fully covered distributed system log anomaly classification and rapid fault discovery closed-loop system, fully realizing the full-process capabilities of high signal-to-noise ratio collection, accurate classification, real-time inference, and intelligent iteration.

[0083] During the overall operation of the system, the system first relies on the non-intrusive AOP log interception component to complete the collection of all business logs. Through the hot-update invalid error feature library, the rampant invalid ERROR logs are uniformly downgraded and denoised, and standardized structured logs are uniformly output. This completely solves the industry pain points of log level abuse, alarm overload, and real faults being drowned out by noise from the source, and significantly improves the signal-to-noise ratio of log data.

[0084] Subsequently, based on the Flink streaming computing framework, structured log data is consumed. A configurable and gray-scale hot-update expert rule engine completes the three-level interpretable anomaly classification of FATAL, ERROR, and WARN. Relying on the sliding window aggregation statistics and adaptive dynamic threshold mechanism at the service + interface dimensions, false alarms due to single log jitter are suppressed. This achieves low-latency, high-reliability, and interpretable fallback classification of known common anomalies, and outputs stable rule-based classification alarm results.

[0085] For novel unknown anomalies, combined superposition anomalies, and boundary scene anomalies that cannot be covered by manual rules, the system calls a lightweight model that has undergone multi-exit knowledge distillation and refined quantification optimization to carry out real-time streaming intelligent supplementation and classification using CPU. Through layer-by-layer reasoning and confidence filtering, it outputs refined classification results to make up for the blind spots in rule coverage.

[0086] Meanwhile, the system automatically collects high-value samples with low confidence, and selects high-quality samples through joint sampling of uncertainty and diversity, continuously driving incremental iterative evolution of the model and constantly improving the ability to identify new anomalies.

[0087] The final system integrates the interpretable results of rules with the high coverage results of the model to generate accurate final fault judgment results, forming a fully automatic closed-loop operation mechanism of "log noise reduction and purification - expert rule bottom-up classification - distillation model intelligent blind filling - sample feedback iterative optimization - accurate fault output". It takes into account the interpretability and low latency advantages of rules and the high coverage and high semantic recognition advantages of deep learning models, and completely solves the defects of traditional solutions such as alarm proliferation, rigid rules, incomplete coverage, and large model inference lag that cannot be implemented in real time. It comprehensively improves the real-time performance, accuracy and completeness of fault detection in distributed systems.

[0088] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the distributed system fault confirmation method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0089] This application also provides a distributed system fault confirmation device, please refer to... Figure 5 The distributed system fault confirmation device includes: The interception module 10 is used to intercept the raw logs of the distributed system through the interception component to obtain intercepted logs, and to downgrade the intercepted logs according to the invalid error feature library to obtain structured logs. The grading module 20 is used to grade the structured logs through an expert rule engine to obtain graded alarm results; The confirmation module 30 is used to supplement the classification of the graded alarm results by a pre-built knowledge distillation classification model to obtain refined classification results, and to confirm the final fault result based on the graded alarm results and the refined classification results.

[0090] The distributed system fault confirmation device provided in this application, employing the distributed system fault confirmation method in the above embodiments, can solve the technical problems of rampant log hierarchical alarms, rigid and incomplete rule coverage, and difficulty in real-time processing of model inference. Compared with the prior art, the beneficial effects of the distributed system fault confirmation device provided in this application are the same as those of the distributed system fault confirmation method provided in the above embodiments, and other technical features in the distributed system fault confirmation device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0091] This application provides a distributed system fault confirmation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the distributed system fault confirmation method in Embodiment 1 above.

[0092] The following is for reference. Figure 6 The diagram illustrates a structural schematic suitable for implementing a distributed system fault confirmation device according to embodiments of this application. The distributed system fault confirmation device in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The distributed system fault confirmation device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0093] like Figure 6As shown, the distributed system fault confirmation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the distributed system fault confirmation device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the distributed system fault diagnosis device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a distributed system fault diagnosis device with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0094] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0095] The distributed system fault confirmation device provided in this application, employing the distributed system fault confirmation method in the above embodiments, can solve the technical problems of rampant log hierarchical alarms, rigid and incomplete rule coverage, and difficulty in real-time processing of model inference. Compared with the prior art, the beneficial effects of the distributed system fault confirmation device provided in this application are the same as those of the distributed system fault confirmation method provided in the above embodiments, and other technical features in this distributed system fault confirmation device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0096] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0097] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0098] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the distributed system fault confirmation method in the above embodiments.

[0099] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0100] The aforementioned computer-readable storage medium may be included in the distributed system fault confirmation device; or it may exist independently and not be assembled into the distributed system fault confirmation device.

[0101] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the distributed system fault confirmation device, the distributed system fault confirmation device: intercepts the original logs of the distributed system through an interception component to obtain intercepted logs, and downgrades the intercepted logs according to an invalid error feature library to obtain structured logs; classifies the structured logs through an expert rule engine to obtain graded alarm results; supplements the graded alarm results with a pre-built knowledge distillation classification model to obtain refined graded results, and confirms the final fault result based on the graded alarm results and the refined graded results.

[0102] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0104] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0105] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described distributed system fault confirmation method. This solves the technical problems of rampant log-based hierarchical alarms, rigid and incomplete rule coverage, and difficulty in real-time model inference processing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the distributed system fault confirmation method provided in the above embodiments, and will not be elaborated upon here.

[0106] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the distributed system fault confirmation method described above.

[0107] The computer program product provided in this application can solve the technical problems of rampant log-based hierarchical alarms, rigid and incomplete rule coverage, and difficulty in real-time processing of model inference. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the distributed system fault confirmation method provided in the above embodiments, and will not be repeated here.

[0108] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for fault confirmation in a distributed system, characterized in that, The distributed system fault confirmation method includes: The raw logs of the distributed system are intercepted by the interception component to obtain intercepted logs, and the intercepted logs are downgraded according to the invalid error feature library to obtain structured logs. The structured logs are classified using an expert rule engine to obtain classified alert results. The graded alarm results are supplemented by a pre-built knowledge distillation classification model to obtain refined graded results, and the final fault result is confirmed based on the graded alarm results and refined graded results.

2. The distributed system fault confirmation method as described in claim 1, characterized in that, The steps of intercepting the raw logs of the distributed system using an interception component to obtain the intercepted logs include: The raw log data is obtained by intercepting the raw logs output by the business nodes of the distributed system using an interception component. The original log data is filtered out to obtain candidate log data, which has a log level of not less than the error level. Extract the timestamp, service name, interface name, and log level of the candidate log data, and obtain the context information corresponding to the timestamp; Interception logs are generated based on the context information, timestamp, service name, interface name, and log level.

3. The distributed system fault confirmation method as described in claim 1, characterized in that, The step of classifying the structured logs using an expert rule engine to obtain classified alert results includes: Based on the exception type, keyword features, and stack pattern in the structured log, the structured log is classified into levels by the expert rule engine to obtain hierarchical logs; Based on the service name and interface name in the structured log, window aggregation statistics are performed on the hierarchical log to obtain window statistics results; Alarm detection is performed on the window statistics results using dynamic thresholds to obtain tiered alarm results.

4. The distributed system fault confirmation method as described in claim 1, characterized in that, Before the step of supplementing the classification of the graded alarm results using a pre-built knowledge distillation classification model to obtain refined classification results, the method further includes: A lightweight code model is trained by distillation using a pre-trained code model, and independent classification exits are set after several Transformer encoding layers of the lightweight code model to obtain a pre-trained distillation model. The pre-trained distillation model is trained using log training samples to obtain the initial distillation model; The initial distillation model is weighted and quantized to obtain a knowledge distillation classification model.

5. The distributed system fault confirmation method as described in claim 4, characterized in that, The step of performing weight quantization on the initial distillation model to obtain the knowledge distillation classification model includes: The linear layer weights in the initial distillation model are quantized to obtain the quantized linear layer weights. The quantization scaling factor is determined based on the numerical range of the linear layer weights; The knowledge distillation classification model is obtained by calibrating the quantized linear layer weights using the quantization scaling factor.

6. The distributed system fault confirmation method as described in claim 4, characterized in that, The step of supplementing the classification of the graded alarm results with a pre-built knowledge distillation classification model to obtain refined classification results includes: The knowledge distillation classification model is used to reason layer by layer about the graded alarm results to obtain the initial graded results and the confidence level of the results output by the classification exit. Extract the target classification results from the initial classification results whose confidence level exceeds the confidence threshold; Extract the set of unlabeled samples from the initial classification results whose confidence level is lower than the confidence threshold; A refined classification result is generated based on the target classification result.

7. The distributed system fault confirmation method as described in claim 6, characterized in that, The step of confirming the final fault result based on the graded alarm results and the refined graded results includes: The hierarchical alarm results and the refined hierarchical results are merged and aligned according to a preset aggregation dimension to obtain a fused hierarchical result; The knowledge distillation classification model is trained using the unlabeled sample set to obtain an updated knowledge distillation classification model; The updated knowledge distillation classification model is used to reclassify the graded alarm results to obtain updated refined graded results. The final fault result is generated based on the fusion classification result and the updated refined classification result.

8. A distributed system fault confirmation device, characterized in that, The distributed system fault confirmation device includes: The interception module is used to intercept the raw logs of the distributed system through the interception component to obtain intercepted logs, and to downgrade the intercepted logs according to the invalid error feature library to obtain structured logs; The grading module is used to grade the structured logs through an expert rule engine to obtain grading alarm results; The confirmation module is used to supplement the classification of the graded alarm results by a pre-built knowledge distillation classification model to obtain refined classification results, and to confirm the final fault result based on the graded alarm results and the refined classification results.

9. A distributed system fault confirmation device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the distributed system fault confirmation method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the distributed system fault confirmation method as described in any one of claims 1 to 7.

Images