Composite network fault classification model based on data intelligence, training method and fault classification method

Abstract

The application discloses a composite network fault classification model based on data intelligence, a training method and a fault classification method, belongs to the field of network fault classification, is used for solving the network fault classification problem, and has the technical points that the embedding vectors corresponding to the same fault category of the fault categories belonging to the minority class are subjected to oversampling operation by a first CVAE model to obtain the synthetic embedding vectors of the fault categories; the embedding vectors corresponding to the same fault category of the fault categories belonging to the majority class are subjected to downsampling operation by the first CVAE model to obtain the representative embedding vectors of the fault categories; a fifth training set is constructed according to the synthetic embedding vectors corresponding to the fault categories belonging to the minority class and the representative embedding vectors corresponding to the fault categories belonging to the majority class; a third BERT model and a classifier adapted to the third BERT model are fine-tuned by using the fifth training set to obtain a fourth BERT model; and the fourth BERT model and the classifier adapted to the fourth BERT model are the network fault classification model.

Description

Technical Field

[0001] This invention belongs to the field of network fault classification, and relates to a composite network fault classification model based on data intelligence, a training method, and a fault classification method. Background Technology

[0002] One of the core challenges of network operations and maintenance (O&M) lies in accurately identifying risks and faults from massive amounts of network device logs—this directly determines the system's fault prevention capabilities and response speed. Network devices generate a large number of logs daily, but the vast majority of these are negligible redundant information; core alarm logs that truly represent serious risks account for only 3%-5%. Adding to the complexity, individual log entries are often limited in information, and related logs representing the same event may not be discontinuously distributed in time series. Therefore, it is necessary to rely on O&M experience to actively retrieve relevant information for comprehensive analysis and accurate judgment.

[0003] For example, when an EIGRP neighbor interruption alarm (%EIGRP-5-NBRCHANGE:EIGRP-IPv4:(1)Neighbor10.0.0.2(GigabitEthernet0 / 3)isdown) appears on the device, this log alone cannot determine the type of fault (it may be a temporary network jitter or configuration problem). Only by combining the port link state change alarm (%LINK-3-UPDOWN:InterfaceGigabitEthernet0 / 3,changedstatetodown) and packet loss threshold exceedance alarm (%LINEPROTO-4-PLTHRESH:PacketlossthresholdexceededonGi0 / 3) logs from the same device can it be determined that the routing protocol interruption is caused by a hardware failure, and then the port hardware or cables can be checked to repair the fault. This type of multi-layered superimposed fault that requires cross-log correlation analysis is called a "complex network fault".

[0004] In the early stages, engineers commonly used methods such as keyword filtering (relying on fixed rules and prone to missed detections), threshold triggering (static thresholds are difficult to adapt to dynamic networks), and manual sampling (randomness leads to the loss of key information) to screen logs, and further relied on expert knowledge for fault diagnosis. These methods suffer from high labor costs, poor dynamic adaptability, and high missed detection rates. With the development of computer science and artificial intelligence, data-driven fault diagnosis technology has reduced reliance on manual labor and improved the efficiency of fault diagnosis, making it a hot topic in the field of fault diagnosis and achieving successful applications in many areas.

[0005] While unsupervised learning (anomaly detection) methods do not rely on labeled datasets, they can only determine whether an anomaly is present or not. Furthermore, network system behavior patterns evolve with business updates and configuration changes, causing pre-trained "normal patterns" to become invalid, resulting in numerous false positives. Supervised learning methods are limited by the lack of labeled data; manually labeling complex network faults requires significant human resources, and many types of complex faults are extremely rare, making it difficult for models to learn their pattern characteristics. Summary of the Invention

[0006] To address the problem of network fault classification and the low accuracy of classification models due to the extreme imbalance in the number of fault type samples, in a first aspect, a network fault classification model training method according to some embodiments of this application includes...

[0007] The first BERT model is pre-trained using the training set to obtain the second BERT model. The pre-training tasks include a log-description contrast learning task and a word mask prediction task for fault-related log groups. The training set includes the second training set, which includes samples from fault-related log groups. The logs in the fault-related log groups represent the same fault type based on causal relationships, and the logs are arranged in chronological order.

[0008] The third BERT model is obtained by fine-tuning the second BERT model and its adapted classifier using the third training set. The third training set includes fault-related log groups and their corresponding fault types from the second training set. Different fault-related log groups may have labels of the same fault category or labels of different fault categories.

[0009] The third BERT model generates embedding vectors corresponding to the samples of the fault-related log groups in the third training set. These vectors are then combined with the labels of the corresponding fault categories to construct the fourth training set. The fourth training set is used to train the CVAE model to obtain the first CVAE model.

[0010] Based on the labels of the fault categories corresponding to each embedding vector in the fourth training set, the number of samples corresponding to the embedding vectors of different fault categories is obtained, and the fault category classification includes minority and majority classes based on the number of samples.

[0011] The embedding vectors corresponding to the same fault category belonging to the minority fault category are obtained by performing an oversampling operation on the first CVAE model to obtain the synthetic embedding vectors of the fault category, thus obtaining the synthetic embedding vectors corresponding to the fault categories belonging to the minority fault category.

[0012] The embedding vectors corresponding to the same fault category belonging to the majority fault category are obtained by performing downsampling operation on the first CVAE model to obtain the representative embedding vectors of the fault category, thus obtaining the representative embedding vectors corresponding to the fault category belonging to the majority fault category.

[0013] A fifth training set is constructed based on the synthetic embedding vectors corresponding to the minority class of faults and the representative embedding vectors corresponding to the majority class of faults. The fifth training set is used to fine-tune the third BERT model and its adapted classifier to obtain the fourth BERT model. The fourth BERT model and its adapted classifier are the network fault classification model.

[0014] In a second aspect, according to the network fault classification method in some embodiments of this application, the network fault classification model is based on a network fault classification model, which includes a fourth BERT model and its adapted classifier, wherein the classifier includes a global average pooling layer, a fully connected layer and a probability transformation layer.

[0015] Network fault classification methods include the following steps:

[0016] The fourth BERT model converts the input data into sequence embedding vectors, where the input data includes fault-related log groups;

[0017] The global average pooling layer compresses the sequence embedding vector into a first feature vector;

[0018] The fully connected layer performs a linear transformation on the first feature vector to obtain the second feature vector;

[0019] The probability transformation layer maps the second feature vector output by the fully connected layer to the probability distribution of the labels for each fault category through the Sigmoid function;

[0020] The network fault classification model is trained based on the training method described in claim 1.

[0021] In a third aspect, a network fault classification apparatus according to some embodiments of this application includes a network fault classification model, which includes a fourth BERT model and an adapted classifier, wherein the classifier includes a global average pooling layer, a fully connected layer and a probability transformation layer.

[0022] The fourth BERT model converts the input data into sequence embedding vectors, where the input data includes fault-related log groups;

[0023] The global average pooling layer compresses the sequence embedding vector into the first feature vector;

[0024] The fully connected layer performs a linear transformation on the first feature vector to obtain the second feature vector.

[0025] The probability transformation layer maps the second feature vector output by the fully connected layer to the probability distribution of the labels for each fault category through the Sigmoid function;

[0026] The network fault classification model is trained based on the training method described in claim 1.

[0027] According to the above-described scheme in some embodiments of this application, fault-related log groups of network logs are obtained, including...

[0028] Log templates are obtained from network logs, and frequency time series of each log template for the device are constructed.

[0029] Construct a fully connected undirected graph of log templates;

[0030] Based on the frequency time series, causal relationship tests are performed on the edges in the fully connected undirected graph, and pseudo-causal edges in the fully connected undirected graph are removed to obtain several connected subgraphs.

[0031] The log templates corresponding to all nodes in the connected subgraph are arranged in chronological order to obtain the corresponding fault-related log groups.

[0032] According to the above scheme in some embodiments of this application, the training set further includes a first training set, wherein the first training set includes log and description data.

[0033] Based on the above-described scheme in some embodiments of this application, the number of samples of the embedded vectors generated corresponding to the fault-related log groups of each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples corresponds to a fault category. The fault categories are classified as minority classes; where, if a fault category is classified as a minority class, the embedding vector corresponding to that fault category is the original embedding vector, and the number of samples is the original sample number; the synthetic embedding vector is obtained based on the following method: a latent variable is sampled from a standard normal distribution, the fault category label vector is used as a condition of the CVAE model, and the decoder of the CVAE model generates the synthetic embedding vector of the latent variable. This step is repeated until the sum of the number of samples in the synthetic embedding vector and the number of samples in the original sample is equal to the original sample number. .

[0034] Based on the above-described scheme in some embodiments of this application, the number of samples of the embedded vectors generated corresponding to the fault-related log groups of each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples corresponds to a fault category. The fault categories are classified into majority classes; where, if a fault category is classified into majority classes, the embedding vector corresponding to the fault category is the original embedding vector; the representative embedding vector of the fault category is obtained based on the following method: according to the original embedding vector and the fault category label vector, a latent variable corresponding to the original embedding vector is generated by the encoder of the CVAE model, and the latent variables of all embedding vectors corresponding to the fault category are obtained; latent points are sampled in the convex hull region of the category latent space constructed by all latent variables, and the nearest neighbor latent variable is found for each sampled point; the embedding vector corresponding to the nearest neighbor latent variable is used as the representative embedding vector.

[0035] According to the above-described scheme in some embodiments of this application, the fault category classification also includes a balanced category;

[0036] The number of samples of the generated embedding vectors corresponding to the fault-related log groups for each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples of the embedding vector corresponding to a fault category falls within the interval [0, 1]. The fault category is classified as balanced.

[0037] Specifically, a fifth training set is constructed based on the embedding vectors corresponding to the fault categories of the balanced class, the synthetic embedding vectors and their corresponding original embedding vectors corresponding to the fault categories of the minority class, and the representative embedding vectors corresponding to the fault categories of the majority class. The fifth training set is used to fine-tune the third BERT model and its adapted classifier to obtain the fourth BERT model. The fourth BERT model and its adapted classifier are the network fault classification model.

[0038] According to the above-described scheme in some embodiments of this application, the training further includes:

[0039] In the model used for network fault classification, the model obtains the confidence level of the fault category of a fault-related log group based on a fault-related log group in the input dataset that includes fault-related log groups.

[0040] Based on the confidence level of the fault category and the confidence level threshold, output the fault type with a confidence level higher than the confidence level threshold. If the highest confidence level among the fault categories is lower than the confidence level threshold, then a fault-related log group belongs to an unknown type of fault.

[0041] For fault-related log groups corresponding to unknown types of faults, fault type labels are obtained by labeling the fault type.

[0042] The fault-related log group and the fault type label are placed in the third training set to obtain an updated third training set, which is then used for model training.

[0043] According to the above scheme in some embodiments of this application, the log and description contrast learning task and the fault-related log group word mask prediction task are jointly trained, wherein the model parameters are jointly optimized by MLM loss and LDA loss during training until they converge to the optimal parameters.

[0044] In the log-description contrastive learning task, BERT's input includes logs and descriptions. BERT encodes logs and descriptions into hidden representations. It then performs contrastive learning on positive pairs of logs and descriptions and negative pairs of logs and descriptions in the hidden representations. The optimization objectives include minimizing the positive sample distance and maximizing the negative sample distance.

[0045] In the word masking prediction task for fault-related log groups, the input of BERT includes fault-related log groups, and a certain proportion of logs in each fault-related log group are randomly masked. BERT encodes the fault-related log groups into hidden representations for predicting the mask, where the optimization objective includes cross-entropy loss to predict the masked logs.

[0046] Beneficial Effects: Existing network device log fault classification faces two main problems. First, insufficient labeled data arises because labeling each fault-related log group with its fault type is extremely time-consuming and labor-intensive. Second, the labeled data suffers from imbalanced fault types due to significant differences in the probability of different faults, leading to an imbalanced dataset for both the intended and labeled data sets. This invention addresses these two problems through two methods. For insufficient labeled data, a pre-training-fine-tuning paradigm is employed. The pre-training tasks include log-description comparison learning and fault-related log group word mask prediction. This utilizes readily available log-related corpora, including log-description pairs and a large number of unlabeled fault-related log groups. The descriptions refer to the log descriptions themselves. This method effectively improves the log semantic embedding capability of the BERT model, further enhancing its ability to be used in downstream fault classification tasks. For the imbalanced data problem, a generative model is used, employing downsampling and oversampling to reduce the ratio of minority class samples to majority class samples. However, CVAE is only used for sample generation. Generating too many samples leads to distortion, while generating too few fails to address the aforementioned problems. Using other models for downsampling increases system complexity. CVAE can only generate vectors, while the original data samples are text. Therefore, this invention designs CVAE for both oversampling and downsampling. Oversampling trains CVAE to generate samples using specific labels. Downsampling further encodes the embedding vectors corresponding to the samples into latent vectors using CVAE, and then performs uniform sampling within these latent vectors to ensure that downsampling reduces the number of samples without losing fault type pattern information. This invention also designs a complex pre-training-fine-tuning-fine-tuning training paradigm, allowing CVAE to be trained and resampled under the data modality of the embedding vectors of samples generated by the BERT model, followed by secondary fine-tuning. This approach solves the problem of data modality mismatch. Finally, in practical applications, this invention also incorporates a continuous optimization method: identifying fault-related log groups for unknown fault types, labeling and recombining the dataset, and retraining to optimize the model. Attached Figure Description

[0047] Figure 1 This is a flowchart of the training process for the fault classification model.

[0048] Figure 2 This is the flowchart for the first round of fine-tuning training of the fault classification model.

[0049] Figure 3 This is a flowchart for the continuous optimization of the fault classification model.

[0050] Figure 4 This is a comparison chart of the original sample distribution and the resampled target distribution in the experimental example. Detailed Implementation

[0051] The existing network device log fault classification involves two main problems. First, there is insufficient labeled data, as labeling each fault-related log group with its fault type individually is extremely time-consuming and labor-intensive. Second, the labeled data suffers from imbalanced fault types, due to the significant differences in the probability of different faults occurring, resulting in an imbalanced dataset for both the intended and actual labeled data.

[0052] Based on the above reasons, this invention addresses the two problems mentioned above through two methods. For the problem of insufficient labeled data, this invention adopts a pre-training-fine-tuning paradigm. The pre-training tasks include a log-description comparison learning task and a word mask prediction task for fault-related log groups. It uses easily accessible log-related corpus, including log-description pairs and a large number of unlabeled fault-related log groups, where the description is a description of the log. This method effectively improves the log semantic embedding capability of the BERT model and further enhances its ability to be used for downstream fault classification tasks.

[0053] To address the data imbalance problem, this invention employs a generative model, using downsampling and oversampling to reduce the ratio of minority class samples to majority class samples. However, CVAE is only used for sample generation; generating too many samples leads to distortion, while generating too few fails to solve the aforementioned problem. Using other models for downsampling increases system complexity. CVAE can only perform vector generation tasks, while the original data samples are text. Therefore, this invention designs CVAE for both oversampling and downsampling. Oversampling trains CVAE to generate samples using specific labels. Downsampling further encodes the embedding vectors corresponding to the samples into latent vectors using CVAE, and then performs uniform sampling within these latent vectors to ensure that downsampling reduces the number of samples without losing fault type pattern information. This invention also designs a complex pre-training-fine-tuning-fine-tuning training paradigm, allowing CVAE to be trained and resampled under the data modality of the embedding vectors of samples generated by the BERT model, followed by secondary fine-tuning. This approach solves the data modality mismatch problem. Finally, in practical applications, this invention also includes a continuous optimization method, which involves identifying fault-related log groups of unknown fault types, labeling and recombining the dataset, and retraining to optimize the model.

[0054] As can be seen, this invention provides a new approach for scenarios with scarce labeled data through pre-trained language models. The two-stage training paradigm—utilizing massive and easily accessible domain corpora for self-supervised learning and then fine-tuning the training for specific tasks—effectively alleviates the dependence on labeled data. This characteristic is highly compatible with the data scarcity problem faced by complex network fault diagnosis. However, while it can solve the above problems, it also faces the challenge of implementing the language model approach due to the extreme imbalance in the number of samples for different fault types in network fault classification. Therefore, this invention, while considering the sample generation capability of generative models, proposes a method to balance the samples, thus solving the problem of low model accuracy caused by the extreme imbalance of fault category samples in network fault classification.

[0055] Specifically, this invention proposes a network fault classification model training method in log-based composite network fault diagnosis and classification. It employs a language model to model log sequences for fault classification. Addressing the challenge of scarce labeled data, a two-stage training paradigm of pre-training and fine-tuning is designed. In the fine-tuning stage, to alleviate the imbalance problem of fault class samples in network fault classification, a conditional variational autoencoder is used to generate synthetic samples for the minority class, while representative samples are selected for the majority class. This addresses the class imbalance problem in two dimensions, making it more suitable for extremely imbalanced distributions of fault class samples in network fault classification, thus improving the model's training stability and classification performance.

[0056] Based on this method, the present invention also proposes a fault classification device with a model and a fault classification method.

[0057] This invention also proposes a model continuous optimization method based on fault classification, which aims to maintain the high performance of the model at a controllable cost by dynamically evaluating model performance, integrating new fine-tuning data, and retraining the model.

[0058] However, directly training a language model for network fault type classification faces the challenge of an extremely imbalanced distribution of the number of fault log related groups corresponding to each fault type, such as... Figure 4 As shown. This invention considers the sample generation capability of generative models. However, on the one hand, the sample generation capability of generative models is mainly used to solve the problem of increasing the number of samples, and it is difficult to solve the balance problem alone. On the other hand, it introduces the CVAE model; however, the CVAE model cannot directly use the data type generated by logs, and can only process vector data. Therefore, the introduction of generative models is not effective.

[0059] The main focus of this invention is to determine how to format the dataset to address the aforementioned issues. Ultimately, the inventors discovered that the BERT model itself can be used as a classification model and can also convert log-generated data types into vector data. Based on this characteristic, this invention modifies the data structure of the dataset. First, the BERT model is used to convert the data into vector data, then the CVAE model is used to balance the dataset, and finally, the balanced dataset is used for BERT model training.

[0060] However, as can be seen from the above, how to transform the sample generation capability of the generative model into the balancing capability is another major problem that this invention addresses. This invention achieves sample quantity balancing in two dimensions by simultaneously performing oversampling and downsampling, selecting representative samples for fault types with a large number of samples, and synthesizing samples for fault types with a small number of samples, in order to solve the imbalance problem caused by extreme differences in the number of samples in the field of network fault classification.

[0061] However, while solving the sample balance problem improves the model's classification accuracy, the inventors discovered that the BERT model, after pre-training and fine-tuning, possesses the ability to understand log semantics and predict word masks for fault-related log groups. This, combined with balanced data, can further improve the model's classification accuracy. The inventors believe this is because the method used in this invention to solve data sample balance obviously increases the difficulty for the model to understand data in the network fault classification domain, especially when based on fault-related log groups as input data, further increasing the learning difficulty. In contrast, the log-description comparison learning task in this invention is used by the first BERT model to understand log semantics, and the word mask prediction task for fault-related log groups is used by the first BERT model to learn the contextual sequence dependencies of fault-related log groups. This solves the aforementioned problem caused by the data balance method, synergistically improving the model's classification accuracy, making it more suitable for network fault classification.

[0062] Another function of BERT model fine-tuning is to improve the accuracy of converting domain data into vector data. Since BERT is a language model with the ability to convert data into vectors, this capability can be fully utilized in this invention to avoid introducing additional models. This invention uses a third training set to train the CVAE model, enabling it to adapt to domain data and improve accuracy during sampling.

[0063] Based on the above, the network fault classification model training method disclosed herein includes the following steps:

[0064] Step S10. Use the training set to pre-train the first BERT model to obtain the second BERT model. The pre-training tasks include a comparative learning task of logs and corresponding descriptions and a word mask prediction task of fault-related log groups. The first training set includes paired data of logs and descriptions and fault-related log group data. The fault-related log group is formed by aggregating logs related to the same fault based on the causal relationship between logs. It includes logs of types such as system status, alarm information, performance indicators, configuration change records, routing / protocol events and hardware abnormal signals that characterize the fault, and the logs in the group are arranged in chronological order.

[0065] Engineers labeled each fault-related log group in the second training set with its corresponding fault category, thus obtaining the third training set. Different fault-related log groups may have the same fault category label or different fault category labels.

[0066] It's understandable that logs in fault-related log groups are aggregated based on causal relationships, essentially representing the same fault. The logs are arranged chronologically, and different fault-related log groups can represent the same fault type. This means that for a given dataset, a fault type can be represented by more than one fault-related log group, with each group considered a sample. This can lead to a fault type potentially corresponding to multiple fault-related log groups, resulting in a large number of samples; conversely, a fault type might correspond to very few fault-related log groups, resulting in a small number of samples. This is because different log combinations can point to the same fault type.

[0067] Understandably, the log-description comparison learning task of this invention is used by the first BERT model to understand log semantics, and the word mask prediction task for fault-related log groups is used by the first BERT model to learn the contextual sequence dependencies of fault-related log groups. Based on the above configuration of the training set, the first BERT model is pre-trained to obtain the second BERT model, enabling it to understand log semantics and predict the word mask of fault-related log groups, thus achieving the initial training of the BERT model.

[0068] In some specific embodiments of the present invention, the training set includes a first training set and a second training set, wherein the first training set includes log and descriptive data. The second training set is constructed based on the following method: parsing network logs to obtain log templates and statistically analyzing the frequency time series of each log template; constructing a fully connected undirected graph using log templates as nodes; based on the frequency time series, performing causal relationship checks on each pair of edges using a causal discovery algorithm, and removing pseudo-causal edges in the fully connected undirected graph to obtain several connected subgraphs; arranging the log templates corresponding to all nodes in each connected subgraph in chronological order to obtain the corresponding fault-related log groups.

[0069] In some specific examples, log parsing uses the Drain3 algorithm to transform raw logs into structured log templates, and filters the templates based on severity and reporting frequency, retaining those with high severity and low reporting frequency. For each network device, such as a router or switch, a corresponding frequency time series is constructed for the log template. This series records the frequency of occurrence of a single log template on a specific device within discrete, non-overlapping 60-second time intervals. In the causal discovery phase, the PC algorithm is used to identify potential causal relationships from the time series data. This algorithm first constructs a fully connected undirected graph containing all log template nodes, and then progressively prunes edges without causal relationships based on conditional independence tests. For any three log templates, the corresponding time series variables... , , Its conditional independence is defined as:

[0070]

[0071] The conditional independence test uses the G-squared test method (significance level 100-100%). This test is based on the cross-entropy principle of discrete data. Before the test, the frequency time series needs to be preprocessed using binary representation:

[0072]

[0073] G-squared statistic Conditional cross-entropy Export:

[0074]

[0075] in Let be the length of the time series. The calculation of conditional cross-entropy relies on empirical counting of the joint distribution of discrete states. Let be... This indicates that the data satisfies the state combination. , , The number of samples, then The statistic can be equivalently represented as:

[0076]

[0077] In the formula , , Each edge is counted for the corresponding state. The conditional independence hypothesis is accepted and the corresponding edge is deleted when the p-value is greater than 0.01.

[0078] The PC algorithm ultimately outputs several connected subgraphs, each representing a set of log templates with statistical causal relationships. The log templates corresponding to all nodes within each subgraph are then integrated in chronological order to form a fault-related log group.

[0079] In some specific embodiments of the present invention, the present invention jointly trains a log-description contrast learning task and a fault-related log group word masking prediction task. The model parameters are jointly optimized during training using MLM loss and LDA loss until convergence to the optimal parameters. In the log-description contrast learning task, the input to BERT includes logs and descriptions, and BERT encodes the logs and descriptions into hidden representations. The positive pairs and negative pairs of logs and descriptions in the hidden representations are contrasted and learned, with optimization objectives including minimizing the positive sample distance and maximizing the negative sample distance. In the fault-related log group word masking prediction task, the input to BERT includes fault-related log groups, and a certain proportion of words in each fault-related log group are randomly masked. BERT encodes the fault-related log groups into hidden representations for predicting the mask, with optimization objectives including using cross-entropy loss to predict the masked words.

[0080] In more specific examples, for the log-description contrast learning task in pre-training, logs often do not conform to natural language grammar and are highly concise, making it difficult to effectively capture their underlying semantic information. To enable the fault classification model to better understand logs, this invention utilizes the natural language description of the logs as a global knowledge enhancement model, hoping that the model can narrow the distance between the logs and their corresponding descriptions in the semantic embedding space, thereby improving the understanding of log semantics. To this end, this invention proposes the aforementioned contrastive learning-based description discrimination (LDA) pre-training task, the core of which is to enhance the model's understanding of logs by identifying the semantic similarity between logs and descriptions. Conceptually, the goal of contrastive learning is to narrow the distance between similar instances and widen the distance between different instances.

[0081] For the log-description comparison learning task in the pre-training task, let the original parameters of the BERT model be... Given a batch of log-description pairs ,in For logs, To correspond with the description, the logs and descriptions are first encoded into hidden representations: , For BERT's encoding function, The parameters to be optimized in the model in, These are the original BERT parameters, which remain fixed. Then, As a pair of semantically similar instances, and to hide other descriptions in the same batch ( ( ) as negative pairs, i.e., semantically distinct instance pairs. The loss function describing the discrimination task is:

[0082]

[0083] In the formula, log hiding means Description of hidden representation , Indicates hyperparameters, Indicates batch size, The cosine similarity hidden representation represents the hidden representation. Hidden representation of logs With the hidden representation of the description The cosine similarity; where, For positive alignment, hide the meaning. For negative pairs, among which, , Represents ordinal numbers; where, This represents the encoding function of BERT. Indicates the first A log. This represents the parameters to be optimized in the model. Indicates the first A description, This represents the original BERT parameters. Wherein, The empirical value can be set to 0.05, and the denominator of the formula is all descriptions within the batch, including the corresponding descriptions. and negative pairs description ( ) and log The sum of similarity indices.

[0084] In some preferred embodiments, this invention also compiles the equipment manufacturer's documentation, extracts log templates and corresponding descriptions, and adds the full names of abbreviations from the logs to the descriptions to enhance the model's understanding of abbreviations. Secondly, since the original materials are from publicly available documents, some descriptions may be inaccurate or incomplete. This invention uses GPT-4 to refine the descriptions, striving for accuracy and conciseness. Furthermore, randomly selected batches may contain logs with similar semantics; setting them as negative pairs would mislead the model. Therefore, this invention sets a threshold based on experience, replacing semantically similar logs when they are identified during batch extraction.

[0085] In some more specific examples, the word mask prediction task for fault-related log groups in the pre-training task is a word mask language modeling task, which enables the fault classification model to effectively capture the contextual sequence dependencies of fault-related log groups through self-supervised learning. Here, given a tokenized log sequence... in This represents the tokenized atomic unit. This invention first constructs a corrupted version through a random replacement operation. Random selection A small portion of the tokens (the quantity is) ,and Replace the tokens at these positions with the [MASK] symbol. The set of target tokens that are blocked is denoted as... ,in , This indicates the position of the masked token in the original sequence. The goal of training is to train on a given corrupted input. In the case of reconstructing the blocked target token set The corresponding optimization objective is:

[0086]

[0087] in, Indicates a given corrupted input At that time, the model reconstructs the joint probability of all masked tokens. Because The joint probability can be approximated by the independence assumption as the product of the conditional probabilities of each masked token, i.e., the summation form on the right side of the equation.

[0088] Furthermore, conditional probability Calculated using the output of the Transformer model:

[0089]

[0090] In the formula, Indicates damaged input After passing through the Transformer encoder (parameters are...) The sequence of hidden states after ) ,in correspond The Middle The hidden state of each token The location is blocked. The hidden state. This represents the token in the Transformer embedding layer. The output vector. This is a matrix transpose operation. This represents the summation of all tokens in the vocabulary. It's worth noting the log sequence. It is a fault-related log group assembled in chronological order. Therefore, the reconstruction process requires the model not only to capture the intra-sentence language patterns of the logs, i.e., the local dependencies between tokens, but also to learn the temporal dependencies across sentences, i.e. the logical associations between different log templates. This is crucial for understanding the contextual semantics of the logs.

[0091] The overall goal of the fault classification model pre-training in this invention is to avoid catastrophic forgetting, i.e., the model forgets the original pre-training knowledge during training on new tasks. This invention trains the model jointly on both tasks. To accelerate training convergence, the model parameters... From BERT raw parameters Initialization is performed, followed by joint optimization using MLM and LDA losses in each training step. After multiple epochs of training on two log corpora, the model eventually converges to the optimal parameters. .

[0092] Step S20. The engineer labels each fault-related log group in the second training set with its corresponding fault category to obtain the third training set. Different fault-related log groups may have the same fault category label or different fault category labels. The second BERT model is connected to the classification head to build a network fault classification model. The model is then fine-tuned using the third training set to obtain the first network fault classification model. The fine-tuned BERT model is the third BERT model.

[0093] Step S30. Generate embedding vectors for the samples of the fault-related log groups in the third training set using the third BERT model, construct a fourth training set with the corresponding fault category labels, and train the CVAE model using the fourth training set to obtain the first CVAE model.

[0094] It is understandable that this step involves using the BERT model to convert the data from the fault-related log group into a vector representation that can be used as input to the CVAE model.

[0095] In some specific instances, a CVAE consists of an encoder, a sampling network, and a decoder. The encoder takes the input... Mapping to latent variables variational distribution Output mean and variance Through reparameterization techniques ( )sampling The decoder is based on reconstruction Output conditional distribution Its training objective is to maximize the lower bound of evidence (ELBO), and the corresponding loss function is:

[0096]

[0097] The first term is the reconstruction loss, which minimizes the reconstruction error; the second term is the KL divergence, which makes the variational distribution approximate the prior. .when , When the KL divergence is , it can be simplified to:

[0098]

[0099] In the formula The latent space dimension. VAE uses reconstruction loss to make the output closer to the input, while simultaneously... Noise is introduced to augment the data. To control the generated data categories, CVAE introduces category labels in both the encoder and decoder. The objective function is expanded to a conditional ELBO:

[0100]

[0101] Although CVAE has two optimization objectives, in practice, the KL divergence is often excessively minimized to zero during training, leading to degraded reconstruction quality. To address this issue, this invention forces each dimension of the latent space to retain at least... Information content, i.e. At the same time, annealing weights are introduced. To prevent premature collapse of the KL divergence. Let the total number of training epochs be... The current epoch is Annealing weight Adjustment based on a phased linear strategy – 0~50% epoch ( )hour Only the reconstruction loss is optimized to learn the basic input-output mapping; 50%~100% of epochs ( )hour It grows linearly from 0 to 0.2 ( The weights of the KL divergence loss are gradually increased to strengthen the latent space constraints. The revised loss function integrates the above improvements and takes the following form:

[0102]

[0103] in To rebuild the losses, For staged linear annealing weights ( ), For the dimension of latent variables, For the first KL divergence of dimension, Minimum threshold for KL divergence ( ).

[0104] Step S40. Based on the labels of the fault categories corresponding to each sample in the fourth training set, count the number of samples in each fault category, and divide the fault categories into balanced, minority, and majority classes according to the number of samples. For fault categories belonging to the minority class, perform an oversampling operation using the first CVAE model to obtain the synthetic embedding vector of the label of the fault category, thus obtaining the synthetic embedding vector corresponding to the label of each fault category belonging to the minority class. For fault categories belonging to the majority class, perform a downsampling operation using the first CVAE model to obtain the representative embedding vector of the label of the fault category, thus obtaining the representative embedding vector corresponding to the label of each fault category belonging to the majority class. Construct a fifth training set based on the synthetic embedding vectors corresponding to the fault categories belonging to the minority class and the representative embedding vectors corresponding to the fault categories belonging to the majority class. Use the fifth training set to fine-tune the third BERT model and its adapted classifier to obtain a fourth BERT model. The fourth BERT model and its adapted classifier are the network fault classification model. During the training process, the parameters of the BERT model are frozen, and only the parameters of the classification head are fine-tuned.

[0105] In some specific instances, fault category classification includes balanced, minority, and majority classes. A fifth training set is constructed based on the synthetic embedding vectors corresponding to the labels of each fault category belonging to the minority class, the representative embedding vectors corresponding to the labels of each fault category belonging to the majority class, and the embedding vectors corresponding to the labels of each fault category belonging to the balanced class.

[0106] In some more specific examples, the label classification method involves counting the number of samples of the generated embedding vectors corresponding to fault-related log groups with the same fault category label, and then calculating the average of all samples of the vectors. and standard deviation , This represents the hyperparameter. Here, the number of samples in the embedding vector corresponding to a fault category label falls within the interval... The fault category labels are classified as balanced labels; the number of samples in the embedding vector corresponding to a fault category label. The fault category label is classified into a minority of labels; the number of samples in the embedding vector corresponding to a fault category label. The fault category labels are classified into multiple labels.

[0107] In some more specific examples, where a fault category is classified as a minority class, the embedding vector corresponding to the fault category label is the original embedding vector, and the number of samples is the original number of samples; the synthetic embedding vector is obtained based on the following method: a latent variable is sampled from a standard normal distribution, the fault category label vector is used as a condition of the CVAE model, and the decoder of the CVAE model generates the synthetic embedding vector of the latent variable. This step is repeated until the number of samples in the synthetic embedding vector is equal to the number of samples in the original sample. .

[0108] In some more specific examples, where a fault category is classified as a majority class, the embedding vector corresponding to the fault category label is the original embedding vector; the representative embedding vector of the fault category label is obtained as follows: based on the original embedding vector and the fault category label vector, a latent variable corresponding to the original embedding vector is generated by the encoder of the CVAE model, thus obtaining the latent variables of all embedding vectors corresponding to the fault category label; the latent variables of all embedding vectors form a category latent space region defined by a convex hull; for any sampled latent point, its nearest neighbor latent variable is found in the category latent space region to obtain the nearest neighbor vector, and this step is repeated to obtain the nearest neighbor vector of all latent points in the category latent space region, and the nearest neighbor vector is the representative embedding vector.

[0109] As a preferred approach, to effectively address the severe class imbalance problem in complex fault classification tasks, and to fully utilize the embedding representations generated by pre-trained language models, a resampling operation is performed in the embedding space using a trained conditional variational autoencoder (CVAE). This process includes oversampling (generating synthetic samples for the minority class) and downsampling (selecting representative samples for the majority class), with the ultimate goal of constructing a relatively class-balanced training set of embedding vectors.

[0110] First, calculate the average number of samples for all fault categories. and standard deviation And set hyperparameters The specific rule is the number of samples. The category was determined to be a minority class, requiring oversampling; the number of samples... The category is determined to be the majority class, requiring downsampling, and the sample size falls within the interval. Categories within the range are considered balanced, and their samples will be fully preserved.

[0111] For a fault category label vector of a minority class that is determined. Its oversampling process utilizes a trained CVAE decoder to generate synthetic embedding vector samples, with the goal of increasing the total number of samples of this type to [amount missing]. During operation, use the target category label. As a condition, latent variables are sampled from the standard normal distribution. Input CVAE decoder Generate synthetic embedding vectors .in, This process is repeated to obtain different latent variables. The synthetic embedding vectors are generated until the required number of synthetic samples are generated for the class, forming the union of the original sample set and the synthetic sample set.

[0112] For each majority class determined Its downsampling process aims to screen out First, a representative set of original samples is used. Then, a CVAE encoder is used to encode all the original embedding vector samples of this class into corresponding latent variables. , Represents the latent variables of all samples. Represents the original embedding vector. This represents a fault category label vector for the majority class. Next, uniform sampling is performed within the category latent space defined by the convex hull of these latent variables. Finally, for each uniformly sampled latent point... Find its nearest neighbor in the original set of latent variables. And retain the original embedding vector sample corresponding to the nearest neighbor. This allows us to obtain a subset of samples that uniformly covers the original potential distribution.

[0113] The resampled training dataset integrates the following samples: complete original samples from all balanced classes, original samples from all minority classes and their corresponding synthetic samples, and a subset of original samples from all majority classes retained after uniform sifting through the latent space. Each sample exists as its embedding vector, accompanied by a true class label. A key feature of this dataset is that the number of samples for each fault class has been adjusted to a level close to the target level. Considering that the number of failure modes varies for different failure types, the sample size was not forcibly standardized. This effectively alleviated the initial class imbalance problem and laid a data foundation for subsequent focus on the secondary fine-tuning of the classifier.

[0114] Step S50. Construct a fifth training set based on the synthetic embedding vectors corresponding to the labels of each fault category belonging to the minority labels and the representative embedding vectors corresponding to the labels of each fault category belonging to the majority labels. Use the fifth training set to train the third BERT model to obtain a fourth BERT model; wherein, the fourth BERT model is the network fault classification model. In some embodiments of the present invention, the BERT model parameters are frozen during training, and only the classification is fine-tuned.

[0115] In one embodiment, a network fault classification method is provided, based on a network fault classification model, which includes a fourth BERT model and an adapted classifier. The classifier includes a global average pooling layer, a fully connected layer, and a probability transformation layer. The network fault classification method includes the following steps: the fourth BERT model converts input data into sequence embedding vectors, where the input data includes fault-related log groups; the global average pooling layer compresses the sequence embedding vectors into a first feature vector; the fully connected layer performs a linear transformation on the first feature vector to obtain a second feature vector; the probability transformation layer maps the second feature vector output by the fully connected layer to the probability distribution of labels for each fault category using a sigmoid function; wherein the fourth BERT model is trained based on the aforementioned training method.

[0116] In a specific example, when the model is used for network fault classification, the model obtains the confidence score of the fault category of a fault-related log group from an input dataset that includes fault-related log groups; based on the confidence score of the fault category and a confidence threshold, the model outputs fault types with confidence scores higher than the confidence threshold, wherein if the highest confidence score among the fault categories is lower than the confidence threshold, the fault belongs to an unknown type of fault; fault type labels are obtained by labeling the fault-related log groups corresponding to unknown type faults; the fault-related log groups and the fault type labels are placed in a third training set to obtain an updated third training set, and the updated third training set is used for model training.

[0117] This invention describes a method for continuous system optimization from two dimensions: system deployment and continuous operation. In the system deployment phase, a log sequence extraction module is first deployed. After running for a period, this module generates a large number of fault-related log groups. These log groups are deduplicated to obtain several distinct samples. Subsequently, these deduplicated fault-related log groups are handed over to operations experts for fault type labeling. Simultaneously, a fault classification model is pre-trained using pre-collected log-description pairs and unlabeled fault-related log groups. After obtaining the expert-annotated dataset, the pre-trained fault classification model undergoes initial fine-tuning. Then, an embedding training conditional variational autoencoder (CVAE) model is generated based on this fine-tuned model, and the trained CVAE is used to resample the labeled dataset. Finally, the resampled embedding vector samples and their labels are integrated, and with the BERT backbone parameters frozen, a second round of fine-tuning training is performed only on the classifier (fully connected layer). After completing these steps, the system reaches a fully functional state.

[0118] During continuous system operation, when undefined fault-related log groups appear, they are submitted to operations and maintenance experts for analysis and fault type labeling. When a certain number of newly labeled samples accumulate, a model update process is triggered. The update method integrates these newly labeled samples with the existing labeled dataset and re-executes the complete training sequence, starting from the initial fine-tuning of the fault classification model, to training the CVAE model, completing resampling, and then to the secondary fine-tuning of the classifier. Due to the relatively small number of samples in the labeled dataset, the computational resources required for this update process are limited.

[0119] In the above scheme, the goal of log sequence extraction is to aggregate logs related to a specific complex network fault. First, this invention uses the Drain3 algorithm to parse the logs and obtain log templates, filtering out alarm-type log templates with higher severity, and grouping them according to device ID and a fixed 1-hour time window. Second, within each group, the PC algorithm is applied to examine the causal relationships between log templates, constructing a causal graph. Finally, all disconnected subgraphs are extracted from the generated causal graph; each disconnected subgraph constitutes a fault-related log group, representing an independent fault event.

[0120] In the above scheme, the goal of pre-training the fault classification model is to enhance its few-shot learning ability in downstream fault classification tasks by utilizing readily available network device log corpora. The model uses the fault-related log groups generated in step 1 as input text sequences, extracts latent state features through the BERT model, generates sequence representation vectors through global average pooling, and finally outputs multi-label fault classification probabilities using a fully connected layer combined with a sigmoid activation function. During the pre-training phase, only the parameters of the BERT model are optimized, enabling it to master the semantic information of network device logs and the contextual sequence relationships of complex fault logs through unsupervised learning. The pre-training tasks are twofold: one is log-description contrastive learning, where the input is log-log description pairs, with the log description containing an explanation of the log. Contrastive learning narrows the cosine distance between the BERT embedding vectors of the logs and log descriptions. The second task is word masking language modeling of fault-related log groups. This task utilizes massive amounts of fault-related log group data to train BERT to perform masking language modeling, i.e., predicting randomly masked words.

[0121] In the above scheme, the goal of fine-tuning the fault classification model is to adapt the pre-trained language model to the task of complex fault classification and to specifically address the class imbalance problem in this task. This invention introduces a Conditional Variational Autoencoder (CVAE) for sample resampling. The specific process is as follows: First, the pre-trained fault classification model is initially fine-tuned using an existing labeled dataset. Then, the pre-fine-tuned BERT is used to process all samples, generating corresponding embedding vector representations. Based on these embedding vectors and their labels, the CVAE model is trained through self-supervised learning. Next, for fault categories with insufficient sample numbers, the trained CVAE decoder generates samples in the form of embedding vectors based on the target labels (oversampling). For fault categories with excessive sample numbers, the CVAE encoder is used to calculate the latent variable Z of all its samples, and uniform sampling is performed in the latent space to achieve downsampling. Finally, the resampled samples in the form of embedding vectors (including original samples, synthetic samples, and selected samples) and their corresponding labels are integrated to form a new balanced training dataset. A second round of fine-tuning training is performed only on the classifier (fully connected layer), during which the BERT parameters remain frozen.

[0122] In the above scheme, the goal of continuous optimization of the fault classification model is to maintain the model's classification accuracy even when the system's operating mode changes. The specific process is as follows: First, a classification confidence threshold is set, and samples with prediction confidence scores below this threshold for all fault categories are labeled as "unknown faults." Then, the samples labeled "unknown faults" are submitted to operations and maintenance experts for fault analysis and manual labeling. The newly labeled samples are then integrated into the existing fine-tuning dataset. Finally, based on the BERT model parameters obtained in step 2, the entire fine-tuning process described in step 3 (including initial fine-tuning, CVAE resampling, and secondary classifier fine-tuning) is re-executed to update the model parameters and adapt to system changes.

[0123] Statistical analysis of the frequency of various network faults revealed a significant long-tail distribution characteristic. The reasons for this phenomenon are multi-dimensional:

[0124] First, from the perspective of fault mechanism, high-frequency faults usually originate from inherent defects in the system, common configuration errors, or failures of standardized components. Their triggering conditions are clear and their patterns are relatively fixed. In contrast, low-frequency faults are often caused by the coupling of complex environmental factors, rare hardware defects, or transient interaction anomalies that are difficult to reproduce. Their root causes are hidden and their manifestations are varied.

[0125] Secondly, from an operational practice perspective, resource allocation often prioritizes quickly resolving high-frequency faults to ensure the stability of basic services, while relatively insufficient investment is made in in-depth investigation of low-frequency faults. This strategy further exacerbates the polarization of fault frequencies. Furthermore, the extreme scarcity of low-frequency faults results in a limited absolute number of samples, significantly increasing the difficulty of effective analysis and accurate identification.

[0126] Specific case statistics show that high-frequency fault types, accounting for approximately 14%, account for 77% of all faults, while low-frequency fault types, accounting for approximately 32%, only account for 14% of all faults. This severe data imbalance causes the model to prioritize learning and fitting high-frequency fault features during training, while neglecting to learn the features of low-frequency faults. In other words, the model performance exhibits a significant "specialization" phenomenon, ultimately making it difficult to effectively identify low-frequency fault instances.

[0127] In model training, the Imbalance Ratio (IR) is commonly used to quantify the imbalance of a dataset. It is defined as the ratio of the number of samples of the fault type with the largest number of samples to the number of samples of the fault type with the smallest number of samples in the training dataset. Generally, an IR value greater than 100 is considered severely imbalanced. In this case, the IR value even exceeds 300. To mitigate this problem, the core objective is to reduce the IR value, primarily through oversampling and undersampling.

[0128] However, oversampling methods have inherent limitations: they are essentially based on inferring target fault modes from a limited number of samples to generate new samples. For low-frequency network faults with a very small absolute number of samples, it is difficult to generate large-scale reliable samples, and forced generation can easily lead to sample distortion (inconsistent with the real pattern) or pattern homogeneity (lack of diversity). Simply relying on undersampling, i.e., reducing the number of majority class samples, can reduce the IR value but cannot substantially improve the model's learning ability for minority class patterns. Therefore, this invention adopts a strategy of synergistic implementation of sample generation and undersampling. The advantages of this scheme are: first, it can efficiently reduce the IR value, and the reduction effect of undersampling and the incremental effect of sample generation form a significant multiplicative superposition effect; second, in the sample generation process, the Conditional Variational Autoencoder (CVAE) model is used to assist the fault classification model in deeply understanding the potential data distribution and feature patterns of low-frequency faults, which is equivalent to providing targeted feature learning guidance for the classification model; third, in the undersampling process, CVAE is used to guide the selection of majority class samples, reducing the total number of samples while preserving the key feature diversity and information richness of majority class samples to the greatest extent, avoiding information loss caused by random undersampling.

[0129] in, This threshold is the boundary for classifying a class as a minority because when the number of samples in a class approaches or exceeds this threshold, that class will not suffer from underfitting due to sample scarcity. However, if the original sample size corresponding to the minority class label is extremely limited, the CVAE model cannot fully learn its class feature distribution. Forcibly generating a large number of samples to reach the required sample size will inevitably lead to poor sample quality, failing to accurately reflect the true data distribution and key feature patterns of minority class faults. Instead, it will introduce noise patterns into the model and impair its generalization performance (i.e., "poison" the model). Therefore, when the original sample size is too small, the minimum generation target is set to double the original sample size. This design primarily considers that doubling the original sample size provides the model with the minimum effective sample size required to learn the basic representation patterns of this type of fault, prioritizing the confidence and pattern fidelity of the generated samples, and avoiding the negative effects caused by blindly pursuing quantity at the expense of quality. For each majority class determined... Its downsampling process aims to screen out First, a representative set of original samples is used. Then, a CVAE encoder is used to encode all the original embedding vector samples of this class into corresponding latent variables. . The embedded vectors are then represented; next, uniform sampling is performed within the category latent space region defined by the convex hull of these latent variables. Finally, for each uniformly sampled latent point... (The quantity is) Find its nearest neighbor in the original set of latent variables. And retain the original embedding vector sample corresponding to the nearest neighbor. This allows us to obtain a subset of samples that uniformly covers the original potential distribution.

[0130] Based on the above explanation of the range of logarithmic values, the following applies to this section. , The materials provided are the basis for their use.

[0131] As mentioned above, A sample size boundary is defined to avoid underfitting in the minority class. Symmetrically, Here, it is defined as an upper bound reference threshold for the distribution of the majority class samples. Its theoretical basis lies in the assumption that the sample size of each class approximately follows a Gaussian distribution in statistics. This represents the average sample size. The standard deviation represents the sample size. Located at the right tail of the Gaussian distribution, this value is selected as one of the lower limits of the target sample size, aiming to retain representative samples within the main distribution area. This is an adjustable hyperparameter used to control the breadth of the retained sample distribution. Setting the target sample size at least this value helps ensure that the sampled subset still covers the main distribution area of ​​the original majority class samples in the latent space, avoiding the loss of key patterns due to excessive reduction. The rationale is as follows: First, combining the minority class sample generation strategy described earlier (aiming to double the original minority class sample size) with the majority class sample size reduced to 1 / 5, theoretically, the imbalance ratio (IR) of the entire dataset can be significantly reduced by approximately 10 times. This magnitude of IR reduction is crucial for mitigating the problem of models overemphasizing the majority class. Second, this is an empirically strong compression ratio. In datasets with inherent structure, a carefully selected subset, much smaller than the original, can often effectively preserve the key feature information of the original distribution, sufficient to train a well-performing model. Finally, It provides a mandatory, absolute lower bound related to the original size to prevent the sample size obtained after sampling from being too low and losing representativeness when the original sample size is extremely large.

[0132] Regarding the values ​​of hyperparameters, the hyperparameters here are This is used to control the breadth of the retained sample distribution. We take... The value is 0.45, rather than the more common 2 or 3, because firstly, we need to ensure concentration, covering approximately 35% of the distribution range (central region). This value aims to balance sampling compression rate and distribution representativeness. Secondly, additional minimum and maximum thresholds related to the original sample size are set in sample generation and sampling to achieve effective division of labor (for extremely low and extremely high categories). Specifically, this scheme uses differentiated processing for fault types with different sample sizes: for fault types with a particularly small original sample size, the generation target is set to twice the original sample size to ensure the reliability of the basic sample; for fault types with a slightly smaller sample size but still higher than the minimum generation target, ... The threshold plays a role in guiding the sample size towards the equilibrium value; for fault types with a sample size within the normal range, no action is taken; for fault types with a large sample size, The threshold plays a role in guiding the sample size towards an equilibrium value; for fault types with a particularly large sample size, Thresholds play a crucial role in maintaining sample representativeness while significantly reducing the imbalance ratio (IR by approximately one order of magnitude). The various thresholds work together to achieve dataset balancing.

[0133] Experimental Example: This invention was deployed in a network of a certain operator, and a real-world application experiment was conducted for one month. First, the invention implemented a log sequence extraction module. After one week of operation, this module collected a total of 3641 different fault-related log groups. Simultaneously, the invention investigated the network equipment models used in the operator's existing network, confirming that it used Cisco's ASR920, C8500, and other models. Based on this, the invention collected 16 system message guides for relevant equipment models from the internet, compiled over 70,000 log-description pairs, and reviewed them using GPT4. Based on the collected fault-related log groups and the compiled log-description pair corpus, a fault classification model was pre-trained.

[0134] A total of 10,576 fault-related log groups were collected during the entire experiment, which were labeled by operations and maintenance experts and belonged to 23 fault types. A stratified random sampling method was used to divide the training and test sets in an 8:2 ratio, resulting in 8,461 training set samples and 2,115 test set samples. This invention uses the training set to complete model fine-tuning, which exhibits a strong long-tail distribution characteristic, with the mean number of samples for each fault category being... Standard deviation First, preliminary fine-tuning training is performed on the pre-trained model; then, based on the embedding vectors generated by this fine-tuned model, a conditional variational autoencoder (CVAE) model is trained, and resampling operations (hyperparameters) are performed using the trained CVAE. (i.e., a minority class threshold of 45 and a majority class threshold of 691). Resampling significantly improved class balance (see related figures for the process and distribution changes), reducing the total sample size from 8,461 to 4,884, a reduction of 42.28% of redundant samples. Finally, based on the resampled embedded vector samples and their labels, a second round of fine-tuning training was performed only on the classifier (fully connected layer) while freezing the parameters of the BERT backbone network.

[0135]

[0136] The table lists 23 categories, including power alarm category A, port channel member link interruption alarm category B, optical module alarm category C, L2VPN bridge domain MAC address restriction alarm category D, and line card fault alarm category E. To evaluate the effectiveness of the method, five schemes were compared on the test set: Scheme 1 (baseline) fine-tuned based on the original BERT parameters (no pre-training, no resampling); Scheme 2 fine-tuned only using the pre-trained model (no resampling); Scheme 3, based on Scheme 2, reduced the dataset imbalance only through oversampling and performed secondary fine-tuning on the adjusted dataset; Scheme 4, based on Scheme 2, reduced the dataset imbalance only through downsampling and performed secondary fine-tuning on the adjusted dataset; Scheme 5 (complete scheme) adopted the pre-training + CVAE resampling + secondary fine-tuning method proposed in this paper. Given the highly imbalanced data, Jaccard coefficients were used for evaluation. The sample average Jaccard coefficient (JSAMPLE) reflects the overall accuracy of a single sample's predicted label set and exhibits good robustness to label imbalance. The label average Jaccard coefficient (JMACRO) equally weights each category and is more sensitive to performance in low-frequency fault categories. The evaluation results are shown in the table below. The complete solution achieves significant improvements in both metrics, fully demonstrating that the pre-training-fine-tuning framework effectively utilizes log-description to improve the basic performance of the corpus and unlabeled log sets. Simultaneously, the CVAE-based resampling strategy plays a crucial role in addressing the problems of scarce labeled data and extreme imbalance, and in improving the model's recognition ability (especially for low-frequency fault categories).

[0137]

[0138] Based on the above embodiments, this application also provides a computer program that, when run on a computer, causes the computer to execute the methods provided in the above embodiments.

[0139] Based on the above embodiments, this application also provides a computer storage medium storing a computer program, which, when executed by a computer, causes the computer to perform the methods provided in the above embodiments.

[0140] The storage medium can be any available medium that a computer can access. For example, but not limited to, a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0141] Based on the above embodiments, this application also provides a chip for reading a computer program stored in a memory to implement the method provided in the above embodiments.

[0142] Based on the above embodiments, this application provides a computer program product that implements the methods provided in the above embodiments when the computer program product is run on an electronic device.

[0143] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0145] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0147] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for training a network fault classification model, characterized in that, include The first BERT model is pre-trained using the training set to obtain the second BERT model. The pre-training tasks include a log-description contrast learning task and a word mask prediction task for fault-related log groups. The training set includes the second training set, which includes samples from fault-related log groups. The logs in the fault-related log groups represent the same fault type based on causal relationships, and the logs are arranged in chronological order. The third BERT model is obtained by fine-tuning the second BERT model and its adapted classifier using the third training set. The third training set includes fault-related log groups and their corresponding fault types from the second training set. Different fault-related log groups may have labels for the same fault category or labels for different fault categories. The third BERT model generates embedding vectors corresponding to the samples of the fault-related log groups in the third training set. The embedding vectors and their corresponding fault category labels are used to construct the fourth training set. The CVAE model is trained using the fourth training set to obtain the first CVAE model. Based on the labels of the fault categories corresponding to each embedding vector in the fourth training set, the number of samples corresponding to the embedding vectors of different fault categories is obtained, and the fault category classification includes minority and majority classes based on the number of samples. The embedding vectors corresponding to the same fault category belonging to the minority fault category are obtained by performing an oversampling operation on the first CVAE model to obtain the synthetic embedding vectors of the fault category, thus obtaining the synthetic embedding vectors corresponding to the fault categories belonging to the minority fault category. The embedding vectors corresponding to the same fault category belonging to the majority fault category are obtained by performing downsampling operation on the first CVAE model to obtain the representative embedding vectors of the fault category, thus obtaining the representative embedding vectors corresponding to the fault category belonging to the majority fault category. A fifth training set is constructed based on the synthetic embedding vectors corresponding to the minority class of faults and the representative embedding vectors corresponding to the majority class of faults. The fifth training set is used to fine-tune the third BERT model and its adapted classifier to obtain the fourth BERT model. The fourth BERT model and its adapted classifier are the network fault classification model.

2. The method according to claim 1, characterized in that, Obtain fault-related log groups from network logs, including Log templates are obtained from network logs, and frequency time series of each log template for the device are constructed. Construct a fully connected undirected graph of log templates; Based on the frequency time series, causal relationship tests are performed on the edges in the fully connected undirected graph, and pseudo-causal edges in the fully connected undirected graph are removed to obtain several connected subgraphs. The log templates corresponding to all nodes in the connected subgraph are arranged in chronological order to obtain the corresponding fault-related log groups.

3. The method according to claim 2, characterized in that, The training set also includes a first training set, which includes log and descriptive data.

4. The method according to any one of claims 1-3, characterized in that, The number of samples of the generated embedding vectors corresponding to the fault-related log groups for each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples corresponds to a fault category. The fault categories are classified as minority classes; where, if a fault category is classified as a minority class, the embedding vector corresponding to that fault category is the original embedding vector, and the number of samples is the original sample number; the synthetic embedding vector is obtained based on the following method: a latent variable is sampled from a standard normal distribution, the fault category label vector is used as a condition of the CVAE model, and the decoder of the CVAE model generates the synthetic embedding vector of the latent variable. This step is repeated until the sum of the number of samples in the synthetic embedding vector and the number of samples in the original sample is equal to the original sample number. .

5. The method according to any one of claims 1-3, characterized in that, The number of samples of the generated embedding vectors corresponding to the fault-related log groups for each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples corresponds to a fault category. The fault categories are classified into majority classes; where, if a fault category is classified into majority classes, the embedding vector corresponding to the fault category is the original embedding vector; the representative embedding vector of the fault category is obtained based on the following method: according to the original embedding vector and the fault category label vector, a latent variable corresponding to the original embedding vector is generated by the encoder of the CVAE model, and the latent variables of all embedding vectors corresponding to the fault category are obtained; latent points are sampled in the convex hull region of the category latent space constructed by all latent variables, and the nearest neighbor latent variable is found for each sampled point; the embedding vector corresponding to the nearest neighbor latent variable is used as the representative embedding vector.

6. The method according to claim 1, characterized in that, in, Fault category classification also includes equalization category; The number of samples of the generated embedding vectors corresponding to the fault-related log groups for each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples of the embedding vector corresponding to a fault category falls within the interval [0, 1]. The fault category is classified as balanced. Specifically, a fifth training set is constructed based on the embedding vectors corresponding to the fault categories of the balanced class, the synthetic embedding vectors and their corresponding original embedding vectors corresponding to the fault categories of the minority class, and the representative embedding vectors corresponding to the fault categories of the majority class. The fifth training set is used to fine-tune the third BERT model and its adapted classifier to obtain the fourth BERT model. The fourth BERT model and its adapted classifier are the network fault classification model.

7. The method according to claim 1, characterized in that, The training also includes: In the model used for network fault classification, the model obtains the confidence level of the fault category of a fault-related log group based on a fault-related log group in the input dataset that includes fault-related log groups. Based on the confidence level of the fault category and the confidence level threshold, output the fault type with a confidence level higher than the confidence level threshold. If the highest confidence level among the fault categories is lower than the confidence level threshold, then a fault-related log group belongs to an unknown type of fault. For fault-related log groups corresponding to unknown types of faults, fault type labels are obtained by labeling the fault category labels. The fault-related log groups and the fault category labels are placed in the third training set to obtain an updated third training set, which is then used for model training.

8. The method according to claim 1, characterized in that, The log-description comparison learning task and the fault-related log group word mask prediction task are jointly trained. The model parameters are jointly optimized during training by MLM loss and LDA loss until they converge to the optimal parameters. In the log-description contrastive learning task, BERT's input includes logs and descriptions. BERT encodes logs and descriptions into hidden representations. It then performs contrastive learning on positive pairs of logs and descriptions and negative pairs of logs and descriptions in the hidden representations. The optimization objectives include minimizing the positive sample distance and maximizing the negative sample distance. In the word masking prediction task for fault-related log groups, the input of BERT includes fault-related log groups, and a certain proportion of logs in each fault-related log group are randomly masked. BERT encodes the fault-related log groups into hidden representations for predicting the mask, where the optimization objective includes cross-entropy loss to predict the masked logs.

9. A network fault classification method, based on a network fault classification model, characterized in that, The network fault classification model includes the fourth BERT model and its adapted classifier, wherein the classifier includes a global average pooling layer, a fully connected layer and a probability transformation layer; Network fault classification methods include the following steps: The fourth BERT model converts the input data into sequence embedding vectors, where the input data includes fault-related log groups; The global average pooling layer compresses the sequence embedding vector into a first feature vector; The fully connected layer performs a linear transformation on the first feature vector to obtain the second feature vector; The probability transformation layer maps the second feature vector output by the fully connected layer to the probability distribution of the labels for each fault category through the Sigmoid function; The network fault classification model is trained based on the training method described in claim 1.

10. The method according to claim 9, characterized in that, Obtain fault-related log groups from network logs, including Log templates are obtained from network logs, and frequency time series of each log template for the device are constructed. Construct a fully connected undirected graph of log templates; Based on the frequency time series, causal relationship tests are performed on the edges in the fully connected undirected graph, and pseudo-causal edges in the fully connected undirected graph are removed to obtain several connected subgraphs. The log templates corresponding to all nodes in the connected subgraph are arranged in chronological order to obtain the corresponding fault-related log groups.

11. The method according to claim 10, characterized in that, The training set also includes a first training set, which includes log and descriptive data.

12. The method according to any one of claims 9-11, characterized in that, The number of samples of the generated embedding vectors corresponding to the fault-related log groups for each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples corresponds to a fault category. The fault categories are classified as minority classes; where, if a fault category is classified as a minority class, the embedding vector corresponding to that fault category is the original embedding vector, and the number of samples is the original sample number; the synthetic embedding vector is obtained based on the following method: a latent variable is sampled from a standard normal distribution, the fault category label vector is used as a condition of the CVAE model, and the decoder of the CVAE model generates the synthetic embedding vector of the latent variable. This step is repeated until the sum of the number of samples in the synthetic embedding vector and the number of samples in the original sample is equal to the original sample number. .

13. The method according to any one of claims 9-11, characterized in that, The number of samples of the generated embedding vectors corresponding to the fault-related log groups for each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples corresponds to a fault category. The fault categories are classified into majority classes; where, if a fault category is classified into majority classes, the embedding vector corresponding to the fault category is the original embedding vector; the representative embedding vector of the fault category is obtained based on the following method: according to the original embedding vector and the fault category label vector, a latent variable corresponding to the original embedding vector is generated by the encoder of the CVAE model, and the latent variables of all embedding vectors corresponding to the fault category are obtained; latent points are sampled in the convex hull region of the category latent space constructed by all latent variables, and the nearest neighbor latent variable is found for each sampled point; the embedding vector corresponding to the nearest neighbor latent variable is used as the representative embedding vector.

14. The method according to claim 9, characterized in that, in, Fault category classification also includes equalization category; The number of samples of the generated embedding vectors corresponding to the fault-related log groups for each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples of the embedding vector corresponding to a fault category falls within the interval [0, 1]. The fault category is classified as balanced. Specifically, a fifth training set is constructed based on the embedding vectors corresponding to the fault categories of the balanced class, the synthetic embedding vectors and their corresponding original embedding vectors corresponding to the fault categories of the minority class, and the representative embedding vectors corresponding to the fault categories of the majority class. The fifth training set is used to fine-tune the third BERT model and its adapted classifier to obtain the fourth BERT model. The fourth BERT model and its adapted classifier are the network fault classification model.

15. The method according to claim 9, characterized in that, The training also includes: In the model used for network fault classification, the model obtains the confidence level of the fault category of a fault-related log group based on a fault-related log group in the input dataset that includes fault-related log groups. Based on the confidence level of the fault category and the confidence level threshold, output the fault type with a confidence level higher than the confidence level threshold. If the highest confidence level among the fault categories is lower than the confidence level threshold, then a fault-related log group belongs to an unknown type of fault. For fault-related log groups corresponding to unknown types of faults, fault type labels are obtained by labeling the fault category labels. The fault-related log groups and the fault category labels are placed in the third training set to obtain an updated third training set, which is then used for model training.

16. The method according to claim 9, characterized in that, The log-description comparison learning task and the fault-related log group word mask prediction task are jointly trained. The model parameters are jointly optimized during training by MLM loss and LDA loss until they converge to the optimal parameters. In the log-description contrastive learning task, BERT's input includes logs and descriptions. BERT encodes logs and descriptions into hidden representations. It then performs contrastive learning on positive pairs of logs and descriptions and negative pairs of logs and descriptions in the hidden representations. The optimization objectives include minimizing the positive sample distance and maximizing the negative sample distance. In the word masking prediction task for fault-related log groups, the input of BERT includes fault-related log groups, and a certain proportion of logs in each fault-related log group are randomly masked. BERT encodes the fault-related log groups into hidden representations for predicting the mask, where the optimization objective includes cross-entropy loss to predict the masked logs.

17. A network fault classification device, comprising a network fault classification model, characterized in that, The network fault classification model includes the fourth BERT model and its adapted classifier, wherein the classifier includes a global average pooling layer, a fully connected layer and a probability transformation layer; The fourth BERT model is used to convert the input data into sequence embedding vectors, where the input data includes fault-related log groups; A global average pooling layer is used to compress the sequence embedding vector into a first feature vector; A fully connected layer is used to perform a linear transformation on the first feature vector to obtain the second feature vector; The probability transformation layer is used to map the second feature vector output by the fully connected layer to the probability distribution of the labels of each fault category through the Sigmoid function; The network fault classification model is trained based on the training method described in claim 1.

18. The apparatus according to claim 17, characterized in that, Obtain fault-related log groups from network logs, including Log templates are obtained from network logs, and frequency time series of each log template for the device are constructed. Construct a fully connected undirected graph of log templates; Based on the frequency time series, causal relationship tests are performed on the edges in the fully connected undirected graph, and pseudo-causal edges in the fully connected undirected graph are removed to obtain several connected subgraphs. The log templates corresponding to all nodes in the connected subgraph are arranged in chronological order to obtain the corresponding fault-related log groups.

19. The apparatus according to claim 18, characterized in that, The training set also includes a first training set, which includes log and descriptive data.

20. The apparatus according to any one of claims 17-19, characterized in that, The number of samples of the generated embedding vectors corresponding to the fault-related log groups for each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples corresponds to a fault category. The fault categories are classified as minority classes; where, if a fault category is classified as a minority class, the embedding vector corresponding to that fault category is the original embedding vector, and the number of samples is the original sample number; the synthetic embedding vector is obtained based on the following method: a latent variable is sampled from a standard normal distribution, the fault category label vector is used as a condition of the CVAE model, and the decoder of the CVAE model generates the synthetic embedding vector of the latent variable. This step is repeated until the sum of the number of samples in the synthetic embedding vector and the number of samples in the original sample is equal to the original sample number. .

21. The apparatus according to any one of claims 17-19, characterized in that, The number of samples of the generated embedding vectors corresponding to the fault-related log groups for each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples corresponds to a fault category. The fault categories are classified into majority classes; where, if a fault category is classified into majority classes, the embedding vector corresponding to the fault category is the original embedding vector; the representative embedding vector of the fault category is obtained based on the following method: according to the original embedding vector and the fault category label vector, a latent variable corresponding to the original embedding vector is generated by the encoder of the CVAE model, and the latent variables of all embedding vectors corresponding to the fault category are obtained; latent points are sampled in the convex hull region of the category latent space constructed by all latent variables, and the nearest neighbor latent variable is found for each sampled point; the embedding vector corresponding to the nearest neighbor latent variable is used as the representative embedding vector.

22. The apparatus according to claim 17, characterized in that, in, Fault category classification also includes equalization category; The number of samples of the generated embedding vectors corresponding to the fault-related log groups for each fault category is counted, and the average value is calculated. and standard deviation , Denotes the hyperparameter, where the number of samples of the embedding vector corresponding to a fault category falls within the interval [0, 1]. The fault category is classified as balanced. Specifically, a fifth training set is constructed based on the embedding vectors corresponding to the fault categories of the balanced class, the synthetic embedding vectors and their corresponding original embedding vectors corresponding to the fault categories of the minority class, and the representative embedding vectors corresponding to the fault categories of the majority class. The fifth training set is used to fine-tune the third BERT model and its adapted classifier to obtain the fourth BERT model. The fourth BERT model and its adapted classifier are the network fault classification model.

23. The apparatus according to claim 17, characterized in that, The training also includes: In the model used for network fault classification, the model obtains the confidence level of the fault category of a fault-related log group based on a fault-related log group in the input dataset that includes fault-related log groups. Based on the confidence level of the fault category and the confidence level threshold, output the fault type with a confidence level higher than the confidence level threshold. If the highest confidence level among the fault categories is lower than the confidence level threshold, then a fault-related log group belongs to an unknown type of fault. For fault-related log groups corresponding to unknown types of faults, fault type labels are obtained by labeling the fault category labels. The fault-related log groups and the fault category labels are placed in the third training set to obtain an updated third training set, which is then used for model training.

24. The apparatus according to claim 17, characterized in that, The log-description comparison learning task and the fault-related log group word mask prediction task are jointly trained. The model parameters are jointly optimized during training by MLM loss and LDA loss until they converge to the optimal parameters. In the log-description contrastive learning task, BERT's input includes logs and descriptions. BERT encodes logs and descriptions into hidden representations. It then performs contrastive learning on positive pairs of logs and descriptions and negative pairs of logs and descriptions in the hidden representations. The optimization objectives include minimizing the positive sample distance and maximizing the negative sample distance. In the word masking prediction task for fault-related log groups, the input of BERT includes fault-related log groups, and a certain proportion of logs in each fault-related log group are randomly masked. BERT encodes the fault-related log groups into hidden representations for predicting the mask, where the optimization objective includes cross-entropy loss to predict the masked logs.

Images