Network fault detection method, device, equipment and medium based on semi-supervised learning

By adopting semi-supervised learning method and Tri-training algorithm in network fault detection, a small amount of label data and a large amount of labelless data are used to improve the performance of the base classifier, and the problems of data distribution imbalance and model overfitting in the existing technology are solved, thereby achieving higher detection accuracy and wider applicable scenarios.

CN115883329BActive Publication Date: 2025-03-14广州广哈通信股份有限公司
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202211593698.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-13
Publication Date
2025-03-14
Estimated Expiration
2042-12-13

AI Technical Summary

Technical Problem

When existing network fault detection methods deal with massive multi-dimensional data in complex networks, it is difficult to effectively solve the problem of uneven data distribution, resulting in deep learning models being easily overfitted and not suitable for a wide range of scenarios.

Method used

The three base classifiers are pre-trained using a small amount of labeled data, and the performance of the base classifier is improved through a large amount of labelless data, and network failures are detected through Tri-training algorithm and weighted voting.

Benefits of technology

It improves the accuracy of network fault detection, is applicable to a wider range of scenarios, reduces the cost of labeling data, and solves the problem of data distribution imbalance through improved loss functions, improving model performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115883329B_ABST
    Figure CN115883329B_ABST
Patent Text Reader

Abstract

The present invention discloses a network fault detection method, device, equipment and medium based on semi-supervised learning, the method comprising: collecting a network status data set from a network environment; preprocessing the network status data set, wherein the data set containing a label is recorded as a labeled data set, and the remaining data is recorded as an unlabeled data set; using the Tri-training algorithm to train the three pre-trained base classifiers to obtain a trained network fault model; assigning weights to the three base classifiers in the network fault model respectively; inputting the network status data to be detected into the trained network fault model to obtain a detection result of the network status. The present invention can use a small amount of labeled data to detect whether the network status is faulty, thereby improving the accuracy of detecting network faults, being applicable to a wider range of scenarios, and reducing the cost of annotating data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of network management technology, and in particular to a network fault detection method, device, equipment and medium based on semi-supervised learning. Background Art

[0002] The services on modern networks are becoming more and more diverse and complex, and network management is very difficult. If a network failure occurs, it will have a huge impact on the business. Therefore, network fault detection is very important for network management and improving network service performance.

[0003] In the early days, network fault detection was mostly done manually, but as networks became increasingly complex, manual detection was costly and inefficient. Complex networks generate massive amounts of data every moment, and the distribution of these data is uneven. Since the network is operating normally most of the time, the amount of data when a fault occurs is relatively small; at the same time, these data are multidimensional, including both character data and numerical data, which represent the state of the network from various different perspectives.

[0004] In the patent "A network fault diagnosis method based on deep learning in a virtual network environment (CN106603293A)", the method obtains network data from a database and preprocesses the data: oversampling the faulty data in the original data set and undersampling the normal data to solve the problem of uneven data distribution, and detecting network faults based on the trained LSTM model. However, the method for solving the imbalanced data distribution in this method is not reasonable, and the trained deep learning model is prone to overfitting and is not suitable for scenarios with massive data. Summary of the invention

[0005] The embodiments of the present invention provide a network fault detection method, apparatus, device and medium based on semi-supervised learning. After pre-training three base classifiers with a small amount of labeled data, the performance of the base classifiers is improved through a large amount of unlabeled data. Finally, the network is detected to be faulty by weighted voting of the three base classifiers, thereby improving the accuracy of detecting network faults, being applicable to a wider range of scenarios, and reducing the cost of annotating data.

[0006] To achieve the above object, an embodiment of the present invention provides a network fault detection method based on semi-supervised learning, comprising:

[0007] Collect network status datasets from the network environment;

[0008] Preprocessing the network status data set; wherein the data set containing labels in the network status data set is recorded as a real label data set, and the remaining data is recorded as an unlabeled data set;

[0009] Use the Tri-training algorithm to train the three pre-trained base classifiers to obtain a trained network fault model;

[0010] Assigning weights to three base classifiers in the network fault model respectively;

[0011] The network status data to be detected is input into the trained network fault model to obtain the detection result of the network status.

[0012] Furthermore, the use of the Tri-training algorithm to train the three pre-trained base classifiers to obtain a trained network fault model specifically includes:

[0013] The real label data set is sampled to obtain three different real label training data sets, denoted as L i (=1,2,3);

[0014] Use three real label training data sets to pre-train three base classifiers until the base classifier models converge. The three base classifiers are denoted as h i (=1,2,3);

[0015] The three pre-trained base classifiers are trained using the Tri-training algorithm until the weights of the base classifiers in the network fault model are no longer updated or the number of training times reaches the set threshold, and a trained network fault model is obtained. Each training includes:

[0016] Use the three pre-trained base classifiers to annotate the unlabeled data set; generate an expanded sample set S for each base classifier i (=1, 2, 3); using the expanded sample set to train the three base classifiers again respectively; wherein the expanded sample set will be updated after each training.

[0017] Furthermore, the three base classifiers are pre-trained using the three real label training data sets until the base classifier models converge, where the three base classifiers are denoted as h i (=1,2,3), specifically including:

[0018] Input the training samples into the base classifier, where the training samples belong to the real label training data set;

[0019] The forward propagation outputs the predicted value and the loss value is calculated using the improved loss function;

[0020] Back-propagation updates the weights of the base classifier until the base classifier model converges.

[0021] Among them, the improved loss function is specifically:

[0022] Loss Function Where y is the true label of the data, y′ is the label output by the model, α is the weight parameter, and γ is the focus parameter, where γ is calculated based on each type of label data set, specifically:

[0023] Calculate C i The sample center point μ i : Among them, the training sample x∈C i , C i is the i-th label dataset;

[0024] Calculate the training sample x to the sample center point μ i Cosine similarity of:

[0025] Calculate the γ parameter: γ = 1-cos(x,μ i ).

[0026] Furthermore, the three pre-trained base classifiers are used to annotate the unlabeled data set; an expanded sample set S of each base classifier is generated. i (=1, 2, 3); using the expanded sample set to train the three base classifiers again, specifically including:

[0027] Use the three trained base classifiers to annotate the unlabeled dataset;

[0028] when When two different base classifiers h i 、h j When the label of an unlabeled sample is the same, the unlabeled sample is labeled and added to the third classifier h k The pseudo-label training dataset L t k among represents the error rate of the ith base classifier in the tth iteration;

[0029] Generate the pseudo-label training dataset L for the tth iteration t i (=1,2,3);

[0030] when When, from L t i Randomly delete a number of samples; where | t i |Represents L t i The number of samples;

[0031] Update the expanded sample set S of each base classifier i, S i =L i ∪L t i ;

[0032] The three base classifiers are trained again respectively using the expanded sample set.

[0033] Furthermore, the weights are assigned to the base classifiers in the network fault model, specifically including:

[0034] Calculate the base classifier h i Micro-F1 score F i , calculate the weight of h1

[0035] Furthermore, the network status data to be detected is input into the trained network fault model to obtain the detection result of the network status, which specifically includes:

[0036] After the network status data to be detected is input into the three base classifiers respectively, the detection result Y is obtained by weighted voting. t ′ :

[0037]

[0038] Among them, i represents the i-th base classifier, j represents the j-th network state category label, X t is the N-dimensional eigenvalue of the network at sample time t that is input into the network fault model.

[0039] Accordingly, an embodiment of the present invention further provides a network fault detection device based on semi-supervised learning, the device comprising:

[0040] A data acquisition module, used for collecting network status data sets from the network environment;

[0041] A data preprocessing module, used to preprocess the network status data set; wherein the data set containing labels in the network status data set is recorded as a real label data set, and the remaining data is recorded as an unlabeled data set;

[0042] The pre-training module is used to train the three pre-trained base classifiers using the Tri-training algorithm to obtain a trained network fault model;

[0043] A weight calculation module, used to assign weights to the three base classifiers in the network fault model respectively;

[0044] The detection module is used to input the network status data to be detected into the trained network fault model to obtain the detection result of the network status.

[0045] Correspondingly, an embodiment of the present invention also provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, any one of the above-mentioned network fault detection methods based on semi-supervised learning is implemented.

[0046] Correspondingly, an embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned network fault detection methods based on semi-supervised learning.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention uses a small amount of labeled data and a large amount of unlabeled data sets, combines the Tri-training algorithm to pre-train three base classifiers, and then uses two of the base classifiers to generate pseudo-labeled samples to expand the training sample set of the third base classifier, and then trains the third base classifier to improve its performance; finally, the network fault state is determined by weighted voting after training multiple base classifiers, so that the performance is better and the detection result is more accurate; in addition, only a small amount of labeled data is used, and a semi-supervised learning method is used to reduce the cost of annotating data, and the applicable scenarios are wider; in the embodiment of the present invention, the imbalanced data distribution is processed through the loss function, and the parameters in the loss function are improved, so that the model pays less attention to outliers, thereby improving the model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flowchart of a network fault detection method based on semi-supervised learning provided by an embodiment of the present invention;

[0050] Figure 2 It is a structural schematic diagram of a network fault detection device based on semi-supervised learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] See also Figure 1 , is a flow chart of a network fault detection method based on semi-supervised learning provided by an embodiment of the present invention.

[0053] The network fault detection method based on semi-supervised learning provided by the embodiment of the present invention includes steps S1 to S5, which are specifically as follows:

[0054] S1, collects network status dataset from the network environment;

[0055] In a specific embodiment, a network status data set is collected in a network environment, including link data such as link delay, bandwidth, packet loss rate, CPU occupancy rate of the switch, memory rate, read and write rate, and network node data;

[0056] S2, preprocessing the network status data set; wherein the data set containing labels in the network status data set is recorded as a real label data set, and the rest of the data is recorded as an unlabeled data set;

[0057] In a specific embodiment, the collected network status data is raw data, and the data needs to be preprocessed to structure, quantify and normalize all the data, wherein the network status data set includes a real label data set and an unlabeled data set;

[0058] S3, use the Tri-training algorithm to train the three pre-trained base classifiers to obtain a trained network fault model;

[0059] In one embodiment, step S3 includes steps S31 to S35, which are specifically as follows:

[0060] S31, sampling the real label data set to obtain three different real label training data sets, denoted as L i (=1,2,3);

[0061] S32, use three real label training data sets to pre-train three base classifiers respectively until the base classifier models converge, where the three base classifiers are denoted as h i (=1,2,3);

[0062] S33, using the Tri-training algorithm to train the three pre-trained base classifiers until the weights of the base classifiers in the network fault model are no longer updated or the number of training times reaches a set threshold, thereby obtaining a trained network fault model, wherein each training includes:

[0063] S331, labeling the unlabeled data set using the three pre-trained base classifiers;

[0064] S332, generate an expanded sample set S for each base classifier i (=1,2,3);

[0065] S333, using the expanded sample set to train three base classifiers again respectively;

[0066] The expanded sample set is updated after each training.

[0067] It should be noted that the three base classifiers are recurrent neural network (RNN), long short-term memory recurrent neural network (LSTM) and gated recurrent neural network (GRU). There is no fixed order, but in the specific implementation, a fixed order is required; three different data sets are used to train three different base classifiers to prevent overfitting due to the performance of the three base classifiers being too similar, which makes it impossible to achieve good network fault detection results.

[0068] In one embodiment, in step S32, three base classifiers are pre-trained using three real label training data sets until the base classifier models converge, wherein the three base classifiers are respectively denoted as h i (=1,2,3), specifically including steps S321 to S324, as follows:

[0069] S321, inputting training samples into a base classifier, wherein the training samples belong to a real label training data set;

[0070] S322, forward propagation outputs the predicted value, and the loss value is calculated using the improved loss function;

[0071] S323, back propagation updates the weights of the base classifier;

[0072] S324, repeat steps S321 to S323 until the base classifier model converges.

[0073] In one embodiment, the improved loss function is specifically:

[0074] Loss Function Where y is the true label of the data, y′ is the label output by the model, α is the weight parameter, and γ is the focus parameter, where γ is calculated based on each type of label data set, specifically:

[0075] Calculate C i The sample center point μ i : Among them, the training sample x∈C i , C i is the i-th label dataset;

[0076] Calculate the training sample x to the sample center point μ i Cosine similarity of:

[0077] Calculate the γ parameter: γ = 1-cos(x,μ i ).

[0078] Specifically, for the training sample x∈C i , if it is closer to the center point of this type of sample, μ i The closer the γ parameter value is, the closer it is to 0, and the larger the loss function value is. Conversely, the closer the γ parameter value is to 2, the smaller the loss function value is. From this, it can be seen that the loss function value of outliers will be smaller, and the model will pay less attention to them.

[0079] It is understandable that there are two parameters, α and γ, in the loss function, where α can increase the weight of minority samples in the loss function and reduce the weight of majority samples in the loss function, so that the influence of majority samples on the model will make the model pay more attention to minority samples, thereby achieving the effect of balancing the data set.

[0080] The γ parameter makes the model pay more attention to samples that are difficult to distinguish, and the model tends to focus on outliers, resulting in a decrease in model performance. For example, assuming that when γ = 2, the more difficult the sample is to distinguish, (1-y′) γ and y′ γ The larger the value, the larger the loss function value of the sample will be, and the more it will affect the weight update of the model; therefore, the γ parameter in the loss function is changed from a fixed value shared by all samples to a parameter value for each sample that is calculated.

[0081] It should be noted that the loss functions of the three base classifiers are all improved Focal Loss loss functions, where the value of parameter α is the proportion of faulty samples in the labeled data set to all samples; the value of parameter γ is calculated using the above improved method.

[0082] In one embodiment, in step S33, the three pre-trained base classifiers are trained using the Tri-training algorithm until the weights of the base classifiers in the network fault model are no longer updated or the number of training times reaches a set threshold, thereby obtaining a trained network fault model, wherein each training includes:

[0083] S331, labeling the unlabeled data set using the three pre-trained base classifiers;

[0084] S332, generate an expanded sample set S for each base classifier i (=1,2,3), specifically including:

[0085] Generate the pseudo-label training dataset L for the tth iteration t i (=1,2,3);

[0086] Specifically, when When, from L t i Randomly delete a number of samples; where | t i |Represents L t i The number of samples, L t i That is, the pseudo-label training set used by the i-th base classifier in the t-th iteration.

[0087] in, This means that the model effect is not good in the tth iteration, which is manifested by a high error rate or L t i There are too many samples in the L t i It is not a real training data set. Some samples are mislabeled, resulting in poor training results. Therefore, it is necessary to t i Some samples are randomly deleted from t to improve the training effect of the tth iteration.

[0088] Update the expanded sample set S of each base classifier i , S i =L i ∪L t i .

[0089] Specifically, L i is the real label dataset, L t i The two datasets are combined as a new training set, that is, the expanded sample set S i , and then expand the sample set S i Used in training at the tth iteration.

[0090] S333, using the expanded sample set to train the three base classifiers again respectively; wherein the expanded sample set is updated after each training.

[0091] S4, assigning weights to the three base classifiers in the network fault model respectively;

[0092] Specifically, the three base classifiers in the network fault model are assigned weights respectively, including:

[0093] Calculate the base classifier h i Micro-F1 score F i , calculate h i Weight

[0094] S5, inputting the network status data to be detected into the trained network fault model to obtain the detection result of the network status.

[0095] Specifically, the network status data to be detected is input into the trained network fault model to obtain the detection result of the network status, which specifically includes:

[0096] After the network status data to be detected is input into the three base classifiers respectively, the detection result Y is obtained by weighted voting. t ′ :

[0097]

[0098] Among them, i represents the i-th base classifier, j represents the j-th network state category label, X t is the N-dimensional eigenvalue of the network at sample time t that is input into the network fault model.

[0099] It should be noted that y j ={y1,y2,...,y M}, that is, the network state set (category tag set), each element in the set represents a network state. It can be mainly divided into two categories, namely, normal network and abnormal network; among them, network abnormality can also be refined into many category tags, such as link failure, such as link congestion, link disconnection, network node failure such as insufficient switch memory, high CPU usage or mixed failure such as abnormal link and node; among them, Y t represents the state of the network at time t, and Y t ∈y j .

[0100] In a specific embodiment, assuming that the network status is divided into only two categories: normal and abnormal. t , each value in the table represents the base classifier h i Judgment sample X t Belong to y j The probability of X t The corresponding probability of the network status being normal is 0.1, that is,

[0101] Specifically, calculate the probability that the network status is normal:

[0102] Calculate the probability that the network status is abnormal:

[0103] That is, through the three base classifiers, the probability value of j = 1 (corresponding to category y1) is 0.24; the probability value of j = 2 (corresponding to category y2) is 0.76; when the argmax function determines that j = 2, f(X t ) takes the maximum value of 0.76 (that is, the probability of anomaly is 0.76), and the result of network status detection is obtained, and its corresponding category is anomaly.

[0104] The embodiment of the present invention uses a small amount of labeled data and a large amount of unlabeled data sets, combined with the Tri-training algorithm, to pre-train three base classifiers, and then expands the training sample set of the third base classifier by using two of the base classifiers to generate pseudo-labeled samples, and then improves its performance by training the third base classifier; finally, the network fault state is determined by weighted voting after training multiple base classifiers, so that the performance is better and the detection result is more accurate; in addition, only a small amount of labeled data is used, and a semi-supervised learning method is used to reduce the cost of annotating data, and the applicable scenarios are wider; in the embodiment of the present invention, the imbalanced data distribution is processed by the loss function, and the parameters in the loss function are improved, so that the model pays less attention to outliers, thereby improving the model performance.

[0105] Correspondingly, an embodiment of the present invention provides a network fault detection device based on semi-supervised learning, which can implement all processes of the network fault detection method based on semi-supervised learning provided by any of the above embodiments. The functions of each module and unit in the device and the technical effects achieved are respectively the same as the functions and technical effects achieved by the network fault detection method based on semi-supervised learning provided by the above embodiments, and will not be repeated here.

[0106] See also Figure 2 , is a structural diagram of an embodiment of a network fault detection device based on semi-supervised learning provided by the present invention.

[0107] An embodiment of the present invention provides a network fault detection device based on semi-supervised learning, comprising:

[0108] The data acquisition module 1 is used to collect a network status data set from a network environment;

[0109] A data preprocessing module 2 is used to preprocess the network status data set; wherein the data set containing labels in the network status data set is recorded as a real label data set, and the remaining data is recorded as an unlabeled data set;

[0110] Pre-training module 3, used to train the three pre-trained base classifiers using the Tri-training algorithm to obtain a trained network fault model;

[0111] A weight calculation module 4 is used to assign weights to the three base classifiers in the network fault model respectively;

[0112] The detection module 5 is used to input the network status data to be detected into the trained network fault model to obtain the detection result of the network status.

[0113] Correspondingly, an embodiment of the present invention also provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, any one of the above-mentioned network fault detection methods based on semi-supervised learning is implemented.

[0114] In addition, an embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned network fault detection methods based on semi-supervised learning.

[0115] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary hardware platform, and of course can also be implemented entirely by hardware. Based on such an understanding, all or part of the contribution of the technical solution of the present invention to the background technology can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.

[0116] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A network fault detection method based on semi-supervised learning, characterized in that: include: Collect network status datasets from the network environment; Preprocessing the network status data set; wherein the data set containing labels in the network status data set is recorded as a real label data set, and the remaining data is recorded as an unlabeled data set; Use the Tri-training algorithm to train the three pre-trained base classifiers to obtain a trained network fault model; Assigning weights to three base classifiers in the network fault model respectively; Input the network status data to be detected into the trained network fault model to obtain the detection result of the network status; The three pre-trained base classifiers are trained using the Tri-training algorithm to obtain a trained network fault model, which specifically includes: The real label data set is sampled to obtain three different real label training data sets, denoted as L i , i=1,2,3; Use three real label training data sets to pre-train three base classifiers until the base classifier models converge. The three base classifiers are denoted as h i , i=1,2,3; The three pre-trained base classifiers are trained using the Tri-training algorithm until the weights of the base classifiers in the network fault model are no longer updated or the number of training times reaches the set threshold, and a trained network fault model is obtained. Each training includes: Use the three pre-trained base classifiers to annotate the unlabeled data set; generate an expanded sample set S for each base classifier i , i=1,2,3; using the expanded sample set to train the three base classifiers again respectively; wherein the expanded sample set is updated after each training; The three base classifiers are pre-trained using three real label training data sets until the base classifier models converge, where the three base classifiers are denoted as h i , i=1,2,3, specifically including: Input the training samples into the base classifier, where the training samples belong to the real label training data set; The forward propagation outputs the predicted value and the loss value is calculated using the improved loss function; Back propagation updates the weights of the base classifier until the base classifier model converges; The improved loss function is specifically: Loss Function Where y is the true label of the data, y′ is the label output by the model, α is the weight parameter, and γ is the focus parameter, where γ is calculated based on each type of label data set, specifically: Calculate C i The sample center point μ i : Among them, the training sample x∈C i , C i is the i-th label data set, i=1,2,3; Calculate the training sample x to the sample center point μ i The cosine similarity of: Calculate the γ parameter: γ = 1-cos(x,μ i ).

2. A network fault detection method based on semi-supervised learning as claimed in claim 1, characterized in that: The unlabeled data set is annotated using the three pre-trained base classifiers; an expanded sample set S of each base classifier is generated. i , i=1,2,3; The three base classifiers are trained again using the expanded sample set, specifically including: Use the three trained base classifiers to annotate the unlabeled dataset; when When , and two different base classifiers have the same label for an unlabeled sample, the unlabeled sample is labeled and added to the pseudo-label training data set of the third classifier; represents the error rate of the ith base classifier in the tth iteration; Generate the pseudo-label training dataset L for the tth iteration t i , i=1,2,3; when When, from L t i Randomly delete several samples; where |L t i |Represents L t i The number of samples; Update the expanded sample set S of each base classifier i , S i =L i ∪L t i ; The three base classifiers are trained again respectively using the expanded sample set.

3. A network fault detection method based on semi-supervised learning as claimed in claim 1, characterized in that: The weights are assigned to the base classifiers in the network fault model respectively, specifically including: Calculate the base classifier h i Micro-F1 score F i , calculate h i Weight 4. A network fault detection method based on semi-supervised learning as claimed in claim 3, characterized in that: The network status data to be detected is input into the trained network fault model to obtain the detection result of the network status, which specifically includes: After the network status data to be detected is input into the three base classifiers respectively, the detection result Y is obtained by weighted voting. t ′ : Among them, i represents the i-th base classifier, j represents the j-th network state category label, X t is the N-dimensional eigenvalue of the network at sample time t that is input into the network fault model.

5. A network fault detection device based on semi-supervised learning, characterized in that: include: A data acquisition module, used for collecting network status data sets from the network environment; A data preprocessing module, used to preprocess the network status data set; wherein the data set containing labels in the network status data set is recorded as a real label data set, and the remaining data is recorded as an unlabeled data set; The pre-training module is used to train the three pre-trained base classifiers using the Tri-training algorithm to obtain a trained network fault model; A weight calculation module, used to assign weights to the three base classifiers in the network fault model respectively; The detection module is used to input the network status data to be detected into the trained network fault model to obtain the detection result of the network status; The pre-training module specifically includes: The real label data set is sampled to obtain three different real label training data sets, denoted as L i , i=1,2,3; Use three real label training data sets to pre-train three base classifiers until the base classifier models converge. The three base classifiers are denoted as h i , i=1,2,3; The three pre-trained base classifiers are trained using the Tri-training algorithm until the weights of the base classifiers in the network fault model are no longer updated or the number of training times reaches the set threshold, and a trained network fault model is obtained. Each training includes: Use the three pre-trained base classifiers to annotate the unlabeled data set; generate an expanded sample set S for each base classifier i , i=1,2,3; using the expanded sample set to train the three base classifiers again respectively; wherein the expanded sample set is updated after each training; The three base classifiers are pre-trained using three real label training data sets until the base classifier models converge, where the three base classifiers are denoted as h i , i=1,2,3, specifically including: Input the training samples into the base classifier, where the training samples belong to the real label training data set; The forward propagation outputs the predicted value and the loss value is calculated using the improved loss function; Back propagation updates the weights of the base classifier until the base classifier model converges; The improved loss function is specifically: Loss Function Where y is the true label of the data, y′ is the label output by the model, α is the weight parameter, and γ is the focus parameter, where γ is calculated based on each type of label data set, specifically: Calculate C i The sample center point μ i : Among them, the training sample x∈C i , C i is the i-th label data set, i=1,2,3; Calculate the training sample x to the sample center point μ i The cosine similarity of: Calculate the γ parameter: γ = 1-cos(x,μ i ).

6. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a network fault detection method based on semi-supervised learning as claimed in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute a network fault detection method based on semi-supervised learning as described in any one of claims 1 to 4.

Citation Information

Patent Citations

  • Network fault diagnosis method based on deep learning in virtual network environment

    CN106603293A

  • Semi-supervised synergistic evaluation method for static parameter of health monitoring of bridge structure

    CN102663264A

  • Network anomaly detection method and device, electronic equipment and storage medium

    CN113554094A