Network anomaly detection method and device based on double-branch model

Abstract

The invention discloses a network anomaly detection method and device based on a double-branch model, and relates to wireless communication, artificial intelligence and big data analysis. The method comprises the following steps: firstly, acquiring performance time sequence data from a communication terminal to perform data preprocessing, including oversampling, undersampling and feature selection; secondly, constructing an anomaly detection network for offline training, inputting historical data into a contrast learning branch and an attention mechanism branch, learning data structure internal association, prior association and sequence association, calculating contrast loss and reconstruction loss, and obtaining the anomaly detection network through joint optimization of a multi-task loss function; in the online detection stage, the anomaly detection network obtained through training in the offline stage is used for network anomaly detection and periodic feedback, network anomaly moment data is collected to a historical database, and anomaly detection network parameters are periodically updated.

Description

Technical Field

[0001] The present invention relates to the technical fields of wireless communications, artificial intelligence and big data analysis, and in particular to a network anomaly detection method and device based on a dual-branch model. Background Art

[0002] With the rapid development of 5G technology and the Internet, the scale of information and communication networks is becoming increasingly large, and the number of access terminal devices is increasing rapidly. How to ensure the long-term stable operation of the network has become an urgent problem that needs to be solved. As a result, Artificial Intelligence for IT Operations (AIOps) has become a development trend and research hotspot at home and abroad, and many companies are actively exploring and implementing AIOps. At the same time, national policies are also promoting the digital and intelligent transformation of industries, promoting the development of the digital economy, and providing strong support for the development of the intelligent operation and maintenance industry. AIOps includes functions such as network anomaly detection, root cause analysis of faults, and fault warning. Among them, anomaly detection is an important part of AIOps. The network management system is constantly collecting key performance indicator (KPI) data from each terminal. By analyzing the KPI data, operation and maintenance personnel can judge the operating status of the network. Therefore, how to ensure timely and accurate detection of network anomalies from massive KPI data has become a key challenge for network operation and maintenance.

[0003] Anomaly detection methods based on machine learning are now widely used. Among them, a large category of methods use supervised learning to train models, which has a certain degree of accuracy, but has high requirements for data samples, over-reliance on data labels, and also has high training and detection overhead, making it difficult to apply to large-scale Internet services. Another large category of methods overcomes the problem of lack of labeled data through unsupervised tasks, learns information representation from complex time dynamics, saves annotation costs, and the learned feature representation usually has good versatility and generalization capabilities. Faced with the problem of high cost of obtaining data labels and difficult operation and maintenance in the era of big data, many anomaly detection methods based on unsupervised deep learning have been proposed. However, existing unsupervised deep learning methods have difficulty in extracting useful features from complex network data, cannot take into account the correlation between various dimensions of data, and have relatively poor interpretability. Summary of the invention

[0004] The first purpose of the present invention is to utilize the advantages of artificial intelligence technology to solve the above-mentioned time series anomaly detection technical problems by constructing a dual-branch anomaly detection network through contrast learning and self-attention mechanism, and to provide a network anomaly detection method based on a dual-branch model, which can detect network anomalies in a timely and accurate manner, provide guarantees for operation and maintenance work, and improve network operation and maintenance efficiency and user experience.

[0005] The second object of the present invention is to provide a network anomaly detection device based on a dual-branch model.

[0006] To achieve the above objectives, the present invention provides the following technical solutions.

[0007] A network anomaly detection method based on dual-branch time series, comprising the following steps:

[0008] 1) Data preprocessing: Obtain KPI data from the communication terminal for data preprocessing, including oversampling, undersampling, and feature selection;

[0009] 2) Construct and optimize the anomaly detection network: Input the preprocessed data in step 1) into the offline training module to construct the anomaly detection network and perform offline training. Input the historical data into the contrast learning branch and the attention mechanism branch respectively, learn the internal association, prior association and sequence association of the data structure, and calculate the contrast loss and reconstruction loss. By jointly optimizing the multi-task loss function, calculate the anomaly score, and obtain the optimized anomaly detection network;

[0010] 3) Online detection and feedback: In the online detection phase, the anomaly detection network obtained in step 2) is used to perform real-time network anomaly detection. The detection results are fed back periodically, and the anomaly data is collected into a historical database. Based on the periodic feedback data, the anomaly detection network parameters are regularly updated to optimize the detection results.

[0011] In step 1), the KPI data includes various network monitoring data, such as RRC connection establishment success rate, cell RLC layer downlink packet loss rate, same-frequency handover execution success rate, wireless disconnection rate, etc.

[0012] In step 1), the data preprocessing step is used to solve the data imbalance problem caused by the rarity of anomalies. Specifically, it includes: first, using oversampling technology, such as synthetic minority oversampling technique (SMOTE), inserting existing anomaly class samples to create new synthetic anomaly class samples; then using clustering algorithms, such as K-Means, hierarchical clustering, etc. to cluster the data, and after clustering, select the cluster containing the largest number of samples for undersampling to balance the number of abnormal and normal samples; finally, feature selection is performed, by calculating the dependency of the class label on each feature, and removing the feature with the smallest dependency. Here, the commonly used feature selection technology mutual information is used to select features that are more relevant to the output class. Feature selection based on mutual information includes calculating the mutual information between each feature and the output class, selecting features with high mutual information scores, and discarding features with low mutual information scores, thereby helping to determine all features that have the least or no effect in classification and discarding them to improve the accuracy of the model, reduce overfitting and reduce computational costs.

[0013] In step 2), the specific steps of constructing the anomaly detection network include: inputting historical data into the contrast learning branch and the attention mechanism branch respectively, learning the internal association of the data structure through the contrast learning branch, and calculating the contrast loss; learning the data prior association and sequence association through the attention mechanism branch, and calculating the reconstruction loss; then jointly optimizing the multi-task loss function, calculating the anomaly score, and obtaining the anomaly detection network.

[0014] In step 2), the preprocessed data is input into the contrastive learning branch and the attention mechanism branch respectively, including: the contrastive learning branch contains two sub-networks with the same structure, both of which use multi-layer perceptrons to map input samples to low-dimensional embedding. Define the time series data measured from a single device as X = {x1, x2, ..., x T}, where x i ∈R d Represents the observation value at time t, and d is the time series dimension. The original sequence X is copied multiple times, and the multivariate time series data at each moment is sharded at different positions (the shard size k is determined according to the number of indicators, and divided into two parts a and b) and combined, that is, data reorganization, and the number of combinations is m = d-k+1. and When these two complementary combinations are a pair of positive examples, the rest of the combinations are negative examples. Represents the data reorganization of input X. Through contrastive learning, a feature space is learned to maximize the similarity between positive samples and minimize the similarity between negative samples, thereby learning the association within the data structure. Contrastive loss L c It is calculated by the following cross entropy loss function:

[0015]

[0016] in, represents the shard data at the jth index position at the ith time point, m is the number of combinations after the shards are reorganized, exp() refers to the exponential function with the natural constant e as the base, ln() refers to the logarithmic function with e as the base, M(·) and N(·) represent two sub-networks for contrastive learning, and the hyperparameter τ is the temperature constant of the contrastive loss.

[0017] In step 2), the preprocessed data is input into the contrastive learning branch and the attention mechanism branch respectively, including: since the single-branch self-attention mechanism cannot model the prior association and the sequence association at the same time, a multi-layer self-attention mechanism with a dual-branch structure is used. For the prior association, a learnable Gaussian kernel is used to calculate the prior about the relative time distance. A learnable scale parameter σ is also used for the Gaussian kernel so that the prior association can adapt to different time series patterns.

[0018] Initialize the attention mechanism parameters query Q (Query), key K (Key), value V (Value), σ, where σ represents the learned scale, generates a priori associations based on the learned scale, and the i-th element corresponds to the i-th time point.

[0019] Specifically, for the i-th time point, its association weight with the j-th time point is calculated by the Gaussian kernel:

[0020]

[0021] in, Represent the parameter matrices of Q, K, V, and σ in the lth layer respectively, and then use rescaling to transform the association weight into a discrete distribution, which is defined as the prior association:

[0022]

[0023] The attention map is normalized along the last dimension using the Softmax function. Each row of the Series forms a discrete distribution. The Series Association is defined as:

[0024]

[0025] Through the self-attention mechanism, the association bias is defined as the symmetric KL divergence between the prior association and the sequence association. The association bias represents the information gain between these two distributions. Define the reconstruction loss L of the self-attention branchrec Weighting for reconstruction differences and associated deviations:

[0026]

[0027] in, represents the reconstruction of the sequence x, ||·|| F and ||·|| K denote the Frobenius and K norms respectively; L denotes the number of attention layers; KL(·||·) denotes the KL divergence calculated between two discrete distributions; A p and A s They represent prior association and serial association respectively; θ is used to weigh the loss term, and when θ>0, the association deviation is amplified.

[0028] In step 2), the joint optimization of the multi-task loss function specifically includes: defining the joint loss Loss as:

[0029] Loss = λ × L rec +(1-λ)L c

[0030] It is used to balance the joint training process of the self-attention network and the contrastive learning network to avoid a certain loss being too large and having a dominant influence. In the formula, λ represents the weight parameter; L rec represents the reconstruction loss of the attention branch; L c The dual-branch anomaly detection network designed by the contrastive loss representing the contrastive learning branch is trained by minimizing the loss function Loss. Among them, the weight parameter λ needs to be adjusted according to different data sets. In order to use the hyperparameter optimization framework (such as Hyperopt, Optuna, etc.) for weight parameter optimization, the hyperparameter optimization framework provides a wealth of optimization algorithms and search strategies, supports parallel evaluation, early stopping, result visualization and other functions, and can efficiently search for the optimal hyperparameter combination.

[0031] Optionally, first define the hyperparameter space, that is, the range of values ​​that each hyperparameter can take or the list of optional values. Here, we define the continuous space Λ of (0,1) = {λ1,…,λ n Initialize the Study object (in frameworks such as Optuna) to store and manage all information in the optimization process, including historical experiments, optimal parameters, etc.

[0032] We then define the objective function, a set of hyperparameters as input, and return a model’s F1 score on the validation set:

[0033] f(Λ)→F1

[0034] Among them, f is the objective function and Λ is the hyperparameter configuration.

[0035] Then, an optimization algorithm, such as random search, Bayesian optimization, etc., is run to search for the optimal solution in the hyperparameter space. In each iteration, the optimization algorithm generates a new hyperparameter configuration and passes it to the objective function for evaluation.

[0036] Λ t =optimize_algorithm(X,f,t)

[0037] Among them, Λ t is the hyperparameter configuration in the tth iteration, and optimize_algorithm is the optimization algorithm. For each newly generated hyperparameter configuration, the framework runs the objective function for evaluation. This usually involves training a machine learning model with the given hyperparameter configuration and evaluating its performance on the validation set. The results of each iteration (including hyperparameter configuration, score, etc.) are stored in the Study object. The framework uses this information to guide the subsequent search process.

[0038] It should be noted that the operation and maintenance personnel need to set the iteration termination conditions, for example, terminating the iteration if there is no obvious improvement in performance in n consecutive iterations.

[0039] After the optimization process is completed, the best hyperparameter configuration λ can be obtained from the Study object.

[0040] In step 2), calculating the anomaly score specifically includes: designing an anomaly score as an anomaly standard. Normal data points have a strong correlation, so the distance between the positive example pairs obtained by contrastive learning is smaller than the negative example pairs. The anomaly score is defined as follows:

[0041]

[0042] Among them, Softmax refers to the normalized exponential function, and ⊙ represents element-by-element multiplication; Representation sequence Reconstruction of ||·|| F represents the Frobenius norm; L represents the number of attention layers; KL(·||·) represents the KL divergence calculated between two discrete distributions; A p and A s represent prior association and serial association respectively; L c Represents contrast loss, i.e., cross entropy loss function. Based on the anomaly score, a hyperparameter threshold δ is used to determine whether a point is anomaly (y=1) or non-anomaly (y=0); if the score exceeds the threshold, the output y is anomaly, i.e.:

[0043]

[0044] Among them, y irepresents the model output at the i-th time point, represents the model input at the i-th time point, It represents the anomaly score at the i-th time point, and the hyperparameter threshold δ is set by the operation and maintenance personnel according to different scenario data.

[0045] In step 3), the online detection phase performs real-time detection and periodic feedback, specifically including: network KPI data is input into the trained anomaly detection network in real time, an anomaly alarm is output when a network anomaly is detected, and the operation and maintenance personnel determine whether the detected network anomaly is a real anomaly. If it is a real network anomaly, the KPI data at this moment is collected into the historical database, and the anomaly detection network parameters are updated periodically.

[0046] The network anomaly detection device based on dual-branch time series is provided with a data acquisition module, an offline training module, a real-time detection module, and a feedback module in sequence;

[0047] The data acquisition module is used to acquire communication terminal performance time series data and model feedback data;

[0048] The offline training module is used to build an anomaly detection model through the historical data of the data acquisition module and periodically update the anomaly detection model;

[0049] The real-time detection module is used to detect anomalies online;

[0050] The feedback module collects abnormal data detected by the real-time detection module, and inputs the abnormal data into the data acquisition module to expand the database after the operation and maintenance personnel confirm the abnormality.

[0051] Compared with the prior art, the present invention has the following outstanding technical effects and advantages:

[0052] The present invention takes advantage of artificial intelligence technology and proposes a network anomaly detection method and device based on a dual-branch model for the network anomaly detection problem that the industry focuses on. The core innovation of the present invention lies in its unique dual-branch structure design, which not only allows the model to simultaneously learn the internal correlation of KPI data and the correlation between sequences, but also significantly improves the efficiency of anomaly detection through parallel processing. The dual-branch architecture enables the model to more comprehensively capture the abnormal patterns reflected by the data, and does not require complex learning strategies, weakening the dependence on manual work, thereby showing higher reliability and stability in practical applications. In the dual-branch structure, the present invention creatively combines the advantages of self-learning ability and strong robustness of contrastive learning, and the characteristics of self-attention mechanism to capture remote dependency information and strong generalization, improves the accuracy of anomaly detection, and provides a solid foundation for network operation and maintenance. In addition, the present invention also introduces data enhancement technology based on contrastive learning and a hyperparameter joint optimization framework to effectively improve the generalization ability of the model, so that the model can adapt well to the new network structure without too much modification in the continuous evolution of LTE mobile networks. The present invention can realize rapid and accurate identification of network anomalies, provide strong guarantee for subsequent operation and maintenance work, improve network optimization efficiency, and enhance network performance, user service quality and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flowchart of network anomaly detection based on dual-branch time series;

[0054] Figure 2 It is a network diagram of a network anomaly detection model based on dual-branch time series;

[0055] Figure 3 A schematic diagram of the contrastive learning principle in an anomaly detection model network based on a dual-branch time series;

[0056] Figure 4 A schematic diagram of a network anomaly detection method based on dual-branch time series;

[0057] Figure 5 A diagram of a network anomaly detection device based on dual-branch time series. DETAILED DESCRIPTION

[0058] The invention content of the present invention is described in detail below, and examples of the invention content are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0059] An inventive content of the present invention provides a network anomaly detection method based on a dual-branch time series, comprising: first obtaining performance data from a communication terminal for data preprocessing, including oversampling, undersampling and feature selection; then constructing an anomaly detection network for offline training, inputting historical data into a contrast learning branch and an attention mechanism branch respectively, learning the internal association, prior association and sequence association of the data structure and calculating the contrast loss and reconstruction loss, and obtaining the anomaly detection network by jointly optimizing the multi-task loss function; in the online detection stage, using the anomaly detection network obtained by offline stage training for detection and periodic feedback, collecting abnormal data into a historical database, and periodically updating the anomaly detection network parameters.

[0060] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific embodiments of the specification.

[0061] Figure 1 The present invention is a flowchart of an embodiment of a network anomaly detection method based on a dual-branch time series, which includes the following steps:

[0062] S101, obtain network performance time series data from the communication terminal, and the performance time series data includes various network monitoring data, such as RRC connection establishment success rate in LTE network, cell RLC layer downlink packet loss rate, same-frequency handover execution success rate, wireless drop rate, etc. Perform data preprocessing, first use oversampling technology, such as synthetic minority oversampling technology SMOTE to insert existing abnormal class samples to create new synthetic abnormal class samples; then use clustering algorithms, such as K-Means, hierarchical clustering, etc. to cluster the data, and after clustering, select the cluster containing the largest number of samples for undersampling to balance the number of abnormal and normal samples; finally, perform feature selection, by calculating the dependency of the class label on each feature, remove the feature with the smallest dependency. Here, the commonly used feature selection technology mutual information is used to select features that are more relevant to the output class. Feature selection based on mutual information includes calculating the mutual information between each feature and the output class, selecting features with high mutual information scores, and discarding features with low mutual information scores, thereby helping to determine all features that have the smallest or no effect in classification and discarding them, so as to improve the accuracy of the model, reduce overfitting and reduce computational costs.

[0063] S102, in the offline training phase, an anomaly detection network is constructed, historical data is input into the contrastive learning branch and the attention mechanism branch respectively, the contrastive loss and the reconstruction loss are calculated, and the final anomaly detection network is obtained by jointly optimizing the multi-task loss function for offline training.

[0064] Specifically, the historical data is input into the contrastive learning branch and the attention mechanism branch respectively, the internal correlation of the data structure is learned through the contrastive learning branch, and the contrast loss is calculated; the data prior correlation and sequence correlation are learned through the attention mechanism branch, and the reconstruction loss is calculated; and then the multi-task loss function is jointly optimized to obtain the anomaly detection network.

[0065] The step of inputting the preprocessed data into the contrastive learning branch specifically includes: the contrastive learning branch includes two sub-networks of the same structure, both of which use multi-layer perceptrons to map the input samples to low-dimensional embeddings. Define the time series data measured from a single device as X = {x1, x2, ..., x T}, where x i ∈R d Represents the observation value at time t, and d is the time series dimension. The original sequence X is copied multiple times, and the multivariate time series data at each moment is sharded at different positions (the shard size k is determined according to the number of indicators, and divided into two parts a and b) and combined, that is, data reorganization, and the number of combinations is m = d-k+1. and When these two complementary combinations are a pair of positive examples, the rest of the combinations are negative examples. Represents the data reorganization of input X. Through contrastive learning, a feature space is learned to maximize the similarity between positive samples and minimize the similarity between negative samples, thereby learning the association within the data structure. Contrastive loss L c It is calculated by the following cross entropy loss function:

[0066]

[0067] in, represents the shard data at the jth index position at the i-th time point, m is the number of combinations after the shards are reorganized, exp() refers to the exponential function with the natural constant e as the base, ln() refers to the logarithmic function with e as the base, M(·) and N(·) represent two sub-networks of contrastive learning, and the hyperparameter τ is the temperature constant of the contrastive loss. The attention mechanism branch for the preprocessed data specifically includes: since the single-branch self-attention mechanism cannot model the prior association and the sequence association at the same time, a multi-layer self-attention mechanism with a dual-branch structure is used. For the prior association, a learnable Gaussian kernel is used to calculate the prior about the relative time distance. A learnable scale parameter σ is also used for the Gaussian kernel so that the prior association can adapt to different time series patterns.

[0068] Initialize the attention mechanism parameters query Q (Query), key K (Key), value V (Value), σ, where σ represents the learned scale, and generate prior associations based on the learned scale. The i-th element corresponds to the i-th time point.

[0069] Specifically, for the i-th time point, its association weight with the j-th time point is calculated by the Gaussian kernel:

[0070]

[0071] in, Represent the parameter matrices of Q, K, V, and σ in the lth layer respectively, and then use rescaling to transform the association weight into a discrete distribution, which is defined as the prior association:

[0072]

[0073] The attention map is normalized along the last dimension using the Softmax function. Each row of the Series forms a discrete distribution. The Series Association is defined as:

[0074]

[0075] Through the self-attention mechanism, the association bias is defined as the symmetric KL divergence between the prior association and the sequence association, which represents the information gain between the two distributions. The reconstruction loss of the self-attention branch is defined as the weighted sum of the reconstruction difference and the association bias:

[0076]

[0077] in, Representation sequence Reconstruction of ||·|| F and ||·|| K denote the Frobenius and K norms respectively; L denotes the number of attention layers; KL(·||·) denotes the KL divergence calculated between two discrete distributions; A p and A s They represent prior association and serial association respectively; θ is used to weigh the loss term, and when θ>0, the association deviation is amplified.

[0078] The joint loss Loss is defined as:

[0079] Loss = λ × L rec +(1-λ)L c

[0080] It is used to balance the joint training process of the self-attention network and the contrastive learning network to avoid a dominant influence caused by a certain loss being too large. The designed dual-branch anomaly detection network is trained by minimizing the loss function L. Among them, the weight parameter λ needs to be adjusted according to different data sets. In order to use the hyperparameter optimization framework (such as Hyperopt, Optuna, etc.) to optimize the weight parameters, the hyperparameter optimization framework provides a wealth of optimization algorithms and search strategies, supports parallel evaluation, early stopping, result visualization and other functions, and can efficiently search for the optimal hyperparameter combination.

[0081] Optionally, first define the hyperparameter space, that is, the range of values ​​that each hyperparameter can take or the list of optional values. Here, we define the continuous space Λ of (0,1) = {λ1,…,λ n Initialize the Study object (in frameworks such as Optuna) to store and manage all information in the optimization process, including historical experiments, optimal parameters, etc.

[0082] We then define the objective function, a set of hyperparameters as input, and return a model’s F1 score on the validation set:

[0083] f(Λ)→F1

[0084] Among them, f is the objective function and Λ is the hyperparameter configuration.

[0085] Then, an optimization algorithm, such as random search, Bayesian optimization, etc., is run to search for the optimal solution in the hyperparameter space. In each iteration, the optimization algorithm generates a new hyperparameter configuration and passes it to the objective function for evaluation.

[0086] Λ t =optimize_algorithm(Λ,f,t)

[0087] Among them, Λ t is the hyperparameter configuration in the tth iteration, and optimize_algorithm is the optimization algorithm. For each newly generated hyperparameter configuration, the framework runs the objective function for evaluation. This usually involves training a machine learning model with the given hyperparameter configuration and evaluating its performance on the validation set. The results of each iteration (including hyperparameter configuration, score, etc.) are stored in the Study object. The framework uses this information to guide the subsequent search process.

[0088] The operation and maintenance personnel need to set the iteration termination conditions, for example, terminating the iteration if there is no obvious improvement in performance in n consecutive iterations.

[0089] After the optimization process is completed, the best hyperparameter configuration λ can be obtained from the Study object.

[0090] After calculating the joint loss, an anomaly score is designed as the anomaly standard. Normal data points have a strong correlation, so the distance between the positive pairs of the representation results obtained by contrastive learning is smaller than that between the negative pairs. The anomaly score is defined as follows:

[0091]

[0092] Among them, Softmax refers to the normalized exponential function, and ⊙ represents element-by-element multiplication; represents the reconstruction of the sequence x, ||·|| F represents the Frobenius norm; L represents the number of attention layers; KL(·||·) represents the KL divergence calculated between two discrete distributions; A p and A s represent prior association and serial association respectively; L c represents the contrast loss, that is, the cross entropy loss function.

[0093] Based on the anomaly score, a hyperparameter threshold δ is used to decide whether a point is anomaly (y=1) or non-anomalous (y=0); if the score exceeds the threshold, the output y is anomaly, that is:

[0094]

[0095] Among them, y i represents the model output at the i-th time point, represents the model input at the i-th time point, Indicates the anomaly score at the i-th time point, and the hyperparameter threshold δ is set by the operation and maintenance personnel according to different scenario data. S103, in the online stage, the time series data of the performance to be tested is obtained, input into the online detection module, and real-time anomaly detection is performed through the anomaly detection model obtained in the offline stage.

[0096] S104, after real-time anomaly detection, the detected anomaly data is accumulated into the historical database, and a periodic model update is performed. Specifically, the operation and maintenance personnel determine whether the detected anomaly is a real anomaly, and collect it into the historical database, and periodically update the anomaly detection network parameters.

[0097] like Figure 2 As shown, an embodiment of the present invention provides a dual-branch network anomaly detection model including an attention network and a contrastive learning network:

[0098] Specifically, two encoders with the same structure but different dimensions are used in the contrastive learning network. The time series data measured from a single device is defined as X = {x1, x2, ..., x T}, where x i ∈R dRepresents the observation value at time t, and d is the time series dimension. The original sequence X is copied multiple times, and the multivariate time series data at each moment is sharded at different positions (the shard size k is determined according to the number of indicators, and divided into two parts a and b) and combined. The number of combinations is m = d-k+1. and When these two complementary combinations are a positive pair, the rest of the combinations are negative pairs. Represents the data reorganization of input X. Through contrastive learning, a feature space is learned to maximize the similarity between positive samples and minimize the similarity between negative samples, thereby learning the association within the data structure. The contrast loss is calculated by the following cross entropy loss function:

[0099]

[0100] in, represents the shard data at the jth index position at the ith time point, m is the number of combinations after the shards are reorganized, exp() refers to the exponential function with the natural constant e as the base, ln() refers to the logarithmic function with e as the base, M(·) and N(·) represent two sub-networks for contrastive learning, and the hyperparameter τ is the temperature constant of the contrastive loss.

[0101] Specifically, in the attention network, since the single-branch self-attention mechanism cannot model both the prior association and the sequence association at the same time, a multi-layer self-attention mechanism with a dual-branch structure is used. For the prior association, a learnable Gaussian kernel is used to calculate the prior about the relative time distance. A learnable scale parameter σ is also used for the Gaussian kernel so that the prior association can adapt to different time series patterns.

[0102] Initialize the attention mechanism parameters query Q (Query), key K (Key), value V (Value), σ, where σ represents the learned scale, and generate prior associations based on the learned scale. The i-th element corresponds to the i-th time point.

[0103] Specifically, for the i-th time point, its association weight with the j-th time point is calculated by the Gaussian kernel:

[0104]

[0105] in, Represent the parameter matrices of Q, K, V, and σ in the lth layer respectively, and then use rescaling to transform the association weight into a discrete distribution, which is defined as the prior association:

[0106]

[0107] The attention map is normalized along the last dimension using the Softmax function. Each row of the Series forms a discrete distribution. The Series Association is defined as:

[0108]

[0109] Through the self-attention mechanism, the association bias is defined as the symmetric KL divergence between the prior association and the sequence association, which represents the information gain between the two distributions. The reconstruction loss L of the self-attention branch is defined as rec Weighting for reconstruction differences and associated deviations:

[0110]

[0111] in, represents the reconstruction of the sequence x, ||·|| F and ||·|| K denote the Frobenius and K norms respectively; L denotes the number of attention layers; KL(·||·) denotes the KL divergence calculated between two discrete distributions; A p and A s They represent prior association and serial association respectively; θ is used to weigh the loss term, and when θ>0, the association deviation is amplified.

[0112] The joint loss Loss is defined as:

[0113] Loss = λ × L rec +(1-λ)L c

[0114] In the formula, λ represents the weight parameter; L rec represents the reconstruction loss of the attention branch; L c represents the contrast loss of the contrastive learning branch; the joint optimization of the multi-task loss function Loss is used to balance the joint training process of the self-attention network and the contrastive learning network to avoid a certain loss being too large and having a dominant influence; the designed dual-branch anomaly detection network is trained by minimizing the loss function Loss; among them, the weight parameter λ needs to be adjusted according to different data sets.

[0115] like Figure 3 The figure shows a schematic diagram of the comparative learning principle in network anomaly detection based on a dual-branch time series provided in the present invention:

[0116] Specifically, in order to learn the internal correlation of multidimensional time series data x, the idea of ​​contrastive learning is used to copy the original sequence x multiple times, and the multivariate time series data at each moment is fragmented at different positions (the fragment size k is determined according to the number of indicators, and is divided into two parts a and b) and combined. The number of combinations is m = d-k+1. As can be seen in the figure, and When , these two complementary combinations are a pair of positive pairs, and the rest of the combinations are negative pairs. In the feature space learned by contrastive learning, the distance between positive pairs is pulled closer, and the distance between negative pairs is pushed further away. In practical applications, there is a correlation between performance indicators, and different dimensions of time series are different indicators. For example, in access problems, the wireless connection rate is related to the success rate of E-RAB establishment. After segmentation and reorganization, the wireless connection rate and other indicator data are spliced ​​together, which does not conform to the original correlation. Through contrastive learning networks, the distance between negative pairs in the feature space will be expanded, so the internal correlation of the data is learned.

[0117] like Figure 4 The figure shows a schematic diagram of a network anomaly detection method based on a dual-branch time series provided in the present invention: first, the performance time series data is obtained from the communication terminal for data preprocessing, including oversampling, undersampling and feature selection. Then, an anomaly detection network is constructed for offline training, and historical data is input into the contrast learning branch and the attention mechanism branch respectively, and the internal association, prior association and sequence association of the learning data structure are calculated, and the contrast loss and reconstruction loss are calculated. By jointly optimizing the multi-task loss function, the anomaly detection network is obtained; in the online detection stage, the anomaly detection network obtained by the offline stage training is used for detection and periodic feedback, and the abnormal data is collected into the historical database, and the anomaly detection network parameters are periodically updated.

[0118] The embodiment of the present invention provides a mobile switching performance optimization device based on a wireless access network intelligent controller, such as Figure 5 As shown, the device includes: a data acquisition module, an offline training module, a real-time detection module, and a feedback module;

[0119] The data acquisition module is used to acquire communication terminal performance time series data and model feedback data;

[0120] The offline training module is used to build an anomaly detection model through the historical data of the data acquisition module and periodically update the anomaly detection model;

[0121] The real-time detection module is used to detect anomalies online;

[0122] The feedback module collects abnormal data detected by the real-time detection module, and inputs the abnormal data into the data acquisition module to expand the database after the operation and maintenance personnel confirm the abnormality.

[0123] In summary, a network anomaly detection method based on a dual-branch time series provided in accordance with an embodiment of the present invention includes the following steps: first, obtaining performance time series data from a communication terminal for data preprocessing, including oversampling, undersampling, and feature selection; then constructing an anomaly detection network for offline training, inputting historical data into a contrast learning branch and an attention mechanism branch respectively, learning the internal association, prior association, and sequence association of the data structure, and calculating the contrast loss and reconstruction loss, and obtaining an anomaly detection network by jointly optimizing a multi-task loss function; in the online detection phase, the anomaly detection network obtained by offline phase training is used for detection and periodic feedback, the anomaly data is collected into a historical database, and the anomaly detection network parameters are periodically updated. The present invention can realize rapid and accurate identification of network anomalies, provide strong guarantees for subsequent operation and maintenance work, improve network optimization efficiency, and enhance network performance, user service quality, and user experience. The proposed invention example can improve the efficiency of anomaly detection in traditional operation and maintenance, and use intelligent means to analyze a large amount of data in the network. Compared with the existing methods, it takes into account the time dependency of the data and the internal structure of the data, and extracts the characteristics of the data more comprehensively and reasonably. The network anomaly detection accuracy, precision, recall rate, F1 score and other indicators can be improved to a certain extent, and the applicable scenarios are more extensive, indicating that the proposed method has a stronger ability to distinguish network anomalies, and has a good anomaly detection ability for unbalanced KPI data. It helps operation and maintenance personnel to discover and handle network anomalies faster and more accurately, and achieve efficient, accurate and reliable network operation and maintenance goals.

[0124] The above embodiments are only preferred embodiments of the present invention and cannot be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the present invention.

Claims

1. A network anomaly detection method based on a dual-branch model, characterized in that The following steps are involved: 1) Data preprocessing: Obtain KPI data from the communication terminal for data preprocessing, including oversampling, undersampling, and feature selection; 2) Construct and optimize the anomaly detection network: Input the preprocessed data in step 1) into the offline training module to construct the anomaly detection network and perform offline training. Input the preprocessed data into the contrast learning branch and the attention mechanism branch respectively, learn the internal association, prior association and sequence association of the data structure, and calculate the contrast loss and reconstruction loss. By jointly optimizing the multi-task loss function, calculate the anomaly score, and obtain the optimized anomaly detection network; 3) Online detection and feedback: In the online detection phase, the anomaly detection network obtained in step 2) is used to perform real-time network anomaly detection. The detection results are fed back periodically, and the anomaly data is collected into a historical database. Based on the periodic feedback data, the anomaly detection network parameters are regularly updated to optimize the detection results.

2. A network anomaly detection method based on a dual-branch model as claimed in claim 1, characterized in that In step 1), in step 1), the KPI data includes RRC connection establishment success rate, cell RLC layer downlink packet loss rate, same-frequency handover execution success rate, and wireless disconnection rate.

3. A network anomaly detection method based on a dual-branch model as claimed in claim 1, characterized in that In step 1), in step 1), the data preprocessing is used to solve the data imbalance problem caused by the rarity of anomalies. The data preprocessing balances the data set by oversampling and undersampling, and optimizes the feature set by feature selection, specifically including oversampling, clustering and undersampling, feature selection, and feature selection based on mutual information; The oversampling inserts existing abnormal class samples to create new synthetic abnormal class samples; the clustering and undersampling first uses a clustering algorithm to cluster the data, and after clustering, selects the cluster containing the largest number of samples for undersampling to balance the number of abnormal and normal samples; the feature selection calculates the dependency of the class label on each feature and removes the feature with the smallest dependency; the feature selection based on mutual information calculates the mutual information between each feature and the output class, and uses the mutual information of the feature selection technology to select features that are more relevant to the output class, selects features with high mutual information scores, and discards features with low mutual information scores; so as to improve the accuracy of the model, reduce overfitting and reduce computing costs.

4. A network anomaly detection method based on a dual-branch model as claimed in claim 1, characterized in that In step 2), the specific steps of constructing the anomaly detection network include: inputting historical data into the contrastive learning branch and the attention mechanism branch respectively, learning the internal association of the data structure through the contrastive learning branch, and calculating the contrast loss; learning the data prior association and sequence association through the attention mechanism branch, and calculating the reconstruction loss; then jointly optimizing the multi-task loss function, calculating the anomaly score, and obtaining the anomaly detection network.

5. A network anomaly detection method based on a dual-branch model as claimed in claim 1, characterized in that In step 2), the preprocessed data is input into the contrastive learning branch and the attention mechanism branch respectively, which includes: the contrastive learning branch contains two sub-networks with the same structure, both of which use multi-layer perceptrons to map the input samples to low-dimensional embeddings; the time series data measured from a single device is defined as X = {x1, x2, ..., x T }, where x i ∈R d represents the observed value at time t, and d is the dimension of the time series. The original sequence X is copied multiple times, and the multivariate time series data at each moment is sharded and combined at different positions, that is, data reorganization. The shard size k is determined according to the number of indicators, and is divided into two parts, a and b. The number of combinations is m = d-k+1. and When j=j', these two complementary combinations are a positive pair, and the rest of the combinations are negative pairs. Represents the data reorganization of input X; learns a feature space through contrastive learning, maximizes the similarity between positive samples, and minimizes the similarity between negative samples, thereby learning the association within the data structure; contrast loss L c It is calculated by the following cross entropy loss function: in, represents the shard data at the jth index position at the ith time point, m is the number of combinations after the shards are reorganized, exp() refers to the exponential function with the natural constant e as the base, ln() refers to the logarithmic function with e as the base, M(·) and N(·) represent two sub-networks for contrastive learning, and the hyperparameter τ is the temperature constant of the contrastive loss.

6. A network anomaly detection method based on a dual-branch model as claimed in claim 1, characterized in that In step 2), the preprocessed data is input into the contrastive learning branch and the attention mechanism branch respectively, which includes: since the single-branch self-attention mechanism cannot model the prior association and the sequence association at the same time, a multi-layer self-attention mechanism with a dual-branch structure is used; for the prior association, a learnable Gaussian kernel is used to calculate the prior about the relative time distance; and a learnable scale parameter σ is used for the Gaussian kernel so that the prior association can adapt to different time series patterns; Initialize the attention mechanism parameter query Q, key K, value V, σ, where σ represents the learned scale, and generates a priori associations based on the learned scale. The i-th element corresponds to the i-th time point; Specifically, for the i-th time point, its association weight with the j-th time point is calculated by the Gaussian kernel: in, The parameter matrices representing Q, K, V, and σ in the lth layer are then rescaled to transform the association weights into discrete distributions, defined as the prior association: The attention map is normalized along the last dimension using the Softmax function. Each row of Series forms a discrete distribution, and the sequence association is defined as: Through the self-attention mechanism, the association bias is defined as the symmetric KL divergence between the prior association and the sequence association. The association bias represents the information gain between the two distributions; the reconstruction loss L of the self-attention branch is defined rec Weighting for reconstruction differences and associated deviations: in, Representation sequence Reconstruction of ||·|| F and ||·|| K denote the Frobenius and K norms respectively; L denotes the number of attention layers; KL(·||·) denotes the KL divergence calculated between two discrete distributions; A p and A s They represent prior association and serial association respectively; θ is used to weigh the loss term, and when θ>0, the association deviation is amplified.

7. A network anomaly detection method based on a dual-branch model as claimed in claim 1, characterized in that In step 2), the joint optimization of the multi-task loss function specifically includes: defining the joint loss Loss as: Loss=λ×L rec +(1-λ)L c In the formula, λ represents the weight parameter; L rec represents the reconstruction loss of the attention branch; L c represents the contrast loss of the contrastive learning branch; the joint optimization of the multi-task loss function Loss is used to balance the joint training process of the self-attention network and the contrastive learning network to avoid a certain loss being too large and having a dominant influence; the designed dual-branch anomaly detection network is trained by minimizing the loss function Loss; among them, the weight parameter λ needs to be adjusted according to different data sets. In order to use the hyperparameter optimization framework for weight parameter optimization, the hyperparameter optimization framework provides a wealth of optimization algorithms and search strategies, supports parallel evaluation, early stopping, and result visualization functions, and can search for the optimal hyperparameter combination; (1) Define the hyperparameter space, that is, the range of values ​​that each hyperparameter can take or the list of optional values, and define the continuous space Λ (0,1) = {λ1,…,λ n }; Initialize the Study object to store and manage all information in the optimization process, including historical experiments and optimal parameters; (2) Define the objective function, a set of hyperparameters as input, and return a model's F1 score on the validation set: f(Λ)→F1 Where f is the objective function and Λ is the hyperparameter configuration; (3) Run the optimization algorithm to search for the optimal solution in the hyperparameter space; in each iteration, the optimization algorithm generates a new hyperparameter configuration and passes it to the objective function for evaluation; Λ t =optimize_algorithm(X,f,t) Among them, Λ t is the hyperparameter configuration in the tth iteration, optimize_algorithm is the optimization algorithm; for each newly generated hyperparameter configuration, the framework runs the objective function for evaluation; this usually involves training a machine learning model with the given hyperparameter configuration and evaluating its performance on the validation set; the results of each iteration are stored in the Study object, including the hyperparameter configuration and the score; the framework uses this information to guide the subsequent search process; (4) After the optimization process is completed, the optimal hyperparameter configuration λ is obtained from the Study object.

8. A network anomaly detection method based on a dual-branch model as claimed in claim 1, characterized in that In step 2), calculating the anomaly score specifically includes: designing an anomaly score as an anomaly standard. Normal data points have a strong correlation, so the distance between the positive example pairs obtained by contrastive learning is smaller than the negative example pairs. The anomaly score is defined as follows: Among them, Softmax refers to the normalized exponential function, and ⊙ represents element-by-element multiplication; Representation sequence Reconstruction of ||·|| F represents the Frobenius norm; L represents the number of attention layers; KL(·||·) represents the KL divergence calculated between two discrete distributions; A p and A s represent prior association and serial association respectively; L c represents contrast loss, i.e., cross entropy loss function; Based on the anomaly score, a hyperparameter threshold δ is used to decide whether a point is anomaly (y=1) or non-anomalous (y=0); if the score exceeds the threshold, the output y is anomaly, that is: Among them, y i represents the model output at the i-th time point, represents the model input at the i-th time point, It represents the anomaly score at the i-th time point, and the hyperparameter threshold δ is set by the operation and maintenance personnel according to different scenario data.

9. A network anomaly detection method based on a dual-branch model as claimed in claim 1, characterized in that In step 3), the online detection phase uses the anomaly detection network obtained in step 2) to perform real-time network anomaly detection, specifically including: The network KPI data is input into the trained anomaly detection network in real time. When a network anomaly is detected, an anomaly alarm is output. The operation and maintenance personnel determine whether the detected network anomaly is a real anomaly. If it is a real network anomaly, the KPI data at this moment is collected into the historical database, and the anomaly detection network parameters are updated periodically.

10. A network anomaly detection device based on dual-branch time series, characterized in that It is equipped with a data acquisition module, an offline training module, a real-time detection module, and a feedback module in sequence; The data acquisition module is used to acquire communication terminal performance time series data and model feedback data; The offline training module is used to build an anomaly detection model through the historical data of the data acquisition module and periodically update the anomaly detection model; The real-time detection module is used to detect anomalies online; The feedback module collects abnormal data detected by the real-time detection module, and inputs the abnormal data into the data acquisition module to expand the database after the operation and maintenance personnel confirm the abnormality.

Images